atw - International Journal for Nuclear Power | 10.2020

atw Vol. 65 (2020) | Issue 10 ı October
ENVIRONMENT AND SAFETY 502
| Fig. 2.
Observation tool to structure the data collection and directly document the results during the Integrated Final Control Room System
Validation (ISV).
every 20 minutes and the self assessment
of workload was performed
immediately without stopping the
scenario.
4 Observation Tool
A special observation tool was developed
to structure the data collection
and directly document the results
during the ISV. This observation tool
was a printed document in book form
in DIN A3 format. For the observation
always the left and right sides were
used simultaneously.
The left page of the tool is divided
into three parts (see Figure 2). In the
upper part of the left page there is a
table for the recording of human
errors. Below the human error table is
as second table in which workload
measurements is to reported with
simulator time and DIM executed. On
the lower part of the left side is a
graphical presentation (flow chart) of
DIMs, which contains the correct path
(marked in red). The graphic contains
additional markers for additional
tasks: markers for simulation freezes
(measurement of situation awareness),
markers for task performance
key nodes and markers for communication
points.
On the right page, the tasks (DIMs)
of the different crew members (TO,
RO, SSV) and all other measurements
are noted. The DIMs are marked in different
colours: yellow for the SSV,
green for the RO and blue for the TO.
Additional measurements of situation
awareness and task performance are
marked in grey. For situation awareness
the column “comment/content”
shows what is expected to be included
in the communication of the crew. The
observers fill in whether these expectations
were met. The same applies to
task performance, where the observer
rate whether defined important tasks
have been completed. The communication
is also rated in the tool. The
simulator time should be entered for
each observation.
5 Experiences
5.1 Methodological approach
Already in the conception of the
study, an attempt was made to avoid
subjective evaluations of behaviour as
far as possible in order to make the
results robust against subjective influences.
This concept has proved highly
successful in the subsequent evaluation,
as a large part of the data was
collected quantitatively and was
therefore easy to evaluate. For the
important variables, (task performance,
situation awareness, communication)
the evaluation was based on
several different methods. This significantly
increased the validity of the
assessment.
In all subjective measurements,
several experts always assessed the
same aspects. In the event of any discrepancies
between them, these were
discussed (e.g. during debriefing) in
order to produce a uniform picture.
The integrated observation tool
was used to document the measurements
which were not recorded
by tablet computer. The tool was
extremely useful for the execution
and evaluation of the ISV and for the
observers’ tasks. With the help of the
observation tool, the course of the
scenario could be followed and it was
used to identify the measuring points
and to document the measurement
results immediately in writing. After
a scenario trial the tool was used
to structure the debriefing. For this
purpose, all pages of the tool were
systematically reviewed, the notes of
the individual observers were queried
and these results were documented
in the observation tools of the HFC
observers. Following the test execution
in the ISV, the tool was used as
test documentation for evaluation.
Here, too, the great usefulness of the
tool was demonstrated, as all observation
results are clearly summarized
in one document.
5.2 Validity
Investigations with behavioural observation
of this kind depend on the
simulation being as realistic as
possible. Only then conclusions can be
drawn from the behaviour in the test
situation to the behaviour in real
situations. With two exceptions, the
behaviour of the crews showed no
signs that a realistic simulation would
not have been successful.
In scenario C the crew is led by a
phone call to the decision to evacuate
the main control room (MCR) and
switch to the remote shutdown station
(RSS). During the trials of scenario C
there were some difficulties to induce
a common understanding of the
dangerousness of the situation. The
crews interpreted the situation more
or less problematically and therefore
wanted to leave the MCR in some
cases very quickly or not at all. An
additional limiting factor to validity
could be the realization of the move
to the RSS, because the evacuation
of the MCR and the move to the
RSS were not simulated realistically.
Although both rooms are present in
the full scope simulator (FSS), their
spatial arrangement does not correspond
to reality (in the FSS, the RSS
is right next to the MCR, with direct
access from the MCR; in reality, the
locations are separated). The move to
RSS was simulated mainly by the shift
waiting in front of the FSS door.
Compliance with certain standards for
the evacuation of the MCR was a key
node in this scenario. The unrealistic
Environment and Safety
Are They Ready for Operation? How to Assess the Control Room System of a New NPP ı Rainer Miller, Rodney Leitner, Sina Gierig and Harald Kolrep