atw - International Journal for Nuclear Power | 10.2020

More documents

Info

atw Vol. 65 (2020) | Issue 10 ı October ENVIRONMENT AND SAFETY 502 | Fig. 2. Observation tool to structure the data collection and directly document the results during the Integrated Final Control Room System Validation (ISV). every 20 minutes and the self assessment of workload was performed immediately without stopping the scenario. 4 Observation Tool A special observation tool was developed to structure the data collection and directly document the results during the ISV. This observation tool was a printed document in book form in DIN A3 format. For the observation always the left and right sides were used simultaneously. The left page of the tool is divided into three parts (see Figure 2). In the upper part of the left page there is a table for the recording of human errors. Below the human error table is as second table in which workload measurements is to reported with simulator time and DIM executed. On the lower part of the left side is a graphical presentation (flow chart) of DIMs, which contains the correct path (marked in red). The graphic contains additional markers for additional tasks: markers for simulation freezes (measurement of situation awareness), markers for task performance key nodes and markers for communication points. On the right page, the tasks (DIMs) of the different crew members (TO, RO, SSV) and all other measurements are noted. The DIMs are marked in different colours: yellow for the SSV, green for the RO and blue for the TO. Additional measurements of situation awareness and task performance are marked in grey. For situation awareness the column “comment/content” shows what is expected to be included in the communication of the crew. The observers fill in whether these expectations were met. The same applies to task performance, where the observer rate whether defined important tasks have been completed. The communication is also rated in the tool. The simulator time should be entered for each observation. 5 Experiences 5.1 Methodological approach Already in the conception of the study, an attempt was made to avoid subjective evaluations of behaviour as far as possible in order to make the results robust against subjective influences. This concept has proved highly successful in the subsequent evaluation, as a large part of the data was collected quantitatively and was therefore easy to evaluate. For the important variables, (task performance, situation awareness, communication) the evaluation was based on several different methods. This significantly increased the validity of the assessment. In all subjective measurements, several experts always assessed the same aspects. In the event of any discrepancies between them, these were discussed (e.g. during debriefing) in order to produce a uniform picture. The integrated observation tool was used to document the measurements which were not recorded by tablet computer. The tool was extremely useful for the execution and evaluation of the ISV and for the observers’ tasks. With the help of the observation tool, the course of the scenario could be followed and it was used to identify the measuring points and to document the measurement results immediately in writing. After a scenario trial the tool was used to structure the debriefing. For this purpose, all pages of the tool were systematically reviewed, the notes of the individual observers were queried and these results were documented in the observation tools of the HFC observers. Following the test execution in the ISV, the tool was used as test documentation for evaluation. Here, too, the great usefulness of the tool was demonstrated, as all observation results are clearly summarized in one document. 5.2 Validity Investigations with behavioural observation of this kind depend on the simulation being as realistic as possible. Only then conclusions can be drawn from the behaviour in the test situation to the behaviour in real situations. With two exceptions, the behaviour of the crews showed no signs that a realistic simulation would not have been successful. In scenario C the crew is led by a phone call to the decision to evacuate the main control room (MCR) and switch to the remote shutdown station (RSS). During the trials of scenario C there were some difficulties to induce a common understanding of the dangerousness of the situation. The crews interpreted the situation more or less problematically and therefore wanted to leave the MCR in some cases very quickly or not at all. An additional limiting factor to validity could be the realization of the move to the RSS, because the evacuation of the MCR and the move to the RSS were not simulated realistically. Although both rooms are present in the full scope simulator (FSS), their spatial arrangement does not correspond to reality (in the FSS, the RSS is right next to the MCR, with direct access from the MCR; in reality, the locations are separated). The move to RSS was simulated mainly by the shift waiting in front of the FSS door. Compliance with certain standards for the evacuation of the MCR was a key node in this scenario. The unrealistic Environment and Safety Are They Ready for Operation? How to Assess the Control Room System of a New NPP ı Rainer Miller, Rodney Leitner, Sina Gierig and Harald Kolrep
atw Vol. 65 (2020) | Issue 10 ı October test environment for the evacuation considering spatial arrangements as well as lack of certain plant systems and equipment could have facilitated some of the key node failures which appeared in scenario C. But how strongly the failures and errors observed in scenario C is related to the limited realism of the move to the RSS can only be assumed. It became apparent in the ISV that the members of the shift teams were sometimes unclear about what behaviour was expected of them. One possible limitation of the validity of the ISV is that it is not certain for every observed behaviour whether this behaviour would be shown in the same way outside the test situation of the ISV. 5.3 Measurement of situation awareness In retrospect, the measurement of situation awareness turned out to be the measurement with the most problems. One reason for the problems was the simple question of what time frame to think of when we ask in the situation awareness questionnaire “when the current task will be completed, what will be the next task?” Does this question refer to a period of 1 minute, 10 minutes, 1 hour? This was interpreted differently by the crew members, which affected the content of the answers. It became apparent that it is absolutely necessary to clarify such questions clearly in the instruction before the test starts. Furthermore, the questions regarding the ‘next task’ were answered very differently in same cases. On the one hand on a very low level of detail (“power increase up to 5 %”), on the other hand on a very high level of detail (“start LAC14 AP001”). Both answers were correct, but it was difficult to assess whether the different operators really had an identical understanding of their tasks. Another critical point in the measurements was the exactly identical position of the simulation freezes in the respective trials. If the freeze is only slightly shifted on the time axis, then this can influence the question of the relevant parameters and process aspects and their assumed course. specifically for ISV has shown to be very successful. The tool for observation and documentation, developed especially for ISVs, has proven to be very successful. Especially the approach to capture as many assessments as possible directly and synchronously was very feasible and highly efficient. Therefore the necessity to use audio or video recordings was minimized. The study also showed that – especially for the survey of situation awareness - careful preparation is necessary to achieve reliable results. This includes, in particular, the provision of clear written instructions for the participants. References [1] IEC 1771 (1995). Nuclear Power Plants – Main Control Room – Verification and Validation of Design. IEC, Geneva. [2] IEC 60964 (1989). Design of Nuclear Power Plants. IEC, Geneva. [3] YVL Guide 5.5 (2002). Instrumentation Systems and Components at Nuclear Facilities. [4] NUREG 0711 (2912). Human Factors Engineering Program Review Model – Rev. 3. NUREG, Brookhaven NY. [5] Endsley, M. R. (1995). Measurement of Situation Awareness in Dynamic Systems. Human Factors, 37 (1), 65-84. [6] Roscoe, A. & Ellis, G. (1990). A Subjective Rating Scale for Assessing Pilot Workload in Flight: A Decade of Practical Use. Royal Aerospace Establishment, Farnborough. Authors Rainer Miller miller@mto-safety.de MTO Safety GmbH Gethsemanestr. 4 10437 Berlin, Germany Dr. Rodney Leitner Sina Gierig Dr. Harald Kolrep HFC Human-Factors-Consult GmbH Köpenicker Str. 325 12555 Berlin, Germany ENVIRONMENT AND SAFETY 503 6 Conclusion Apart from a few detailed problems in recording situation awareness the combination of methods developed Environment and Safety Are They Ready for Operation? How to Assess the Control Room System of a New NPP ı Rainer Miller, Rodney Leitner, Sina Gierig and Harald Kolrep
Page 1 and 2: nucmag.com 2020 10 ISSN · 1431-525
Page 3 and 4: atw Vol. 65 (2020) | Issue 10 ı Oc
Page 5 and 6: Kommunikation und Training für Ker
Page 37: atw Vol. 65 (2020) | Issue 10 ı Oc
Page 67 and 68: Kommunikation und Training für Ker

atw - International Journal for Nuclear Power | 10.2020

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?