Evaluating organizational stress-management interventions using ...

More documents

Recommendations

Info

26 RANDALL, GRIFFITHS, COX Bohle, & Quinlan, 2000; Saksvik, Nytro, Dahl-Jorgensen, & Mikkelsen, 2002). This process evaluation can then be used to adapt outcome evaluation by using it to determine whether each participant is more appropriately placed in a intervention/exposed or control/not exposed group. Measured exposure patterns can thus be exploited as an evaluation design variable. This approach is different to that used in ‘‘natural experiments’’ where exposure patterns are predictable and controlled (see Jackson, 1983). Rather, it is a constructive use of the manipulation check: The study design is adapted to reflect actual exposure patterns. Treating uncontrolled and unpredictable exposure patterns as a natural manipulation may help to expand the size of the ‘‘pool’’ of organizational intervention evaluation research and make informative evaluation possible in chaotic organizational settings. THE PRESENT STUDY This article presents two empirical studies drawn from the authors’ research on risk management and work-related stress (Cox et al., 2000a; Cox, Randall, & Griffiths, 2002b). Together they illustrate that through a simple exploration of the intervention processes, exposure patterns can be identified and used to adapt the final study design and analysis. This achieves two things: It measures actual exposure to the intervention to allow a valid evaluation of its effectiveness (thus controlling Type III error), and it permits informative evaluation where exposure patterns cannot be planned or tightly controlled. Both studies examine, ad hoc, the actual ‘‘organizational penetration’’ (Cox et al., 2000a) of an intervention into a participant group. This strategy enables evaluation without the need to (1) make possibly untenable assumptions about stable intervention exposure patterns, (2) to control exposure to an intervention, or (3) to rely on events or organizational structures to coincidentally support predictable and stable exposure patterns. Both studies have one central hypothesis: The intervention effect (improved well-being) will only be apparent when measures of actual intervention exposure are used to partition the participant group prior to analysis (i.e., actual intervention exposure is a moderator of change). METHOD Design In both studies, the interventions were designed by stakeholders from within the participating organizations, through the feedback of risk assessment data and discussions facilitated by the research team. These discussions were
EVALUATION OF INTERVENTIONS 27 based on an initial problem analysis carried out using a risk assessment questionnaire survey of working conditions. Both studies began with a simple and traditional pre (Time 1) – post (Time 2) longitudinal intervention design. Because of the constraints operating within the organizations involved it was not possible to allocate participants to ‘‘intervention’’ and ‘‘control’’ groups: each intervention was intended to reach all study participants. In both studies, participants were asked to report on their awareness of (Study 1) or their involvement in (Study 2) the intervention. In both studies the dependent variable was self-reported well-being (measured in terms of levels of exhaustion). Time 2 measures of well-being and intervention exposure were taken 18 months after the Time 1 measures of well-being. The evaluation design was adapted by partitioning the participants groups according to their reported exposure to the intervention (using a Measured exposure 6 Time interaction term) in a repeated measures analysis of covariance (ANCOVA). Participants and interventions Study 1: Railway staff. Thirty-seven station managers from a railway transport company provided data at both Time 1 and Time 2. This represented a response rate of approximately 50% (based on the number of supervisors providing both Time 1 and Time 2 data). Response rates for Times 1 and 2, taken separately were higher (68% and 64%). All were male with an average age of 40 years (SD = 8.0) and average tenure of 14 years (SD = 7.1) at Time 1. The supervisors worked at different sized stations: Some stations were larger and busier than others. Inspection of company records indicated that the supervisors returning questionnaires came from a representative sample of stations and that they did not differ from the whole station manager population in terms of average age or length of service. Some months before Time 1, a central part of the station managers’ role had been changed. Because of budgetary constraints, their responsibility for managing the repair of faulty station equipment (including reporting faults and authorizing and managing repairs 2 ) had been removed. At Time 1 senior managers, not supervisors, managed equipment repair. However, when the risk assessment data collected at Time 1 was reported, the organization interpreted these as indicating that the removal of roles and responsibilities had not been well received by staff. In an attempt to improve staff satisfaction and well-being, responsibility for managing the repair of faulty station equipment was returned to station supervisors (i.e., super- 2 This does not refer to track and trains, but rather to equipment found in the station building. Inspection and repair of track equipment was carried out by other specialist members of staff.
Page 1 and 2: EUROPEAN JOURNAL OF WORK AND ORGANI
Page 3: EVALUATION OF INTERVENTIONS 25 patt
Page 7 and 8: EVALUATION OF INTERVENTIONS 29 expe
Page 9 and 10: EVALUATION OF INTERVENTIONS 31 F(1,
Page 11 and 12: EVALUATION OF INTERVENTIONS 33 TABL
Page 13 and 14: EVALUATION OF INTERVENTIONS 35 Figu
Page 15 and 16: EVALUATION OF INTERVENTIONS 37 inte
Page 17 and 18: EVALUATION OF INTERVENTIONS 39 Camp
Page 19: EVALUATION OF INTERVENTIONS 41 Saks

Evaluating organizational stress-management interventions using ...

Create successful ePaper yourself

Delete template?

Save as template?