Document - european association of national research facilities

Maintenance and Reliability Workshop
9-10 November 2011
Synchrotron SOLEIL, Saint-Aubin, France
Context
One of the main tools used by large scale research facilities in maintaining the thousands of
items of equipment installed and deployed on site and to ensure that this equipment operates
at a very high level of reliability is a Computer-based Maintenance Management System
(CMMS):
- by the Maintenance groups to manage all the equipment in their databases (from
receipt of parts and stock control through to installation and renewal);
- by the Buildings and Infrastructure groups to manage almost all preventive
maintenance operations and regulatory controls;
- and by several other groups (e.g. Vacuum, fluids, security, etc) primarily during
machine downtime;
- by a large number of support laboratories (e.g. Chemistry, Biology, Surfaces labs) are
starting to enter their own equipment in the CMMS.
All these issues surrounding equipment maintenance, i.e. all the aspects that contribute to
maintaining equipment in optimal operating conditions, are in fact common to all major
research facilities in Europe and a remarkable success-key of performance for the whole
facility.
For this reason, the CMMS working group of SOLEIL decided to organize a workshop at
SOLEIL for benchmarking the experience of several facilities in Europe. The workshop has
been focused on two topics, Maintenance and Reliability, with the aim of exchanging and
sharing our experience with maintenance management tools, organizational requirements,
working methods, difficulties encountered, and so on.
1. Presentations
SOLEIL: Presentation of the SOLEIL’s CMMS tool (Hélène Rozelot)
Hélène Rozelot reviewed the tool’s main features (purchase requests, inventory, equipment
location, history of movements, configuration management, traceability of work orders, stock
management, and so on) and presented a few indicators extracted from the database using
Business Objects (BO). She also looked back at the context in which the system was set up
between 2002 and 2004. Lastly, she explained the organization set up to deploy the CMMS
and highlighted the difficulties encountered, which stemmed primarily from the introduction of
new working methods and a new formalism to which SOLEIL’s teams were unaccustomed.
MAX IV: Maintenance at the MAX IV laboratory (A. Månsson and J. Thånell)
MAXLab may be described as a laboratory that has grown almost “organically”. This means
that, right from the outset, maintenance has always been seen as one of many activities
conducted by a small group of scientists and research engineers in charge of designing and
building the laboratory. On the one hand, this reinforced the commitment of each person
involved in maintenance, but on the other hand, as the laboratory has grown and the MAX IV
project has got under way, the teams are now working with a maintenance system that is
unsuitable for an organization of over 100 people.

ALBA: Maintenance at ALBA Synchrotron (L. Miralles)
Lluis Miralles presented the organization of ALBA’s “Technical Support” division. The division
is split into four “sections”: Infrastructure, R&D, Vacuum and Transversal. The group
responsible for all maintenance at ALBA is in the Transversal section, and comprises only six
technicians. Note that the Experiments division does not have any dedicated technicians.
Maintenance operations are subcontracted, but maintenance management (scheduling of
operations and so on) is handled in-house. Lluis Miralles described the features of the
PRISMA 3 software used. This program features many of the typical functions of a CMMS,
including topologies, stock, process workflow and statistics. He concluded with a
presentation of conditional maintenance based on monitoring a number of parameters (pump
pressures, motor vibrations, and so on) to trigger alerts that result in either preventive
maintenance actions or warnings about potential breakages (predictive maintenance).
SOLEIL: Maintenance & Reliability for the Process (P. Betinelli & L. Manciet)
Pascale Betinelli started by explaining what SOLEIL wants from a CMMS and how it is used
in practice. MAINTIMEDIA is an assembly of several modules (topology, stock, purchasing,
process workflow and statistics). While each of the modules can be used independently, this
approach is somewhat restrictive. The modules are in fact interconnected and the CMMS
should be used as widely as possible to make the best use of the system. This implies that
users must thoroughly understand and analyze their maintenance processes in order to
make the best use of the CMMS. Pascale then displayed some data extracted with BO and
showed that the use of the CMMS as a maintenance tool is very variable. She also
mentioned other tracking tools used at SOLEIL, including MANTIS bug tracker, Excel, ELOG
and MS Project. She emphasised the difficulty of imposing the use of a single tool or even
defining rules for using each tool.
Laurent Manciet then presented some more technical aspects of MAINTIMEDIA, including
the Oracle database, the number of licences, the access rights defined, network architecture,
and the technical problems encountered in particular last year (primarily slow database
access). In their joint conclusion, Pascale and Laurent highlighted the general difficulties in
using a CMMS, a program with complex functions that require users to define their
requirements and processes properly beforehand. For this new modus operandi to be
successful, teams must overcome their resistance to change.
DIAMOND: Buildings & utilities maintenance – Contracted service or in-house staff (L.
Walters)
Lee Walters presented the organization of DIAMOND’s Support division, which includes the
group responsible for infrastructure maintenance. He explained that maintenance operations,
which had previously been contracted out, are now managed in-house, having fired the
former contractors on site. He reviewed this situation, which has the advantage of enabling
maintenance teams to be managed directly and makes the most of in-house resources.
However, it requires considerable effort to set up, and also offers less flexibility in terms of
available resources, in that external staff are required to handle peak loads.
CERN: Maintenance for CERN’s cryogenic installations (S. Knoops)
Sigrid Knoops presented the cryogenic equipment in use on the LHC, noting that the devices
are prototypes designed by CERN. She reviewed how maintenance is organised, and who
does what, depending on the complexity of operations:
- Straightforward adjustments (e.g. setting a parameter) are handled by CERN in-house
operators. Likewise, much more complex operations that require a redesign or that involve
manufacturing new parts are handled in-house.
- Conversely, standard operations (repairs, replacements, corrective and preventive
maintenance) are contracted out.
Her slides also included various operating statistics. INFORM EAM asset management
software is used (http://www.infor.com/solutions/eam/). Data are extracted with Business

Objects, and the database is linked to the technical documentation system (Engineering Data
Management System or EDMS).
LASER MEGAJOULE: Maintenance at the facility (H. Graillot & I. Granet)
Alongside synchrotrons, LMJ is a representative of another class of major research
infrastructure (lasers). Although the infrastructure is still under construction, maintenancerelated
aspects are already being taken into account. Almost 100,000 items of equipment
have already been entered in the INFOR EAM database, ranging from ordinary devices to
more complex systems such as the “target chamber“ with its 10-metre diameter. Hervé
Graillot described the tool’s features (topology, stock, purchasing, process workflow and
statistics) and explained that the CMMS is interconnected with the JIRA project tracking tool
and the SIROCO technical database. Business Objects is also used to extract data which is
analyzed to optimise maintenance procedures. In conclusion, Hervé noted that the CMMS as
a whole (including purchasing, work order tracking and so on) is used throughout LMJ. He
also noted that LIL, the Laser Integration Line used as a prototype for Laser MegaJoule, had
used a different CMMS that its operators had found too complex to be workable.
SOLEIL: Buildings & Utilities Maintenance (T. Didier)
Thierry Didier, in charge of utilities at SOLEIL, presented the activities managed using the
CMMS (general services, fluids and electricity), for monitoring and maintenance operations in
particular. Certain preventive maintenance operations are mandated by regulations. This
preventive maintenance entails drawing up work instructions, from which work orders are
generated so as to schedule the group’s activities and required resources. Nonconformity
sheets are drawn up where applicable. Thierry then presented the data extracted using BO
and used for management purposes by team leaders. Lastly, he mentioned the motivational
difficulties encountered and the steps taken to resolve them, including simplifying the
technical data entered in the database, using work instructions suited to the group, and
making extensive use of Business Objects.
ESRF: Maintenance of Technical Infrastructures (T. Marchial)
Thierry Marchial briefly presented the activities of the 30-strong Buildings and Infrastructure
group, part of ESRF’s Technical Infrastructure Division. He described the equipment
managed in the CMMS (using CARL Master software). This equipment is classified in 3
levels:
- Equipment that could cause beam losses (accelerator and source)
- Equipment that could affect users (beamlines and mains power)
- Other equipment.
He then reviewed the outsourced maintenance contracts managed in the CMMS (which do
not include “primary maintenance” or regulatory inspections). These account for:
- 33 documents
- 4-year contracts, revisable annually
- 14,000 hours of work on site in 2011.
Contracts are managed using the Plan-Do-Check-Act (PDCA) quality system. Each incident
is analyzed by the subcontractor and the Buildings and Infrastructure group, and corrections
are made:
- Identification of expected performance,
- Quality control and tracking (in particular through indicators).
A bonus/penalty system has also been set up for subcontractors.
Orsay Proton Therapy Centre: Maintenance and reliability (S. Meyroneinc)
Samuel Meyroneinc briefly presented the principles of proton therapy and the facilities at the
Orsay Proton Therapy Centre (CPO). He emphasized the operational constraints (medical
environment) which restrict maintenance to scheduled slots, and the need for a decisionmaking
structure to analyze the potential impacts of maintenance work and how treatment
results may be affected. He explained how reliability and maintenance can result in

incompatibilities with R&D: when maintenance and upgrades affect several fields of
expertise, it is difficult to guarantee that the results will comply with specifications (and be on
schedule and within budget). With more recent technologies, preventive maintenance is
necessarily limited and dependent on experts.
Maintenance at CERN: On the brink of a new maintenance project (G. Perinić)
Goran Perinić briefly presented the LHC and detectors. He emphasized the fact that even a
minor incident on a device during operation can require the entire accelerator to be shut
down, and cited examples where this has occurred. Accordingly, preventive maintenance
activities are given increasing importance. Emergency repairs must therefore be
distinguished from maintenance operations, and the latter must be taken into account
properly to improve operation. He also explained the link between the maintenance tool,
INFORM EAM, and the EDMS technical documentation system. CERN is currently setting up
a new cross-department maintenance project, with the aim of reducing the frequency of
scheduled technical downtime from once every six weeks to once every twelve weeks. With
this in mind, an audit was conducted (based on a questionnaire sent to support groups in
order to assess their approach to maintenance), which showed the diversity of maintenance
methods and tools used by support groups. The aim is now to define a modus operandi
(methods, tools, indicators) common to all these groups. A dedicated organization and staff
have been set up to complete this project within two years. A CMMS working group has been
formed to define the common modus operandi, which an expert group will help to transpose
into operational procedures.
2. Round tables
FIRST ROUND TABLE: What are the necessary resources and organization for an effective
maintenance management system?
First discussion on this topic: “centralised” or “distributed” maintenance?
If maintenance is distributed between several groups or departments, there is a risk of
diluting responsibilities, duplicate work, different methodologies and tools, resulting in lower
efficiency. Centralised maintenance appears preferable, but should be considered as a
process in its own right
Second discussion on this topic: who is responsible for maintenance?
For the majority of participants, the owner of the equipment is responsible for the function
that it provides and should therefore bear responsibility for maintaining it. The maintenance
team, in turn, is responsible for performing the maintenance requested by the person
responsible for the equipment. It is also the maintenance team’s responsibility to propose
equipment improvements to the development department.
Third discussion on the topic of common tools.
Some participants believe there is no need to use the same tool in all areas of activity.
CERN, on the other hand, is working to enhance the reliability of its facilities, and is moving
to a single common tool for all tracking.
Fourth discussion on the topic of rights management
Most participants feel that topology management rights (creation / deletion) should be
restricted in order to minimise accidental errors and ensure consistency. CERN does not
restrict user rights to any great extent and counts on training and understanding of the
system.

SECOND ROUND TABLE: What common indicators could research infrastructures set up
and share?
First discussion: motivation of teams involved in maintenance (who are also involved in
R&D work which is generally felt to be more rewarding).
• Remark on financial aspects: it is possible to have two separate teams (R&D and
maintenance) provided sufficient resources are available.
• For complex equipment, the teams must have well-honed skills Accordingly it is important
to involve them in development as well as maintenance.
• Training and educational aspects are important: the value of maintenance work must be
highlighted to show that it contributes to enhancing reliability.
• Senior management support is essential. The participants noted that defining indicators is a
way of providing recognition that maintenance has an important role to play in a facility’s
performance. Indicators such as those set up by CERN are a motivating factor for staff.
The bonus/penalty system set up by ESRF was mentioned again, though it was pointed out
that the system is difficult to implement on an in-house basis.
Second discussion: what should be the ratio of preventive maintenance to corrective
maintenance?
This ratio has consequences on the financial cost and the cost in terms of resources. How
can this be quantified? The question remains open.
Third discussion on this topic: Indicators
The participants noted that a number of indicators are defined by standards (MTBF,
equipment availability, EN 15341, EN 1306, etc). While it is easy to implement such
Indicators in some fields (infrastructure, accelerators, etc.), they are not easily accepted by
scientists for whom publications are the key indicator.
3. Miscellaneous
ESRF proposes to make its technical documentation relating to maintenance contracts
available to the participants. DESY would like feedback about the CMMS software used by
the various institutes. CERN proposes to share its indicators.
4. Conclusions
The workshop has highlighted the importance of maintenance, although there are still many
open issues. Various points have been revealed:
• CMMS software is relatively widely used in departments which outsource part of their
activities and which have contractual relations with subcontractors; however, it is
much harder to impose the use of a CMMS “internally” (within a department or a
research body, for example).
• Senior management commitment is essential to ensuring CMMS use.
• Most of the organizations in attendance have IT support personnel dedicated to
setting up the CMMS (and responsible in particular for defining and administering
user access rights to topologies)
• The participants proposed to review in a year’s time all the questions that arose;
• CERN is volunteer for hosting the next workshop in 2012; The intention is to hold
more round table sessions in parallel next year, so that all issues can be discussed

Participants
Institute Surname Name
ALBA CAMPS Antoni
ALBA CARLES David
ALBA MANOTAS Juan Jose
ALBA MATILLA Oscar
ALBA MIRALLES Lluis
ALBA NARDELLA Alberto
ALBA SALVAR Daniel
CEA/LMJ GRAILLOT Herve
CEA/LMJ GRANET Isabelle
CERN KNOOPS Sigrid
CERN MUCNIER Christophe
CERN PERINIĆ Goran
CERN PILON Regis
CPO MEYRONEINC Samuel
DESY HATJE Jan
DESY KUDLA Ignacy
DESY PENNING Joerg
DESY WEDDIG Henning
DIAMOND WALTERS Lee
ESRF FAVIER Frederic
ESRF MARCHIAL Thierry
ESRF ROUX-BUISSON Pascal
IPNO BZYL Harold
MAXLAB MANSSON Anders
MAXLAB THANEL Johan
SOLEIL BETINELLI Pascale
SOLEIL CORTES Bruno
SOLEIL DELETOILLE Xavier
SOLEIL DIDIER Thierry
SOLEIL GAGEY Brigitte
SOLEIL HERBEAUX Christian
SOLEIL MANCIET Laurent
SOLEIL ROZELOT Helene