View - HEPHY

6 Test Validation Using Cosmic EventsThe TestCell software was deployed to the CMS environment in September 2009,after an initial development period in Vienna. It quickly proved useful in verifyingpreviously known hardware defects and discovering new ones. Since the experimentwas at the time taking data from cosmic radiation, the interesting opportunity tolook for the consequences of undetected errors exists. This section will discuss someexamples and their consequences.6.1 Data/Emulator ComparisonThe primary tool for after-the-fact evaluation of the consequences of electronics errorsis the comparison between the results of the hardware - as recorded by the DAQreadout - and a reprocessing of the input values with the GT emulator.The process setup for the comparison between data and emulator results is verysimilar to the one used for pattern generation in Sec. 5.2.1. The setup of the CMSSWframework for this purpose is shown in Fig. 6.1. The two readout records obtainedfrom the data file and the emulator are compared by a framework analyzer, whichproduces a number of histograms on disagreements and errors encountered.Limits of data/emulator comparison for trigger decisionsWhile this form of validation is a very powerful tool, it is not perfect. First, there isthe obvious bias inherent in looking only at data recorded by the experiment - eventsthat were erroneously missed by the trigger will never make it to the input dataset andwill therefore not be noticed. In practice, this is mitigated by the fact that there areusually multiple independent trigger criteria, so that some subsets of the errors willshow up because algorithms not affected by the malfunction will still select the event.Second, due to the fact that the readout only contains the algorithm decision wordafter prescaling has been applied, it is difficult to interpret disagreements on prescaledalgorithms. Running the trigger emulator with prescaling disabled allows an error tobe flagged when the emulator does not match an emulator and the hardware does, butfor the converse, it is impossible to distinguish locally whether there is a malfunctionor the prescaling suppressed the bit. Even statistical interpretations are complicatedby the interaction of multiple prescalers and trigger masks.70