Muon Identification Using Decision Trees

BABAR Analysis Document #1853, version 2
Muon Identification Using Decision Trees
C. O. Vuosalo, 1 A. V. Telnov, 2 K. T. Flood 1
1 University of Wisconsin, Madison
2 Princeton University
September 2, 2008
Abstract
The development, performance and implementation of a muon identification selector based on a
decision tree algorithm is presented and compared with the previous neural net selector.

Contents
1 Introduction 3
2 The Muon Detector (IFR) 3
3 Variables for Muon Identification 4
4 Bagger Decision Trees 5
5 Control Samples 7
6 Training and Validation 13
7 Muon BDT Selectors in ROOT 14
8 Performance of Muon BDT Selectors 15
9 Implementation in BetaPid 19
2

1 Introduction
The momentum spectrum of muons from generic Υ (4S) → BB decays is shown in Fig. 1. Highperformance
muon identification is important in many of the physics analyses at BABAR; its uses
include B flavor tagging, B 0 − Bzb mixing, reconstruction of semileptonic decays of B and cascade
D mesons, which are the primary source of muons at this center-of-mass energy, reconstruction of
the J/ψ resonance in the J/ψ → µ + µ − channel, study of the b → sµ + µ − decays, etc.
In this context, it is very important to have a good muon identification system in terms of
the hardware (detector) and the software algorithm. A brief description of the muon detector is
provided in the next section. Up to now, two types of muon identification algorithms were being
used for the data analysis in BABAR, the Cut-based muon selector [1] and the Neural Network
selector [5].
This note reports on the development of a new muon selector based on a Bagger Decision
Tree (BDT) algorithm. Following a brief description of the IFR and the existing muon selectors,
the functionality, performance and implementation of the new selector is presented. The selector
performance is evaluated on standard µ and π PID control samples.
momentum

elatively small polar angles; however, due to the boost of the Υ (4S) system at PEP-II, about 40%
of the muons within the BABAR detector’s acceptance pass through the forward IFR endcap). There
is a total of 65 cm (4 interaction lengths) of iron in the Barrel and 60 cm of iron in each End Cap.
There is ∼ 1 interaction length of material before the first RPC layer.
The original BABAR RPCs, installed in 1998, exhibited a rapid (∼ 10 − 15%/year) loss of singlelayer
efficiency. To address this problem, during the summer of 2002 the thickness of the forward
EC was increased to ∼ 6λ by the addition of non-magnetic absorber (brass), and 18 layers of the
original forward RPCs were replaced with 16 layers of new, high-quality RPCs. In the summer
2004, the top and bottom barrel IFR sectors were upgraded to 12 layers of Limited Streamer Tubes
(LSTs); additional brass absorber was added in place of the other 6 RPC layers to firther enhance
muon ID in the barrel. The LST upgrade of the entire IFR barrel was concluded in summer 2006,
making BABAR Runs 6 and 7 the only to have fully benefited from the superior muon ID offered by
the LST technology.
Electrical signals in a RPC are collected capacitively by strip electrodes along orthogonal directions
to obtain a two-dimensional readout for each active layer; in LSTs, the track’s ϕ coordinate is
read off the wires that run the length of the barrel. The hit strips associated with a charged track
are grouped into “3-D” clusters by extrapolating a track reconstructed in the Drift Chamber to
the IFR. The intersections of the extrapolated track with the active layers are computed. Only hit
strips which are less than a given distance from those intersections are associated to the charged
cluster.
3 Variables for Muon Identification
The quantities used for muon identification in the Cut-based Selector are [1]
1. The energy released in the Electromagnetic Calorimeter (E cal )
2. The number of IFR hit layers in the 3-D cluster (N L )
3. The measured number of interaction lengths traversed by the track in the BABAR detector
(λ meas ). It is estimated from the last layer hit by the extrapolated track in the IFR.
4. Delta Lambda, ∆λ = λ exp −λ meas , where (λ exp ) is the number of interaction lengths expected
to be traversed by the track in the muon hypothesis
5. The χ 2 /d.o.f. of the IFR hit strips w.r.t. a 3rd-order polynomial fit of the cluster (χ 2 fit )
6. The χ 2 /d.o.f. of the IFR hit strips in the cluster with respect to the IfrKalman track extrapolation
(χ 2 mat)
7. The continuity of the track in the IFR (T C )
8. The average multiplicity of hit strips per layer (m)
9. Standard deviation of the average multiplicity of hit strips per layer (σ m ).
The cut-based muon selector provides four different selection levels depending on different combinations
of the upper and lower cutoff values of the above parameters. These are termed as
V eryLoose, Loose, T ight, and V eryT ight, and are defined as shown in the Table 1.
The Neural Network selector provides similar selection levels as the cut-based selector. It uses
eight input variables: ∆λ, χ 2 mat, σ m , T C , E cal , λ meas , (χ 2 fit
) and (m).
4

Table 1: Four levels of selection criteria used in the cut-based muon selector algorithm.
Selection Variables VeryTight Tight Loose VeryLoose
E cal [0.4, 0.5] [0.4, 0.5] < 0.5 < 0.5
No. of Layers (N L ) > 1 > 2 > 2 > 2
Meas. Lambda (λ meas ) > 2.2 > 2.2 > 2 > 2
Delta Lambda (∆λ) < 0.8 < 1 < 2 < 2.5
Track Fit Chisq. (χ 2 fit ) < 3 < 3 < 4
Track Match Chisq. (χ 2 mat) < 5 < 5 < 7
Track Continuity (T C ) < 0.34 < 0.3 < 0.2 > 0.1
Average Strip Mult. (m) < 8 < 8 < 10 < 10
Sigma Strip Mult. (σ m ) < 4 < 4 < 6 < 6
4 Bagger Decision Trees
StatPatternRecognition (SPR) is a pattern-recognition software package developed by Ilya Narsky [6].
Preliminary investigation of using SPR for muon identification indicated that it held the promise
of noticeably exceeding the performance of the neural network.
SPR Bootstrap Aggregating Decision Trees (BDT) [6] were chosen for the muon ID task because
their performance exceeded other tools in SPR. A decision-tree algorithm involves splitting training
data into rectangular nodes. All possible binary splits of all input data in each dimension are
considered to find the split that produces the highest figure of merit. After the split, the algorithm
repeats the process recursively on the two nodes that resulted from the first split.
If the figure of merit of at least one of the resulting nodes after a split is not greater than the
previous figure of merit, the split is rescinded, and the node becomes a terminal node. If it has more
signal than background events, it is labelled as a signal node; otherwise, it becomes a background
node. In addition, the user building the tree can specify a minimum node size, so that splitting
ceases when a node reaches the minimum size.
Various figures of merit can be used. The negative Gini index is used by default. It is equal to
−2pq, where p and q = 1 − p are the fractions of correctly and incorrectly classified events in each
node.
Bootstrap aggregating involves training many trees on different subsets of the training data.
Each subset is a random draw from the full training data set. After training, data are classified by
a majority vote of the trained trees.
For the task of muon identification, the SPR BDT algorithm is trained on a set of muon and
pion tracks. The muon and pion training data are divided into 720 bins by p, polar angle θ, and
charge and then any excess muons or pions are discarded so that each bin holds equal numbers
of muons and pions. This process allows p, θ, and charge to be used by the tree as classification
variables without biasing the classification with regard to these variables; that is, the tree will not
separate muons from pions purely based on p, theta, or charge. The following list shows the 30
variables used for classification of the tracks:
1. momentum p
2. polar angle θ
5

3. charge
4. YYYYMM “date” – month and year of the track
IFR Variables
5. ifrns – number of strips in IFR cluster
6. χ 2 mat (ifrmatchchi2) – χ 2 between the IfrKalman track and cluster
7. χ 2 fit (ifrfitchi2) – χ2 for the cluster w.r.t. 3rd-order polynomial fits in both views
8. T C (ifrcont) – continuity of IFR hits in the 3-D IFR cluster
9. σ m (ifrsigmu) – truncated sigma of strip multiplicity
10. λ meas (ifrmeasintlen) – number of measured interaction lengths
11. ∆λ (deltalambda) – difference between the expected number of interaction lengths and the
measured number
12. ifrcrackphi – distance of the track from the nearest crack in the IFR detector; zero for tracks
that go through a crack
EMC Variables
13. E cal (ecal) – EMC energy, corrected for leakage of photon showers
14. lmom – lateral moment of the EMC shower
15. zmom20 – Zernicke 20 moment of shower shape
16. zmom42 – Zernicke 42 moment of shower shape
17. ncry – number of crystals in the EMC bump
18. s1s9 – energy of centroid crystal divided by the energy of the nine nearest crystals
19. s9s25 – energy of nine nearest crystals over the energy of the twenty-five nearest crystals
20. secmom – smoothness of shower in theta and phi
21. emcdepth – depth of the shower in the EMC
22. ecaldivp – ecal divided by p
DCH Variables
23. dEdxdchPullmu – the ratio of the expected dE/dx for a muon in the DCH over the measured
dE/dx, adjusted to make a Gaussian centered at 0
24. ndch – number of hits in the DCH
6

Figure 2: Ratios of BDT efficiency to neural network efficiency for the forward endcap (θ < 0.7) on the left
and the barrel (θ > 0.7) on the right. Muon and pion efficiency ratios are given for the eight selector criteria
(Very Loose, Loose, Tight, Very Tight, Very Loose Fake Rate, Loose Fake Rate, Tight Fake Rate, and Very
Tight Fake Rate).
25. lhit – last layer hit in the DCH (helps detect decays in flight)
DRC Variables
26. drcmuprob – probability for the track to be a muon, based upon DRC data
27. smsdrcmuprob – probability track is a muon, as computed by the SMS selector
28. drcpiprob – probability track is a pion, based upon DRC data
29. drckprob – probability track is a kaon, based upon DRC data
30. nphot – number of DRC photons
The result of the training is one SPR BDT classifier. Tracks can be passed to the classifier,
and it outputs a quantity that varies from 0 for pions to 1 for the tracks that are the most likely
to be muons. The performance of this BDT classifier meets or exceeds that of the previous neural
network classifier, as shown in Figure 2.
5 Control Samples
Several control samples have been used for the performance evaluation of the muon BDT. The
discrimination power of the algorithm is tested with the muon and pion control samples prepared
by the BABAR PID group. The muon control sample used are the very pure muons extracted from
the e + e − → µ + µ − γ channel. For pions, we select e + e − → τ + τ − events, with one τ decaying into
a lepton and the other into three pions, showing a 1-3 prong signature. Figure 3 shows typical p lab
and θ distributions of the muon and pion tracks from the control samples.
The distributions of the muon BDT classifier variables listed in section 4 are shown in Figs. 4
through 29.
7

0.06
0.05
0.025
0.02
π (τ 31
)
µ (µµγ)
0.04
0.03
0.02
0.01
π (τ 31
)
µ (µµγ)
0.015
0.01
0.005
0
0 2 4 6 8 10
P lab
0
0 0.5 1 1.5 2 2.5 3
θ
Figure 3: Momentum (left) and θ (right) distribution of muon (dashed) and pion (solid) tracks in lab from
µµγ and τ 31 PID control samples.
Figure 4: ifrns distributions for muons and pions.
Figure 5: χ 2 mat distributions for muons and pions.
Figure 6: χ 2 fit
distributions for muons and pions. Figure
7: T C distributions for muons and pions.
8

Figure 8: σ m distributions for muons and pions.
Figure 9: ∆λ distributions for muons and pions.
Figure 10: ifrcrackphi for muons and pions.
Figure 11: λ meas distributions for muons and pions.
Figure 12: E cal distributions for muons and pions.
Figure 13: ecaldivp distributions for muons and pions.
9

Figure 14: ncry for muons and pions.
Figure 15: lmom distributions for muons and pions.
Figure 16: zmom20 for muons and pions.
Figure 17: zmom42 distributions for muons and pions.
Figure 18: s1s9 for muons and pions.
Figure 19: s9s25 distributions for muons and pions.
10

Figure 20: secmom for muons and pions.
Figure 21: emcdepth for muons and pions.
Figure 22: dEdxdchPullmu for muons and pions.
Figure 23: ndch distributions for muons and pions.
Figure 24: lhit distributions for muons and pions.
Figure 25: nphot for muons and pions.
11

Figure 26: drcmuprob for muons and pions.
Figure 27: smsdrcmuprob for muons and pions.
Figure 28: drcpiprob for muons and pions.
Figure 29: drckprob for muons and pions.
12

6 Training and Validation
Training and validation of the decision tree involves several steps. The ROOT analysis framework
[4] is used along with SPR in the entire process, and it also provides the format for storing data
files. The starting point for training are the muon and pion control samples. These samples are
stored in ROOT ntuples. The µµγ muon sample is in the standard PID ntuple ntp207, and the
τ 31 pion sample is in ntp300. A ROOT macro called prepflat.C creates copies of these ROOT
ntuples, adds the calculated classifier variables (deltalambda, ecaldivp, and ifrcrackphi), and divides
the resulting files by particle charge. Then, a ROOT macro called splitchg.C randomly divides
these data files into training and testing samples.
The next step is the flattening of the training samples. Four classifier variables (p, theta,
charge, and date) are intended as secondary correlated variables to the other primary discriminating
variables. date should be inherently unbiased between muons and pions, since there should be
no favored times for the production of one over the other. Using the other three as primary
discriminating variables would highly bias the resulting selectors, so the numbers of muons and
pions in the training set for each of these variables must be made equal.
A ROOT macro called flattenSavMuons.C performs the necessary flattening. It takes four
files (muons and pions of each charge) and creates 360 bins in p and theta and discards particles in
a uniform, deterministic manner until the number of particles in each bin in the four files is equal.
The result is that the p and theta spectra of the training-set muons and pions are identical, and
the numbers of muons and pions of each charge are the same in each of the (p, θ) bins. The muons
discarded in the flattening process are saved so they can be added to the testing set.
The training set is used to train an SPR bagger decision tree. After extensive testing, the
following tree parameters were chosen as optimal: leaf size of 30, and 100 cycles (-l 30 -n 100).
No other tree parameters, like the Random Forest parameter, were found to be beneficial.
After training, the testing set is used to validate the performance of the decision tree. The BDT
selectors have two requirements. First, a selector should maintain its target muon or pion efficiency
across a range of momenta and across runs. Second, the standard selector (not the low-momentum
selector) should limit pion efficiency (and thereby muon efficiency) below a momentum value of 1
GeV/c so that it is no more than five times the pion efficiency produced by the corresponding neuralnetwork
selector. To achieve these requirements, a set of cuts on the classifier output produced by
the decision tree must be calculated from the testing set.
The ROOT macros cuts btchr24.C and cuts btchr24 03.C calculate the cuts for the standard
and low-momentum selectors, respectively. The following efficiencies are targeted. For the
Very Loose, Loose, Tight, and Very Tight criteria of the standard selector, the targets are 90%,
80%, 70%, and 60% muon efficiency, respectively. For the Very Loose Fake Rate, Loose Fake Rate,
Tight Fake Rate, and Very Tight Fake Rate criteria of the standard selector, the targets are 5%, 3%,
2%, and 1.2% pion efficiency, respectively. For the Loose and Tight criteria of the low-momentum
selector, the targets are 70% and 60% muon efficiency, respectively. Cuts on the classifier output
are calculated to achieve these efficiencies in momentum bins of 250 MeV/c for momenta from 0.5
to 4.0 GeV/c for the standard selector and in momentum bins of 200 MeV/c for momenta from
0.3 to 0.7 GeV/c for the low-momentum selector. These calculations are also binned by run for
the following run divisions: Run 1-1900V, Run 1-1960V, Run 2-2001, Run 2-2002, Run 3, Run 4,
Run 5a, Run 5b, Run 6, and Run 7.
13

7 Muon BDT Selectors in ROOT
The muon BDT selector is incorporated into the BABAR analysis framework known as “BetaPid”.
In the ntuples produced by its companion package BetaPidCalibNtuple, standard in Release 24,
are the following muon BDT variables. First are the booleans that tell whether a track matched a
muon BDT standard selector criterion:
• ismuBDTVeryLoose
• ismuBDTLoose
• ismuBDTTight
• ismuBDTVeryTight
• ismuBDTVeryLooseFakeRate
• ismuBDTLooseFakeRate
• ismuBDTTightFakeRate
• ismuBDTVeryTightFakeRate
The booleans for the low-momentum selector are:
• ismuBDTLoPLoose
• ismuBDTLoPTight
For diagnostic purposes, the classifier output and the cut values for each selector criterion are stored
in the ntuple. The standard and low-momentum classifier output values are named as follows:
• ifrBDTVal
• ifrBDTLoPVal
These two values are equal, except for tracks with momenta greater than 0.7 GeV/c, in which case
ifrBDTLoPVal is always −1.
The cut values for each selector criterion are stored in the following:
• ifrmuBDTVeryLooseCut
• ifrmuBDTLooseCut
• ifrmuBDTTightCut
• ifrmuBDTVeryTightCut
• ifrmuBDTVeryLooseFakeRateCut
• ifrmuBDTLooseFakeRateCut
• ifrmuBDTTightFakeRateCut
• ifrmuBDTVeryTightFakeRateCut
14

• ifrmuBDTLoPLooseCut
• ifrmuBDTLoPTightCut
A selector criterion boolean should be true when the classifier output is greater than the cut value.
For example:
(ifrBDTVal > ifrmuBDTTight) == ismuBDTTight
A boolean statement like the above should always be true, though computer rounding of floating
point values can cause occasional errors.
8 Performance of Muon BDT Selectors
This section presents the performance of the muon BDT selectors. Figures 30 and 31 show the
performance of the standard selector criteria that have muon efficiency targets, and Figs. 32 and
33 show the performance of the standard selector fake rate criteria. Figures 34 to 37 show the
performance of the low-momentum selector. Figure 38 shows how the performance of the fake rate
criteria varies by run number.
Figure 30: BDT selector efficiency vs. lab momentum in GeV/c for the forward endcap. Muon efficiencies
are on the left, pion efficiencies on the right. These plots only cover Runs 1-6, but the addition of Run 7
would not noticeably change the plots.
15

Figure 31: BDT selector efficiency vs. lab momentum in GeV/c for the barrel. Muon efficiencies are on the
left, pion efficiencies on the right. These plots only cover Runs 1-6, but the addition of Run 7 would not
change the plots.
Figure 32: BDT fake rate selector efficiency vs. lab momentum in GeV/c for the forward endcap. Muon
efficiencies are on the left, pion efficiencies on the right. These plots only cover Runs 1-6, but the addition
of Run 7 would not change the plots.
Figure 33: BDT fake rate selector efficiency vs. lab momentum in GeV/c for the barrel. Muon efficiencies
are on the left, pion efficiencies on the right. These plots only cover Runs 1-6, but the addition of Run 7
would not change the plots.
16

Figure 34: Low-momentum tight BDT selector efficiency vs. lab momentum in GeV/c for the forward endcap.
Muon efficiencies are on the left, pion efficiencies on the right. Each Run division is plotted.
Figure 35: Low-momentum tight BDT selector efficiency vs. lab momentum in GeV/c for the barrel. Muon
efficiencies are on the left, pion efficiencies on the right. Each Run division is plotted.
Figure 36: Low-momentum loose BDT selector efficiency vs. lab momentum in GeV/c for the forward endcap.
Muon efficiencies are on the left, pion efficiencies on the right. Each Run division is plotted.
17

Figure 37: Low-momentum loose BDT selector efficiency vs. lab momentum in GeV/c for the barrel. Muon
efficiencies are on the left, pion efficiencies on the right. Each Run division is plotted.
Figure 38: Efficiency vs. run number for the fake rate selector criteria. Black is the Very Loose Fake Rate,
blue is the Loose Fake Rate, green is the Tight Fake Rate, and red is the Very Tight Fake Rate.
18

9 Implementation in BetaPid
The muonBDT selector is implemented in the 24-series releases of the BABAR software, with release
24.3.2 (analysis-50) being the first release where the muonBDT selector and other SPR-based PID
selectors are officially supported (along with anal50boot conditions for analysis of R22-reconstructed
data).
The trained SPR kernel for the muonBDT selector is stored in the Release 24 BABAR Conditions
DataBase (CDB) in the CDB container /physicstools/pid/test/muonBDT2. At present, a single
kernel covers Runs 1–7, the YYYYMM “date” variable proving a handle and allows the muonBDT
selector to track the strong time dependence of the IFR detector performance. Ten muon BDT
selector lists are implemented. The muonBDT selectors are run as part of the standard PID
sequences, with an interface identical to that of the muMicro cut-based selectors except for the
list names. The Run-dependent cut values, binned in momentum, polar angle, and charge and
interpolated in momentum and polar angle in the selector code, are stored in the CDB container
/physicstools/pid/test/muonBDTcuts2 and are chosen automatically for each event via the CDB
proxy mechanism, which is implemented in package BetaPidProxy, new in Release 24.
Unlike the muMicro selectors, for which muon ID efficiency and pion mis-ID rate both vary over
time, p, and θ, the BDT selectors attempt to deliver either a constant efficiency or a constant mis-ID
rate. Six of the BDT lists (muBDTVeryLoose, muBDTLoose, muBDTTight, muBDTVeryTight,
muBDTLoPLoose, and muBDTLoPTight,) are tuned to give a constant muon ID efficiency. For
these lists the pion mis-ID rate varies with momentum, polar angle, and time. The other four
(muBDTVeryLooseFakeRate, muBDTLooseFakeRate, muBDTTightFakeRate, and muBDTVery-
TightFakeRate) are tuned to give a constant pion mis-ID rate. For these criteria the muon efficiency
will vary. Table 2 summarizes these lists.
The muon BDT selectors are available in Release 24 (analysis-50 and later).
List name Muon efficiency Pion mis-ID
muBDTVeryLoose 90.0% Variable
muBDTLoose 80.0% Variable
muBDTTight 70.0% Variable
muBDTVeryTight 60.0% Variable
muBDTVeryLooseFakeRate Variable 5.0%
muBDTLooseFakeRate Variable 3.0%
muBDTTightFakeRate Variable 2.0%
muBDTVeryTightFakeRate Variable 1.2%
muBDTLoPLoose 70.0% Variable
muBDTLoPTight 60.0% Variable
Table 2: Target efficiencies or mis-ID rates for the ten muon BDT lists.
19

References
[1] D. Azzopardi et al. “Muon Identification in the BABAR Experiment”, BABAR Analysis Document
#60.
[2] B. Aubert et al. “The BABAR Detector”, Nucl. Instr. and Meth. A 479 (2002) 1-116.
[3] R. Santonico, R. Cardarelli, Nucl. Instr. and Meth. A 187 (1981), 377.
[4] “ROOT, An Object Oriented Data Analysis Framework”, See http://root.cern.ch/
[5] A. Mohapatra et al., “Studies of a Neural Net Based Muon Selector for the BABAR Experiment”,
BABAR Analysis Document #474.
[6] I. Narsky, “Optimization of Signal Significance by Bagging Decision Trees”, arXiv:physics/0507157.
20