TrkQual BDT

README.md

@@ -31,7 +31,9 @@ You can get some basic information about the EventNtuple file useing ```checkEve
 checkEventNtuple file1.root file2.root
 ```
 
-This will print the version number (for versions after v6.3.0) and the trigger branches in the file (for versions after v6.8.0)
+This will print the version number (for versions after v6.3.0), configured
+TrkQual model metadata when available (for versions after v6.13.0), and the trigger branches in the file
+(for versions after v6.8.0).
 
 
 ## How to Analyze an EventNtuple

bin/checkEventNtuple

@@ -3,6 +3,14 @@
 import argparse
 import ROOT
 
+ROOT.gInterpreter.Declare("""
+void printTrkQualMetadata(TH1* metadata) {
+  for (int bin = 1; bin <= metadata->GetNbinsX(); ++bin) {
+    std::cout << " - " << metadata->GetXaxis()->GetBinLabel(bin) << std::endl;
+  }
+}
+""")
+
 # A main function so that this can be run on the command line
 def main():
     parser = argparse.ArgumentParser(
@@ -16,6 +24,7 @@ def main():
     foldername = "EventNtuple"
     histname = "version"
     treename = "ntuple"
+    trkqualmetadataname = "trkqual_metadata"
     for filename in args.filenames:
         try:
             f = ROOT.TFile(filename, "READ");
@@ -28,6 +37,12 @@ def main():
                 else:
                     print("{} histogram does not exist in {} (it is either v06_02_00 or older)".format(histname, filename))
 
+                # Print configured track-quality algorithms, if available.
+                if folder.GetListOfKeys().Contains(trkqualmetadataname):
+                    metadata = f.Get(foldername+"/"+trkqualmetadataname)
+                    print("\nIt contains the following trkqual branches:", flush=True)
+                    ROOT.printTrkQualMetadata(metadata)
+
                 # Print triggers
                 if (folder.GetListOfKeys().Contains(treename)):
                     t = f.Get(foldername+"/"+treename)

fcl/README.md

@@ -8,6 +8,23 @@ from_tier-type_extra.fcl
 
 where ```tier``` is the data tier of the input dataset, ```type``` is the type of dataset (e.g. primary-only, extracted position), and ```extra``` gives some extra information (optional)
 
+## Multiple TrkQual outputs
+
+Configure each TrkQual result in a track fit with `trkQualLeaves`. `leafname` is
+appended to `<branchname>qual`, so stable output names do not depend on the
+order of the configured algorithms:
+
+```
+trkQualLeaves : [
+  { leafname : ""           inputTag : "TrkQualAll:ANN"       modelVersion : "TrkQual_ANN1_v2" },
+  { leafname : "_candidate" inputTag : "TrkQualCandidate:ANN" modelVersion : "TrkQual_ANN1_v3_rc1" }
+]
+```
+
+This writes `trkqual` and `trkqual_candidate`. EventNtuple also records the
+track branch, output branch, input tag, and model version in
+`EventNtuple/trkqual_metadata`; `checkEventNtuple` prints this metadata.
+
 ## Table of Fcl Files
 
 | fcl file | runs on | additional info |
@@ -21,7 +38,7 @@ where ```tier``` is the data tier of the input dataset, ```type``` is the type o
 | from_mcs-ceSimRecoVal.fcl | output of EventNtuple/validation/ceSimReco.fcl | for validating the ```trkhitcalibs``` branch |
 | from_mcs-mockdata_separateTrkBranches.fcl | mock datasets | example on how to separate the tracks into separate branches again|
 | from_mcs-mockdata_selectorExample.fcl | mock datasets | example on how to use a selector to select certain types of tracks before putting them into the EventNtuple |
-| from_mcs-mixed_trkQualCompare.fcl | reconstructed mixed (i.e. primary+background hits) datasets | shows how to output result of more than one TrkQual |
+| from_mcs-mixed_trkQualCompare.fcl | reconstructed mixed (i.e. primary+background hits) datasets | shows explicitly named TrkQual outputs and embedded model-version provenance; requires the listed comparison ONNX models |
 | from_mcs-primary_addVDSteps.fcl | reconstructed primary (i.e. no background hits) datasets | shows how to add the branch for virtual detector steps |
 | from_mcs-Run1B.fcl | reconstructed Run-1B (backup plan) datasets | adds the branch for virtual detector steps |
 | from_mcs-DeMCalib.fcl | reconstructed primary or mixed datasets | only writes one track per event |

fcl/from_mcs-mixed_trkQualCompare.fcl

@@ -1,17 +1,22 @@
 #include "EventNtuple/fcl/from_mcs-mockdata.fcl"
 
-# Add another TrackQuality module
+# Compare explicitly named TrackQuality branches.  The v1.0 and v1.1 ONNX
+# files must be installed beside the v2 model before running this example.
 physics.producers.TrkQualAllV10 : @local::TrkQualAll
-physics.producers.TrkQualAllV10.datFilename : "Offline/TrkDiag/data/TrkQual_ANN1_v1.dat"
+physics.producers.TrkQualAllV10.onnxFilename : "ArtAnalysis/TrkDiag/data/TrkQual_ANN1_v1.onnx"
 
 physics.producers.TrkQualAllV11 : @local::TrkQualAll
-physics.producers.TrkQualAllV11.datFilename : "Offline/TrkDiag/data/TrkQual_ANN1_v1.1.dat"
+physics.producers.TrkQualAllV11.onnxFilename : "ArtAnalysis/TrkDiag/data/TrkQual_ANN1_v1.1.onnx"
 physics.producers.TrkQualAllV2 : @local::TrkQualAll
-physics.producers.TrkQualAllV2.datFilename : "Offline/TrkDiag/data/TrkQual_ANN1_v2.dat"
+physics.producers.TrkQualAllV2.onnxFilename : "ArtAnalysis/TrkDiag/data/TrkQual_ANN1_v2.onnx"
 physics.EventNtuplePath : [ @sequence::EventNtuple.Path, TrkQualAllV10, TrkQualAllV11, TrkQualAllV2 ]
 
-# Add it to the EventNtuple output
-physics.analyzers.EventNtuple.branches[0].trkQualTags : [ "TrkQualAllV10", "TrkQualAllV11", "TrkQualAllV2" ]
+# Add explicitly named results and record their model versions in the output.
+physics.analyzers.EventNtuple.trk.fits[0].trkQualLeaves : [
+  { leafname : "_v1_0" inputTag : "TrkQualAllV10:ANN" modelVersion : "TrkQual_ANN1_v1.0" },
+  { leafname : "_v1_1" inputTag : "TrkQualAllV11:ANN" modelVersion : "TrkQual_ANN1_v1.1" },
+  { leafname : "_v2"   inputTag : "TrkQualAllV2:ANN"  modelVersion : "TrkQual_ANN1_v2" }
+]
 
-# Removin hits
+# Remove hits
 physics.analyzers.EventNtuple.trk.fillHits : false

fcl/from_mcs-reflection.fcl

@@ -83,14 +83,14 @@ physics.trigger_paths : [ "eTrig", "muTrig"]
 physics.end_paths : [ "eEnd", "muEnd", "CLPrint" ]
 physics.analyzers.ENe.trk.fits : [
   { @table::ENBranch
-    trkQualTags : ["TrkQualReflecte"]
+    trkQualLeaves : [ { leafname : "" inputTag : "TrkQualReflecte:ANN" modelVersion : "TrkQual_ANN1_v2" } ]
     trkPIDTags : ["TrkPIDReflecte"]
     input: "Reflecte"
   }
 ]
 physics.analyzers.ENmu.trk.fits : [
   { @table::ENBranch
-    trkQualTags : ["TrkQualReflectmu"]
+    trkQualLeaves : [ { leafname : "" inputTag : "TrkQualReflectmu:ANN" modelVersion : "TrkQual_ANN1_v2" } ]
     trkPIDTags : ["TrkPIDReflectmu"]
     input: "Reflectmu"
   }

fcl/prolog.fcl

@@ -10,6 +10,7 @@ BEGIN_PROLOG
 TrkQual : {
   module_type : TrackQuality
   onnxFilename : "ArtAnalysis/TrkDiag/data/TrkQual_ANN1_v2.onnx"
+  xgbFilename : "ArtAnalysis/TrkDiag/data/TrkQual_BDT1_v2.0.ubj"
   debugLevel : 0
 
 }
@@ -192,91 +193,94 @@ DeM : { input : "MergeKKDeM"
   branchname : "dem"
   fill : true
   options : { fillMC : true  fillHits : false genealogyDepth : -1 matchDepth : -1 }
-  trkQualTags : ["TrkQualDeM"]
+  trkQualLeaves : [ { leafname : "" inputTag : "TrkQualDeM:ANN" modelVersion : "ANN1_v2" } ]
   trkPIDTags : ["TrkPIDDeM"]
 }
 UeM : { input : "MergeKKUeM"
   branchname : "uem"
   fill : true
   options : { fillMC : true  fillHits : false genealogyDepth : -1 matchDepth : -1 }
-  trkQualTags : [ ]
+  trkQualLeaves : [ ]
   trkPIDTags : [ ]
 }
 DmuM : { input : "MergeKKDmuM"
   branchname : "dmm"
   fill : true
   options : { fillMC : true  fillHits : false genealogyDepth : -1 matchDepth : -1 }
-  trkQualTags : [ ]
+  trkQualLeaves : [ ]
   trkPIDTags : [ ]
 }
 UmuM : { input : "MergeKKUmuM"
   branchname : "umm"
   fill : true
   options : { fillMC : true  fillHits : false genealogyDepth : -1 matchDepth : -1 }
-  trkQualTags : [ ]
+  trkQualLeaves : [ ]
   trkPIDTags : [ ]
 }
 DeP : { input : "MergeKKDeP"
   branchname : "dep"
   fill : true
   options : { fillMC : true  fillHits : false genealogyDepth : -1 matchDepth : -1 }
-  trkQualTags : [ ]
+  trkQualLeaves : [ ]
   trkPIDTags : [ ]
 }
 UeP : { input : "MergeKKUeP"
   branchname : "uep"
   fill : true
   options : { fillMC : true  fillHits : false genealogyDepth : -1 matchDepth : -1 }
-  trkQualTags : [ ]
+  trkQualLeaves : [ ]
   trkPIDTags : [ ]
 }
 DmuP : { input : "MergeKKDmuP"
   branchname : "dmp"
   fill : true
   options : { fillMC : true  fillHits : false genealogyDepth : -1 matchDepth : -1 }
-  trkQualTags : [ ]
+  trkQualLeaves : [ ]
   trkPIDTags : [ ]
 }
 UmuP : { input : "MergeKKUmuP"
   branchname : "ump"
   fill : true
   options : { fillMC : true  fillHits : false genealogyDepth : -1 matchDepth : -1 }
-  trkQualTags : [ ]
+  trkQualLeaves : [ ]
   trkPIDTags : [ ]
 }
 Ext : { input : "MergeKKLine"
   branchname : "trk"
   fill : true
   options : { fillMC : true  fillHits : true genealogyDepth : -1 matchDepth : -1 }
-  trkQualTags : [ ]
+  trkQualLeaves : [ ]
   trkPIDTags : [ ]
 }
 Off : { input : "MergeKKOff"
   branchname : "trk"
   fill : true
   options : { fillMC : true  fillHits : true genealogyDepth : -1 matchDepth : -1 }
-  trkQualTags : [ ]
+  trkQualLeaves : [ ]
   trkPIDTags : [ ]
 }
 All : { input : "MergeKKAll"
   branchname : "trk"
   fill : true
   options : { fillMC : true  fillHits : true genealogyDepth : -1 matchDepth : -1 }
-  trkQualTags : ["TrkQualAll"]
+  trkQualLeaves : [
+    { leafname : ""        inputTag : "TrkQualAll:ANN" modelVersion : "ANN1_v2" },
+    { leafname : "_bdt" inputTag : "TrkQualAll:BDT" modelVersion : "BDT1_v2" }
+  ]
   trkPIDTags : ["TrkPIDAll"]
 }
 DeCalib : { input : "MergeKKDeCalib"
   branchname : "trk"
   fill : true
   options : { fillMC : true  fillHits : true genealogyDepth : -1 matchDepth : -1 }
-  trkQualTags : [ ]
+  trkQualLeaves : [ ]
   trkPIDTags : [ ]
 }
 De : { input : "MergeKKDe"
   branchname : "de"
   fill : true
   options : { fillMC : true  fillHits : false genealogyDepth : -1 matchDepth : -1 }
-  trkQualTags : ["TrkQualDe"]
+  trkQualLeaves : [ { leafname : "" inputTag : "TrkQualDe:ANN" modelVersion : "ANN1_v2" } ]
   trkPIDTags : ["TrkPIDDe"]
 }
 
@@ -411,7 +415,7 @@ TTMCBranch : {
   branchname : "trk"
   fill : true
   options : { fillMC : true  fillHits : true genealogyDepth : -1 matchDepth : -1 }
-  trkQualTags : [ ]
+  trkQualLeaves : [ ]
   trkPIDTags : [ ]
 }
 
@@ -451,7 +455,7 @@ ENDeCalib.trk.fits : [
     branchname : "trk"
     fill : true
     options : { fillMC : true  fillHits : true genealogyDepth : -1 matchDepth : -1 }
-    trkQualTags : ["TrkQualDe"]
+    trkQualLeaves : [ { leafname : "" inputTag : "TrkQualDe:ANN" modelVersion : "ANN1_v2" } ]
     trkPIDTags : ["TrkPIDDe"]
   }
 ]

inc/TrkQualMetadata.hh

@@ -0,0 +1,14 @@
+#ifndef EventNtuple_TrkQualMetadata_hh
+#define EventNtuple_TrkQualMetadata_hh
+
+#include <string>
+
+namespace mu2e {
+  struct TrkQualMetadata {
+    std::string output_branch;
+    std::string input_tag;
+    std::string model_version;
+  };
+}
+
+#endif

rooutil/README.md

@@ -111,6 +111,18 @@ The ```CaloCluster``` class contains all information related to a single calorim
 
 Examples: [PlotCaloClusterEnergy.C](./examples/PlotCaloClusterEnergy.C), [PlotCaloClusterEnergy_RecoVsTrue.C](./examples/PlotCaloClusterEnergy_RecoVsTrue.C), [PlotCaloClusterAndHits.C](./examples/PlotCaloClusterAndHits.C), [PlotCaloCluster_SimParticles.C](./examples/PlotCaloCluster_SimParticles.C)
 
+### The ```UserBranch``` Class
+There are some branches in EventNtuple that can have their names defined at runtime. For example, additional ```trkqual``` branches can be added in order to compare different trainings or different algorithms. The ```UserBranch``` class handles these sorts of branches.
+
+Example: [CompareTrkQualTrainings_UserBranches.C](./examples/CompareTrkQualTrainings_UserBranches.C)
+
+#### TrkQual Branches
+RooUtil can bind either output by name with `MakeTrackUserBranch<mu2e::MVAResultInfo>`.
+Analysis code can stop on unexpected provenance with:
+
+```
+util.RequireTrkQualVersion("trkqual", "ANN2_v2");
+```
 
 ### Branches not contained within a class
 Some branches are not contained in any of the above classes: