Migrate calo calibration modules from CaloCalibration repository - #1936
Migrate calo calibration modules from CaloCalibration repository#1936giro94 wants to merge 28 commits into
Conversation
|
Unsure if we should keep the inner directory structure (CosmicsCalib, NoiseAnalysis, etc), or we should flatten everything into inc/src/fcl directories. @brownd1978 |
|
☀️ The build tests passed at c41473e.
N.B. These results were obtained from a build of this Pull Request at c41473e after being merged into the base branch at 0878d6b. For more information, please check the job page here. |
oksuzian
left a comment
There was a problem hiding this comment.
PR Review Summary — #1936
Reviewed at head c41473e9eb9851233e4da5ea8beef3876c2bc38b. First pass.
Decision
- 🔴 request changes
Scope understood
- Migrates four calorimeter calibration modules out of
Mu2e/CaloCalibrationintoOffline/CaloCalibration/:CosmicsCalib(CaloCosmicEnecalib,CaloCosmicEnergy,caloT0alig) andNoiseAnalysis(BaselineAnalyzerplusanalyzeBaselines.fcl). Source and combination steps are stated as following later. - The stated intent is that anything needed at online level or in Pass-1/2/N lives in Offline; the rest stays in the separate repo.
- Note on the prior review: @sophiemiddleton approved at this same head with an empty body, so there are no findings to carry forward. The blocker below is a build-system issue that an approval does not address; the CI green also does not cover it, for the reason given in finding 1.
Findings
-
🔴 [S0] The CMake build of Offline no longer configures.
- Evidence:
CMakeLists.txt:126addsadd_subdirectory(CaloCalibration), but at this headCaloCalibration/contains onlyCosmicsCalib/andNoiseAnalysis/— there is noCaloCalibration/CMakeLists.txt
(gh api repos/Mu2e/Offline/contents/CaloCalibration?ref=c41473e9returns exactly those two entries). Reproduced against the real cmake:Adding the missing intermediate file then exposes a second, independent error, becauseCMake Error at CMakeLists.txt:3 (add_subdirectory): The source directory .../CaloCalibration does not contain a CMakeLists.txt file.CaloCalibration/NoiseAnalysis/CMakeLists.txt:16begins with a stray%:CMake Error at CaloCalibration/NoiseAnalysis/CMakeLists.txt:1: Parse error. Expected a command name, got unquoted argument with text "%install_headers".CaloCalibration/CosmicsCalib/has noCMakeLists.txtat all, so its three modules would not be built by CMake even once configuration succeeds. - Why CI is green anyway, and why this is not caught:
mu2e/buildtestis the scons-via-Muse build, and it did compile and link all four modules (scons.loglines 713-780, including-Wl,--no-undefined). Thecheck_cmakejob iteratesfor dir in $PWD/*and only descends where$dir/srcexists (bin/check_cmake.sh:33, guarded by the-d $dir/srctest at:8), so a two-level package is invisible to it — it reported success without ever looking atCaloCalibration. - Suggested fix: add
CaloCalibration/CMakeLists.txtwithadd_subdirectory(CosmicsCalib)andadd_subdirectory(NoiseAnalysis); addCaloCalibration/CosmicsCalib/CMakeLists.txtwith acet_build_pluginblock per module; and inNoiseAnalysis/CMakeLists.txtdrop the%and use the spelling the rest of the repo uses,install_headers(USE_PROJECT_NAME SUBDIRS inc)— or drop that line entirely, since there is noinc/directory here. Two smaller items in the same file:install_fhicl(SUBDIRS fcl SUBDIRNAME CaloCalibration/NoiseAnalysis/fcl)is missing theOffline/prefix that every otherinstall_fhiclin the repo carries, and the file has no trailing newline.
- Evidence:
-
🔴 [S0] Out-of-bounds writes in
CaloCosmicEnergywhen a hit lands exactly on the top of the energy range.- Evidence:
CaloCosmicEnergy_module.cc:408-416if (sipm_mean_e <= 55.) { int whichband = sipm_mean_e / 5; Energy_band[whichband] += sipm_mean_e; counter_energy_band[whichband]++; LR[whichband]->Fill(...); ALR[whichband]->Fill(...); CryALR[crystal_id][whichband]->Fill(...); Cry_Energy_band[crystal_id][whichband] += sipm_mean_e; Cry_counter_energy_band[crystal_id][whichband]++; }
Ebinis 11 (:98) and all five of those arrays are dimensioned[Ebin](:120-122,:128-129). A hit withenergyDep()of exactly 55.0 MeV giveswhichband == 11, one past the end of every one of them.LR,ALRandCryALRare arrays ofTH1F*, so the write is preceded by a read of an out-of-range pointer which is then dereferenced through->Fill(). A negativeenergyDep()— which the reconstruction can produce on a noise-dominated channel — indexes at-1by the same path, since the guard has no lower bound. - Impact: heap corruption or a segfault in a calibration job, dependent on input values, so it will not show up reliably in a short test.
- Suggested fix: make the guard exclusive and two-sided —
if (sipm_mean_e >= 0. && sipm_mean_e < Ebin * 5.)— and derive the band width from a named constant rather than the literal5repeated at:186,:204,:207and:409.
- Evidence:
-
🟠 [S1]
std::stringconstructed from a possibly-nullgetenv, with the emptiness check placed after the fact.- Evidence: three sites.
caloT0alig_module.cc:142std::string _fileT0 = getenv("MUSE_WORK_DIR");,caloT0alig_module.cc:446-450andCaloCosmicEnergy_module.cc:433-437:Constructingstd::string outDir = std::getenv("OUTDIR"); if (outDir.length() == 0) { mf::LogError("OUTDIR-NOT-SET") << "Environmental variable for calib output file not set "; }
std::stringfrom a null pointer is undefined behaviour, so theLogErrorbelow it can never run for the case it is written for; and when it does run, execution continues and the job writes to/tcorr.datand/calib_parameters.datat the filesystem root.MUSE_WORK_DIRin particular is set by Muse and will not be present in a CMake/spack-installed release, which is the environment this migration is meant to serve. - Impact: a segfault, or output silently written outside the intended directory, from an unset environment variable.
- Suggested fix:
CaloCosmicEnecalibin this same PR already shows the pattern to follow — anOutCalibFilefhicl atom, opened in the constructor, withthrow cet::exception(...)when the open fails (CaloCosmicEnecalib_module.cc:73-75,:187-190). Give the other two modules the same treatment and drop thegetenvcalls; the T0 data file path should likewise come from fhicl rather than fromMUSE_WORK_DIR.
- Evidence: three sites.
-
🟠 [S1] The migration is incomplete:
caloT0aligreads a data file that was not moved, and neither was any fcl forCosmicsCalib.- Evidence:
caloT0alig_module.cc:142-157reads$MUSE_WORK_DIR/CaloCalibration/CosmicsCalib/data/t0s_allchan_1ns.dat. That file exists in the source repo (Mu2e/CaloCalibration→CosmicsCalib/data/t0s_allchan_1ns.dat) but is not in this PR —CaloCalibration/CosmicsCalib/contains onlysrc.CosmicsCalib/vst/was not migrated either, andNoiseAnalysisis the only one of the two packages that brings itsfcl/directory across. - Impact:
if (T0File.is_open())simply fails and the job proceeds withToff[]all zeros — no warning, no error, and a plausible-looking set of residuals out the far end. The threeCosmicsCalibmodules also have no runnable configuration in Offline, so nothing in the repo exercises them. - Suggested fix: bring
data/and a driver fcl across with the modules, and treat a missing T0 file as fatal rather than as a silent zero. Worth stating explicitly in the PR body ifvst/is deliberately staying behind.
- Evidence:
-
🟠 [S1]
caloT0aligwrites into fixed arrays using an index read straight out of a text file.- Evidence:
caloT0alig_module.cc:150-151while (T0File >> iChanT0 >> TvalT0) { Toff[iChanT0] = TvalT0; ... }and:174-175while (inpFile >> iChan >> Tval >> ...) { Tcor[iChan] = Tval; ... }.ToffandTcorarefloat[nROchan]withnROchan == 2696(:99-102), and neither loop bounds-checks the index. The count is only checked afterwards, at:181, and only for the second file. - Impact: a stale or corrupted calibration file — exactly the class of input this iterative procedure regenerates each pass — overwrites arbitrary memory.
- Suggested fix: reject
iChan < 0 || iChan >= nROchaninside both loops with acet::exception.
- Evidence:
-
🟠 [S1]
CaloCosmicEnergydivides by a path length that its own helper can return as zero.- Evidence:
CaloCosmicEnergy_module.cc:759-849—findpathinitialisesfloat path = 0;and has anelsebranch (:840-844) that assigns nothing when no crystal face is crossed; it also leavesxup/xlow/yleft/yrightat zero whenm == 0(:769). The result is used unguarded at:384:... ->energyDep() * cryDim / path[iCry]. - Impact: an infinity is filled into
hSiPMfp, where it lands in the overflow bin and quietly biases the normalized-track MPV. - Suggested fix: the newer
CaloCosmicEnecalibalready guards this —else if ((chi2norm < CutChi2Norm) && (path[kk] > 0))atCaloCosmicEnecalib_module.cc:429. Apply the same guard here, and havefindpathsignal "no path" explicitly rather than returning a value that reads as a real length.
- Evidence:
Smaller items
- 🟡 Dead code, several kinds:
caloT0alig_module.cc:424-426is unreachable afterreturn retval;at:422;int diag = 0;with a dozenif (diag == 1)blocks that can never run appears in bothCaloCosmicEnergy_module.cc:766andCaloCosmicEnecalib_module.cc:602; commented-out code atcaloT0alig_module.cc:468-470,CaloCosmicEnergy_module.cc:213-215and:236-241, andanalyzeBaselines.fcl:18(#@local::Services.Reco);TFitResultPtr fitresultatCaloCosmicEnergy_module.cc:315is never read;_nProcessed/_nFilteredare counted incaloT0aligand never reported anywhere. - 🟡
CaloCosmicEnergy_module.cc:292assignsmax_y = PosX[h];inside the loop that is scanningPosY.Dyis only ever printed at_diagLevel > 0(:303), so nothing downstream is wrong today, but the variable is both mis-computed and otherwise unused — either fix it or drop it. The two loops at:273and:289are also labelled "bubble sort" when they are min/max scans. - 🟡
caloT0alig_module.cc:326-333mixes an SiPM-local id with a vector position:idxisCaloSiPMId::SiPMLocalId(), which is_id % 2(DataProducts/inc/CaloSiPMId.hh:25), but it is then used to subscripthit.recoCaloDigis().at(idx)while the loop itself runs overiCha. On a crystal with a single surviving readout whose local id is 1,.at(1)throwsstd::out_of_range; where both are present but not stored in local-id order, one digi is read twice and the other never. Index withiChaand use the local id only where a local id is meant. - 🟡 Silent degradation in
BaselineAnalyzer: a CSV that will not open producesstd::cout << "Warning! ..."and setswriteCSV_ = false(:272-278), so the job exits 0 having produced no thresholds; and channels with no data are given a fabricated baseline of 2048 that is then written into the threshold CSV alongside the measured ones (:401-412), with nothing in the file marking them as defaults. Both should be errors. - 🟡 Numbers that already have a home elsewhere.
BaselineAnalyzer_module.cchardcodes16100(:139,:510) whereCaloConst::_nDIRACis 161,20(:403,:418,:424,:567) where it usesCaloConst::_nChPerDIRACcorrectly at:182,board < 80for the disk split (:382), and2048as the pedestal (:136-137,:408).CaloCosmicEnergy_module.ccfills the position error with9.81fand a comment deriving it from a 34 mm crystal (:231) while reading the real crystal dimension from the geometry twelve lines earlier (:156), and carries three different vertical-track thresholds —Dx < 33(:325),Dx < 35(:368) andMaxDxVertical = cryDim * 1.1(:158,:340).caloT0alig_module.cc:310uses3.1416for π and:94redefines the speed of light ascvel = 299.792458rather thanCLHEP::c_light. - 🟡
CaloCosmicEnergyandCaloCosmicEnecalibcarry verbatim copies offindpath(~70 lines), of the Landau-Gauss convolution (langaufun/langaus), and of the9.81/144./cryDim * 1.1constants. Since the header ofCaloCosmicEnecalib_module.cc:5-6describes it as the successor toCaloCosmicEnergy, it is worth saying in the PR body whether both are meant to live in Offline long-term; if they are, the shared pieces belong in one place. - 🟡
BaselineAnalyzer'swriteTXT/TXTfoldername,writeCSV/CSVfilenameandwritePDF/PDFfilenameare the flag-plus-loose-atoms shape thatfhicl::OptionalTable<Config>exists for; withwriteTXT: trueand the default empty folder the module writes to/dirac000.baseline. Separately, the C++ defaults arethresholdOffset = 100,thresholdOffsetPin = 50(:73-74) whileanalyzeBaselines.fcl:56-57sets50and100— the two are swapped relative to each other, which is worth confirming is deliberate. - ⚪ Collapsed nits:
BaselineAnalyzer_module.cc:12-13and:19-22include five artdaq headers (Fragment,ContainerFragment,EventHeader,DTCEventFragment,CalorimeterDataDecoder,FragmentType) that the module never uses, and those are what pull the fourartdaq-core*entries into its link list;TFile,TEllipse,TTreeand<sys/stat.h>are unused there too, as areGlobalConstantsHandle.hh,TDirectory.h,Selector.handSequence.hincaloT0alig_module.cc. ClasscaloT0aligstarts lowercase where the repo capitalises type names; itsbeginJob/endJob/filterarevirtualwithoutoverride(:78-80). Prints go tostd::coutrather than message-facility throughout, several of them unguarded by any verbosity flag (BaselineAnalyzer_module.cc:275,:291,:331,:434). The "*** TO BE IMPLEMENTED ***" markers atcaloT0alig_module.cc:132and:244are real TODOs that the FIXME/TODO CI counter does not match on.
On the directory-structure question
Keeping CosmicsCalib/NoiseAnalysis as subdirectories is fine and has precedent: ExtinctionMonitorFNAL/ is a two-level package whose top-level CMakeLists.txt is nothing but seven add_subdirectory lines plus its own install_fhicl, and each leaf carries the usual cet_build_plugin / install_source / install_headers block. Copying that shape is exactly what finding 1 asks for. The one real cost is the check_cmake.sh blindness described above, which ExtinctionMonitorFNAL shares — that is a pre-existing gap in the CI script, not something this PR introduced, but it does mean a nested package gets less automatic protection than a flat one.
Validation check
- Build/tests run: partial.
mu2e/buildtestis green at this head and genuinely compiled and linked all four modules under-Werror(scons.log:713-780); whitespace clean; FIXME/TODO 0 in 4 files; clang-tidy reported 8 errors / 942 warnings, which I did not attribute. I reproduced the two CMake errors in finding 1 against a minimal tree with the same file layout. - Config contract check: pass for the one new fcl.
fhicl-dump Offline/CaloCalibration/NoiseAnalysis/fcl/analyzeBaselines.fclunderSimJob/MDC2025avwith an appended shim exits 0 and resolves to 1216 lines;CaloDigisFromDTCEvents(DAQ/src/) andCaloVisualizer/inc/THMu2eCaloDisk.hhboth exist in Offline at this head.CosmicsCalibhas no fcl to check. - Cross-repo consistency: needs follow-up — see finding 4 for what stayed behind in
Mu2e/CaloCalibration.
Residual risk
- I did not review the physics of the calibration procedures themselves (langaus fitting strategy, the asymmetry-to-Npe inversion, the T0 iteration scheme), only their implementation.
- Nothing in Offline runs the three
CosmicsCalibmodules, so none of them has runtime coverage here; the failure modes in findings 2, 5 and 6 are reached by particular input values and would not surface in a short smoke test.
Author follow-ups
- Add
CaloCalibration/CMakeLists.txtandCaloCalibration/CosmicsCalib/CMakeLists.txt, and fix the%install_headersline, the missingOffline/prefix oninstall_fhicl, and the missing trailing newline inNoiseAnalysis/CMakeLists.txt. Please confirm with a local CMake configure, since buildtest will not catch it. - Bound the energy-band index in
CaloCosmicEnergyand the file-read indices incaloT0alig. - Replace the three
getenvcalls with fhicl parameters that throw when the target cannot be opened, followingCaloCosmicEnecalib. - Migrate
CosmicsCalib/data/and a driver fcl, or say in the PR body what is deliberately staying inMu2e/CaloCalibration. - Guard the
cryDim / pathdivision inCaloCosmicEnergy. - Say whether
CaloCosmicEnergyandCaloCosmicEnecalibare both intended to live here long-term; if so, the duplicatedfindpathand langaus code should get a single home.
bechenard
left a comment
There was a problem hiding this comment.
Ok, but I would like to avoid creating 50 Calo folders in the future
Do you mean within this CaloCalibration folder or the number of CaloXXX folders in Offline? @bechenard |
|
📝 The HEAD of |
oksuzian
left a comment
There was a problem hiding this comment.
PR Review Summary — #1936
Reviewed at head 415724542483228793acda0e9e94453e8f925a81. Re-review of c41473e9.
Decision
- 🔴 request changes
Both blockers from the previous pass are fixed, and I verified each rather than taking
the commit messages for it. The delta — 764a2eca "added cmakelists and prologs" and
41572454 "Addressed energy range bug", six files — introduces one new S0, the
NoiseAnalysis prolog cannot be parsed, plus one new S1 in the CMake library lists.
Findings 3, 5 and 6 from the previous pass are untouched; finding 4 is partly addressed.
Fixed since c41473e9
- 🟢 [was S0] The CMake build configures again. With
CaloCalibration/CMakeLists.txt,
CosmicsCalib/CMakeLists.txtand the%gone fromNoiseAnalysis/CMakeLists.txt:16,
a configure over the new tree with thecet_*macros stubbed exits 0 and registers
all four plugins —CaloCosmicEnecalib,CaloCosmicEnergy,caloT0alig,
BaselineAnalyzer. Bothinstall_fhiclcalls now carry theOffline/prefix, and
the missing trailing newline is gone. - 🟢 [was S0] The energy-band index is bounded.
CaloCosmicEnergy_module.cc:409is now
if (sipm_mean_e >= 0. && sipm_mean_e < Ebin * Erange)with
whichband = sipm_mean_e / Erangeat:410. WithEbin = 11,Erange = 5
(:98-99) that admits 0 through 10 against arrays dimensioned[11](:123,
:129-130), and the negative case is closed. The literal5is gone from all four
sites.
Findings
-
🔴 [S0]
NoiseAnalysis/fcl/prolog.fcldoes not parse — it references a name that
does not exist- Evidence:
:8definesCaloBaselineAna, but:21reads
CaloBaselineAna : @local::CaloCaloBaselineAna—Calodoubled.fhicl-dumpon a
file whose only content is an include of this prolog aborts, exit 134:The sibling---- Parse error BEGIN Local lookup error ---- Can't find key BEGIN CaloCaloBaselineAna (at part "CaloCaloBaselineAna") ---- Can't find key END at line 21, character 27, of file ".../NoiseAnalysis/fcl/prolog.fcl"CosmicsCalib/fcl/prolog.fcldumps clean at exit 0, so it is this one
line. - Impact: the prolog is unusable as committed, and nothing in CI includes it, so it
will fail for the first person who tries to use it. - Suggested fix:
CaloBaselineAna : @local::CaloBaselineAna.
- Evidence:
-
🟠 [S1] The three new
cet_build_pluginblocks carryNoiseAnalysis's library list
verbatim, which is not the list these modules need- Evidence: the
LIBRARIES REGblock is byte-identical across all four plugin
declarations in the two new CMakeLists. It namesOffline::CaloConditions,
Offline::CaloVisualizer,Offline::CaloVisualizer_dict,Offline::DAQand
Offline::ProditionsService— the packagesBaselineAnalyzerincludes, none of
which appears in anyCosmicsCalibmodule. Meanwhile all threeCosmicsCalib
modules constructGeomHandle<Calorimeter>(CaloCosmicEnergy_module.cc:155,
CaloCosmicEnecalib_module.cc:202,caloT0alig_module.cc:253) and
Offline::GeometryServiceis not listed.Offline::CalorimeterGeomdoes arrive
transitively viaCaloConditions, butGeometryServiceappears in the PUBLIC list
of none of the eight entries. The closest sibling,CaloReco'sCaloHitMaker—
the other module in Offline doingGeomHandle<Calorimeter>— declares both
Offline::CalorimeterGeomandOffline::GeometryServiceexplicitly. - Impact: the standard asks for all first-order dependencies to be declared, and CI
never builds with CMake, so a wrong CMake link list stays green through
buildtestindefinitely — the same gap that hid the previous blocker. - Suggested fix: derive each list from that module's own includes instead of copying
the donor, and confirm with a realcmakebuild, not a configure. I traced the
PUBLIC lists one level only and did not close the dependency graph exhaustively, so
please let the build be the authority.
- Evidence: the
-
🟠 [S1] Carried over, unaddressed —
std::stringfrom a possibly-nullgetenv.
caloT0alig_module.cc:142and:446,CaloCosmicEnergy_module.cc:434, all
unchanged at this head. -
🟠 [S1] Carried over, partly addressed — incomplete migration.
CosmicsCalib/fcl/prolog.fclnow exists, which closes part of this, but it
configures onlyCaloCosmicEnecalib;CaloCosmicEnergyandcaloT0aligstill have
no configuration in Offline, andCosmicsCalib/data/t0s_allchan_1ns.dat— read at
caloT0alig_module.cc:142-157— is still not in the PR, so that read still degrades
silently to all-zero offsets. When you addcaloT0alig: it is anart::EDFilter
(:57), so it belongs underfilters:, notanalyzers:. -
🟠 [S1] Carried over, unaddressed — unbounded array indices read from a text
file,caloT0alig_module.cc:150-151and:174-175. -
🟠 [S1] Carried over, unaddressed —
CaloCosmicEnergy_module.cc:385still
divides bypath[iCry]unguarded, whileCaloCosmicEnecalib_module.cc:429still
carries thepath[kk] > 0guard it needs.
Smaller items
- 🟡 New, on a line this delta touched: the
CryALRtitles are off by one band.
CaloCosmicEnergy_module.cc:186-187labels binibinas
[(ibin+1)*Erange, (ibin+1)*Erange + Erange), so bin 0 is titled "[5, 10)" while it
is filled from[0, 5). TheLR/ALRtitles at:205and:208get it right with
i * Erange, (i + 1) * Erange. - 🟡 The threshold offsets now disagree three ways within this PR: C++ defaults are
thresholdOffset 100/thresholdOffsetPin 50(BaselineAnalyzer_module.cc:73-74),
analyzeBaselines.fcl:56-57sets50/100, andNoiseAnalysis/fcl/prolog.fcl:15-16
sets100/100. Worth settling on one and saying which is right. - 🟡 Both new prologs set
writeCSV : trueandwritePDF : truewhile leaving
CSVfilenameandPDFfilenameat their empty defaults, so the flag-plus-empty-path
combination the previous review flagged is now committed rather than hypothetical. - ⚪ Collapsed:
#install_headers(SUBDIRS inc)is commented out rather than deleted, now
in both new CMakeLists, and neither package has aninc/. Both new prologs#include
minimalMessageService.fcl,standardProducers.fclandstandardServices.fcl, where
every sibling calo prolog (CaloReco,CaloMC,CaloCluster,CaloDiag) includes
only other prologs and leaves those to the job fcl — I checked the double include is
harmless, a job includingstandardServices.fclthenCosmicsCalib/fcl/prolog.fcl
dumps clean at exit 0, so this is convention rather than a defect. Every other ⚪ and
🟡 from the previous review still stands; those files are untouched in this delta.
On the open directory-structure question
@giro94 asked @bechenard on 2026-08-24 whether "50 Calo folders" meant inside
CaloCalibration/ or across Offline, and that is still unanswered — a question between
the two of you, not something I am carrying as a finding. For what it is worth, the
previous review's answer stands: the nested shape has precedent in
ExtinctionMonitorFNAL/, and the CMakeLists just added are exactly that shape.
Validation check
- Build/tests run:
mu2e/buildtestis pending at this head, so there is no result
for the new commits; a run is already queued and I did not trigger a second. The green
fromc41473e9describes code that has changed, and FNALbuild has marked it out of
date. My own checks at this head were the stubbed CMake configure andfhicl-dumpon
both new prologs. - Config contract check: partial. Every key in both new prologs exists as a
fhicl::Atomwith a default in the matchingConfig
(CaloCosmicEnecalib_module.cc:55-74,BaselineAnalyzer_module.cc:64-76), so no
required key is missing. I could not runmu2e -c, since these modules are not built
in the release I dumped against, so full art validation is unchecked. - Cross-repo consistency: needs follow-up, unchanged — see finding 4.
Residual risk
- Unchanged: I have not reviewed the physics of the calibration procedures, and nothing
in Offline runs the threeCosmicsCalibmodules, so findings 5 and 6 are reached only
by particular input values.
Author follow-ups
- Fix
@local::CaloCaloBaselineAnaatNoiseAnalysis/fcl/prolog.fcl:21. - Give each
cet_build_pluginthe list its own module needs, and confirm with a local
CMake build — the configure will not catch it, and neither will CI. - Items 3, 5 and 6 are unchanged from the previous review.
- Add fcl for
CaloCosmicEnergyandcaloT0alig(the latter as a filter), and bring
CosmicsCalib/data/across or state in the PR body that it is staying behind. - Settle the threshold-offset values across the code and the two fcl files.
|
☀️ The build tests passed at fe78246.
N.B. These results were obtained from a build of this Pull Request at fe78246 after being merged into the base branch at 8ca6473. For more information, please check the job page here. |
|
☀️ The build tests passed at 137a913.
N.B. These results were obtained from a build of this Pull Request at 137a913 after being merged into the base branch at 8ca6473. For more information, please check the job page here. |
oksuzian
left a comment
There was a problem hiding this comment.
PR Review Summary — #1936
Reviewed at head fe782469b5b62c99976e23203000c0d740db5bf3. Re-review of 0d1585fa.
Decision
- 🔴 request changes
@giro94 — this round cleared almost everything, including all four of the S1s. The one
blocker left is a single character: the prolog spells the module CaloT0Align while the
build produces CaloT0align. Everything else below is 🟡 or carried.
Delta reviewed: 7662ea46..fe782469, eight files, including the
caloT0alig → CaloT0align rename.
Findings
-
🔴 [S0]
module_type : CaloT0Aligndoes not match the plugin the build produces- Evidence: the build side is
CaloT0alignthroughout —cet_build_plugin(CaloT0align art::module)withREG_SOURCE src/CaloT0align_module.cc
(CosmicsCalib/CMakeLists.txt:25-26), andDEFINE_ART_MODULE(mu2e::CaloT0align)
(CaloT0align_module.cc:503). The prolog spells it with a capital A at
CosmicsCalib/fcl/prolog.fcl:30-31and:45. Running the two side by side on this
head:The other two agree exactly; this one differs by one character. art resolves$ cmake ... $ fhicl-dump <includes CosmicsCalib prolog> -- plugin: CaloCosmicEnecalib module_type: "CaloCosmicEnecalib" -- plugin: CaloCosmicEnergy module_type: "CaloCosmicEnergy" -- plugin: CaloT0align module_type: "CaloT0Align" <-- only mismatch
module_typeagainst the plugin/target name, not the C++ class —DAQ's
CaloDigisFromDTCEventsis the proof in this same PR: its class is
CaloDigiFromDTCEvents, singular, and the workingmodule_typeis the plural target
name. - Impact: any job using
@local::CaloT0Alignor theCosmicsCalib.filtersblock fails
at construction with a plugin-not-found error. Note this is invisible to the checks
run so far —fhicl-dumpexits 0 because the config parses fine, andbuildtest
never instantiates the module. - Suggested fix: since the rename was for capitalisation anyway, finish it — make the
file, target, class andmodule_typeallCaloT0Align, which also getsAlign
into bouncingCapitals. Renaming the prolog key down toCaloT0alignworks too, but
leaves the odd spelling.
- Evidence: the build side is
-
🟡 [S2]
fileTcoris now both the input and the outputCaloT0align_module.cc:181opens_fileTcorNamefor reading inbeginJob, and
:463opens the same_fileTcorNamefor writing inendJob. Before this delta the
output went to$OUTDIR/tcorr.dat, a different path. The atom is still documented as
an input —Comment("T0 corrections input file")at:72. Each pass of the
iteration therefore overwrites the file it read, so there is no history to compare
across iterations and a crash mid-write destroys the input. If that is deliberate,
the comment should say so; otherwise a separatefileTcorOutatom keeps the two
roles apart.
-
🟡 [S2] Two leftovers from the file-handling rework, both in
CaloT0align_module.cc:164-167: theelsebranch logging "T0 file from previous iteration not found" is
unreachable — the constructor at:122-126already throws when that file will not
open.:199-203: inside theelseofif (_fileTcor.is_open()), so the guard
if (!_fileTcor.is_open())is always true, and its message says "Cannot open output
file" for what is an input at that point.
-
🟠 [S1] Carried over, still open —
CosmicsCalib/data/t0s_allchan_1ns.datis still
not in the PR. The prolog now shipsfileT0 : "fileT0.dat"(:38) as a bare relative
name with nothing in the repo supplying it, and the constructor throws when it is
absent, so the entry as committed cannot run anywhere. Either migrate the data file and
point at it, or say in the PR body that it stays inMu2e/CaloCalibrationand the path
is the operator's to supply.
Fixed since 0d1585fa — all verified against the code, not the commit messages
- 🟢 [was S1] The zero-path division is guarded.
findpathnow initialises
float path = -1(CaloCosmicEnergy_module.cc:771) instead of0, and the only
consumer is wrapped inif (path[iCry] > 0)at:398, around the
cryDim / path[iCry]at:405. - 🟢 [was S1] Every
getenvis gone from the package —grep -rn getenvover
CaloCalibration/at this head returns nothing.CaloT0align::endJobwrites through
the fcl-supplied path and throws when it cannot open it. - 🟢 [was S1] The first-iteration blocker is resolved. The
fileTcorstream is now a
local opened insideif (_iteration == "middle" || "last")(:180-181), so
iteration : "first"no longer needs a corrections file to construct. - 🟢 [was S1]
CosmicsCalibis complete and correctly kinded:CaloCosmicEnergyjoins
analyzers, andCaloT0Aligngoes under a newfilters:block — right, since it is an
art::EDFilter. - 🟢 [was S2] The four
Config().fileX()calls are replaced by the stored_fileT0Name/
_fileTcorName, so those messages now name the file. - 🟢 [was S2] The
CryALRtitles are correct:ibin * Erange, (ibin + 1) * Erange
(:203-204), matchingLR/ALR. - 🟢 [was S2] The threshold offsets agree in all three places —
100/100at
BaselineAnalyzer_module.cc:73-74,analyzeBaselines.fcl:56-57and
NoiseAnalysis/fcl/prolog.fcl:12-13. - 🟢 [was S3]
16100is nowCaloConst::_nDIRAC*100at both sites (:139,:510); the
commented-out#install_headerslines are deleted from both CMakeLists; and both
prologs now include onlyCaloMC/CaloRecoprologs, matching the sibling calo
convention. Both still dump clean, exit 0, after dropping those includes. - 🟢 A stubbed CMake configure over the tree at this head still exits 0 and registers all
four plugins.
Withdrawn
- The previous review carried "both prologs set
writeCSV/writePDFtrue while leaving
the filenames empty". That was wrong, and I should have checked it then:
BaselineAnalyzer::beginRunderives a run-numbered name whenever either is empty
(:193-200), so the shipped combination is fine and evidently intended. The
OptionalTablepoint survives only forwriteTXT/TXTfoldername, which gets no such
treatment —:286buildsForm("%s/dirac%03d.baseline", TXTfoldername_.c_str(), board),
so an empty folder would write to the filesystem root.writeTXTisfalseeverywhere
in the PR, so nothing ships broken; it is a latent trap, ⚪ at most.
Resolved between reviewers
- @bechenard's directory-structure point is answered and explicitly non-blocking — "I
meant the latter (avoid having 50 caloXXX in Offline). This is still ok, but we should
avoid having uncontrolled growth!" I am not carrying it as a finding.
Validation check
- Build/tests run: the commit status for
mu2e/buildtestat this head is success
(09:30:33). Worth noting that FNALbuild's own result comment names137a9130, three
commits back, so the table and the status disagree about which commit was tested; the
three commits in between aredef8cc5b,b8c2b4cfandfe782469. I did not trigger
another run. My own checks at this head: the stubbed CMake configure,fhicl-dumpon
both prologs and on theCosmicsCalibblock, and the plugin-name comparison in
finding 1. - Config contract check: partial. Prolog keys match the
Configatoms for all three
CosmicsCalibmodules and forBaselineAnalyzer;module_typeis where it breaks, and
that is finding 1. Still nomu2e -crun, since these modules are not built in the
release I dumped against. - Cross-repo consistency: needs follow-up — see finding 4.
Residual risk
- Unchanged: I have not reviewed the physics of the calibration procedures. The remaining
🟡 items inBaselineAnalyzerandCaloCosmicEnecalibfrom the first review still
stand — those files are untouched apart from the_nDIRACchange.
Author follow-ups
- Make the module name agree across file,
cet_build_plugintarget, class and
module_type.CaloT0Aligneverywhere is the version I would pick. - Say whether
fileTcoris meant to be read and rewritten in place, or split the output
onto its own atom. - Drop the unreachable
elseat:164-167and the always-true guard at:199-203. - Migrate
CosmicsCalib/data/, or state in the PR body that thefileT0path is
supplied by the operator.
|
@FNALbuild run build test |
|
⌛ The following tests have been triggered for 657b9fa: build (Build queue - API unavailable) |
|
☀️ The build tests passed at 657b9fa.
N.B. These results were obtained from a build of this Pull Request at 657b9fa after being merged into the base branch at 3f6e789. For more information, please check the job page here. |
oksuzian
left a comment
There was a problem hiding this comment.
PR Review Summary — #1936
Reviewed at head 657b9fa4ccec1ad55d78b4e212579f0b01696b48. Re-review of fe782469.
Decision
- 🟡 comment only
@giro94 — the blocker is fixed. All four names now agree, and I verified it the same way
I found the mismatch. Nothing left that gates the merge; what remains is one carried S1
about migration completeness and two small cleanups in CaloT0Align_module.cc.
Delta reviewed: fe782469..657b9fa4, one commit ("fix name typo"), two files — the
CaloT0align → CaloT0Align rename of the file, the cet_build_plugin target, the class
and every cet::exception category and std::cout prefix inside it.
Fixed since fe782469
- 🟢 [was S0] The module name agrees across all four places. Running the same two commands
side by side at this head, the threeCosmicsCalibplugins now match exactly:Source of each:$ cmake ... (stubbed macros) $ fhicl-dump <CosmicsCalib prolog> -- plugin: CaloCosmicEnecalib module_type: "CaloCosmicEnecalib" -- plugin: CaloCosmicEnergy module_type: "CaloCosmicEnergy" -- plugin: CaloT0Align module_type: "CaloT0Align"CosmicsCalib/CMakeLists.txt:25-26(cet_build_plugin(CaloT0Align ...),
REG_SOURCE src/CaloT0Align_module.cc),DEFINE_ART_MODULE(mu2e::CaloT0Align)at
CaloT0Align_module.cc:503, andmodule_type : CaloT0Alignat
CosmicsCalib/fcl/prolog.fcl:31.grep -rn T0alignover the whole tree at this head
returns nothing, so no reference to the old spelling survives anywhere — including the
scons path, which derives the plugin name from the same file base name. - 🟢 The case-only rename landed cleanly in git. That is worth checking rather than
assuming: a rename differing only in case is the one git records as a rename on Linux
but can silently drop on a case-insensitive checkout, leaving both paths. The tree at
this head contains exactly one file,CosmicsCalib/src/CaloT0Align_module.cc, and the
compare API reports it asrenamedwithprevious_filenameset. - 🟢
Alignin bouncingCapitals also resolves the last of the naming nits from the first
review, where the class wascaloT0alig.
Findings still open
-
🟠 [S1] Carried over, unchanged — migration completeness for the
fileT0input.- Evidence:
CosmicsCalib/fcl/prolog.fcl:38shipsfileT0 : "fileT0.dat", a bare
relative name; there is noCosmicsCalib/data/in the PR, and nothing in the tree
provides that file. The constructor throws when it cannot be opened
(CaloT0Align_module.cc:122-126), so the entry as committed cannot run in the
directory it is launched from. - Impact: bounded. No fcl anywhere in Offline includes the
CosmicsCalibprolog
(grep -rn CosmicsCalib --include=*.fclfinds only the prolog itself), so nothing
breaks at merge and no CI job touches it — which is why this is a comment and not a
change request. The cost is that the shipped default is a value nobody can use, and
the next person to pick this up has no way to tell whether the file was forgotten or
is meant to come from outside. - Suggested fix: either migrate the data file and point
fileT0at a repo-relative
path, or say in the PR body that it stays inMu2e/CaloCalibrationand the path is
the operator's to supply. One sentence closes this.
- Evidence:
-
🟡 [S2] Carried over —
fileTcoris both the input and the output.
CaloT0Align_module.cc:181opens_fileTcorNamefor reading inbeginJob;:463
opens the same name for writing inendJob. The atom is documented as an input only —
Comment("T0 corrections input file")at:72. Each iteration overwrites the file it
read, so nothing survives to compare across iterations and a crash mid-write destroys
the input. If in-place is deliberate, say so in the comment; otherwise a separate
fileTcorOutatom keeps the two roles apart. -
🟡 [S2] Carried over — two leftovers from the file-handling rework, both still at
the same lines after the rename.:164-167: theelselogging "T0 file from previous
iteration not found" is unreachable, since the constructor at:122-126already threw.
:199-203:if (!_fileTcor.is_open())sits inside theelseof
if (_fileTcor.is_open()), so it is always true, and its message says "Cannot open
output file" for what is an input at that point. -
⚪ On lines this delta touched:
:156and:186both read<< "from file "with no
leading space, so each message renders as... invalid channel 5from file .... At
:155-156the rename also left the continuation two columns short of aligning under
the<<above it —clang-formatfixes that one.
Validation check
- Build/tests run: green, and this time it covers the delta. There was no result at
657b9fa4when I started —mu2e/buildtestread "This test has not been triggered
yet", so the earlier green described the pre-rename code — so I triggered a run and
waited for it. It passed at 15:41 (build 3309, prof, 4 min 28 sec), with the full job
list green includingceSimReco,ceMix,ceDigiandg4test_03MT. That is what
matters here, since the whole delta is a rename of a class, a target and a source file.
My own checks at this head: the stubbed CMake configure (exit 0, all four plugins
registered),fhicl-dumpon both prologs (exit 0), and the plugin-name comparison
quoted above. - Config contract check: pass. Prolog keys match the
Configatoms for all three
CosmicsCalibmodules and forBaselineAnalyzer, andmodule_typenow matches the
built plugin in all four cases — this was the one item outstanding last round. - Cross-repo consistency: needs follow-up — see finding 1.
Residual risk
- Unchanged: I have not reviewed the physics of the calibration procedures. The remaining
🟡 items inBaselineAnalyzerandCaloCosmicEnecalibfrom the first review still
stand — those files are untouched by this delta.
Author follow-ups
- Say whether
fileT0ships with the PR or is supplied by the operator. - Say whether
fileTcoris meant to be read and rewritten in place, or split the output
onto its own atom. - Drop the unreachable
elseat:164-167and the always-true guard at:199-203.
|
The "fileT0.dat" will be provided by the user and shouldn't be in the repo |
|
@FNALbuild run build test |
|
⌛ The following tests have been triggered for a37fd7c: build (Build queue - API unavailable) |
|
☀️ The build tests passed at a37fd7c.
N.B. These results were obtained from a build of this Pull Request at a37fd7c after being merged into the base branch at a432e8d. For more information, please check the job page here. |
For now, cosmics and noise modules moved over.
Source and combination will follow later.
The idea is to move here anything that must run at online level or in Pass-1/2/N steps.
Anything else can remain in the separate repo.