Skip to content

[HS3] Deduplicate HistFactory modifiers during HS3 export#22824

Merged
guitargeek merged 8 commits into
root-project:masterfrom
Phmonski:fix/hs3-canonicalize-modifiers
Jul 16, 2026
Merged

[HS3] Deduplicate HistFactory modifiers during HS3 export#22824
guitargeek merged 8 commits into
root-project:masterfrom
Phmonski:fix/hs3-canonicalize-modifiers

Conversation

@Phmonski

Copy link
Copy Markdown
Contributor

Motivation

RooFit allows a reconstructed HistFactory sample to contain multiple modifiers with the same name and type. HS3, however, requires modifiers within a sample to be unique by (name, type).

Previously, the HistFactory exporter could therefore produce invalid HS3 when serializing such RooFit workspaces.

Changes

This PR canonicalizes HistFactory modifiers before serializing a channel:

  • Duplicate normsys modifiers are combined by multiplying their lo and hi anchors.

  • Duplicate histosys modifiers are combined additively around the nominal histogram:

    combined variation = nominal + Σ(variation - nominal)
    
  • Compatible duplicates are exported as a single modifier.

  • A RooFit WARNING/IO message is emitted for every combined group, identifying the channel, sample, modifier name and type, and number of duplicates.

  • Shared constraints are queued for export only once.

  • Modifier ordering remains deterministic.

Duplicates are considered compatible only if their parameter, interpolation code, constraint metadata, and—where applicable—histogram binning agree. Incompatible duplicates produce a descriptive RooFit ERROR/IO message and throw, preventing the HistFactory object from being exported.

A final uniqueness check rejects any remaining duplicate modifier type that cannot be represented safely as a standard HS3 modifier.

Additional fixes

  • Correct RooJSONFactoryWSTool::warning() to log at RooFit::WARNING rather than RooFit::ERROR.
  • Preserve duplicate RooFit factors during modifier reconstruction so that the exporter can detect and handle them explicitly.
  • Fix the HS3 round-trip test helper, which previously evaluated the original object twice instead of evaluating the imported object.

Tests

The new tests cover:

  • Merging compatible duplicate normsys and histosys modifiers.
  • Correct multiplication of normsys anchors.
  • Correct addition of histosys deviations around the nominal histogram.
  • Warning emission at RooFit WARNING level.
  • Bin-by-bin agreement, including normalization, between the original and round-tripped channel predictions.
  • Rejection of duplicates with mismatched:
    • parameter names;
    • interpolation codes;
    • constraint names or types;
    • histogram binning.
  • Rejection of duplicate modifier types that cannot be combined safely.

The complete testRooFitHS3 test suite passes. The motivating workspace was also successfully exported with no remaining duplicate (name, type) modifier groups.

Interpolation note

For code-4 normsys modifiers, multiplying the lo and hi anchors preserves the combined response at nuisance values -1, 0, and +1. A single code-4 interpolation cannot, in general, exactly reproduce the product of multiple interpolations between those anchors.

@cburgard @stalbrec

@github-actions

Copy link
Copy Markdown

Test Results

    23 files      23 suites   3d 11h 21m 5s ⏱️
 3 873 tests  3 872 ✅ 0 💤 1 ❌
76 181 runs  76 180 ✅ 0 💤 1 ❌

For more details on these failures, see this check.

Results for commit cdff5c1.

@guitargeek guitargeek self-assigned this Jul 16, 2026
@guitargeek
guitargeek force-pushed the fix/hs3-canonicalize-modifiers branch 2 times, most recently from 9568c75 to 393df87 Compare July 16, 2026 10:27
guitargeek and others added 8 commits July 16, 2026 12:31
Also fixes RooJSONFactoryWSTool::warning() to log at WARNING level
instead of ERROR, and a self-comparison bug in the test validate()
helper.
Extract the RooPoisson/RooGaussian constraint dispatch that was duplicated
in the Barlow-Beeston (mc-stat) and shapesys branches of readChannel into a
single constraintRelError() helper.
Replace the hand-rolled character-by-character number parser with a
std::istringstream extraction that requires the whole string to be
consumed. This mirrors the parsing done by toDouble(), so isNumber(s) is
true exactly when toDouble(s) can convert the entire string.

std::from_chars for floating-point types is not portably available on all
platforms ROOT supports, so the stream extraction is used instead.
- Drop the pointless static_cast<RooSimultaneous*> around workspace.pdf()
  in importAnalysis: the result is stored as RooAbsPdf* anyway, and the
  cast is undefined behaviour when the pdf is not a RooSimultaneous.
- Use dynamic_cast with a null check instead of an unchecked cast + deref
  in importDecay (resolutionModel) and importBinWidthFunction (histogram),
  so malformed input raises a clear error instead of crashing.
- Remove the file-global 'verbose' flag in JSONFactories_HistFactory that
  was never set to true, along with its two dead diagnostic blocks.
- Remove the unused <mutex> include in JSONIO.cxx.
- Remove an unused local and two commented-out lines.
Replace the verbose Doxygen blocks on the file-local (anonymous-namespace)
helpers and the private member functions of RooJSONFactoryWSTool with
concise one-line comments. Full Doxygen is kept only for the user-facing
public API declared in RooJSONFactoryWSTool.h.

Also move the exportCategory() doc block, which had become detached from
its function, back onto it.
@guitargeek
guitargeek force-pushed the fix/hs3-canonicalize-modifiers branch from 393df87 to fa04120 Compare July 16, 2026 10:32

@guitargeek guitargeek left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

I have also added a few cleanup commits at the same time, to ensure the codebase doesn't grow too much.

@guitargeek
guitargeek merged commit afe0d33 into root-project:master Jul 16, 2026
31 of 32 checks passed
@guitargeek

Copy link
Copy Markdown
Contributor

/backport to 6.40

@root-project-bot

Copy link
Copy Markdown

Preparing to backport PR #22824 to branch 6.40 requested by guitargeek

@root-project-bot

Copy link
Copy Markdown

This PR has been backported to branch 6.40: #22836

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants