Tracking issue for the correctness fixes to itk::JointHistogramMutualInformationImageToImageMetricv4 (and the related ScaleLogarithmicTransform Jacobian) proposed in #6569, collecting the independent verification, downstream-forest impact, empirical before/after evidence, and the remaining baseline-regeneration work in one place.
These are bug fixes, not enhancements. The previous metric value and derivative were correct only by coincidence — on symmetric marginals (an image registered to a shifted/rotated copy of itself, i.e. mono-modal) — and wrong for any asymmetric image pair. That coincidence is exactly why the existing tests, which all use near-symmetric fixtures, never caught it.
Status: fixes verified correct; baselines regenerated locally; downstream consumers identified; empirical improvement demonstrated on BRAINSTools. Remaining work tracked in the checklist below.
Related: #6569 (the fix) · #6571 (the independent ScaleLogarithmicTransform half) · migration-guide companion doc Documentation/docs/migration_guides/joint_histogram_mutual_information_metric_correction.md.
This issue is the focal point for JointHistogram-MI test/CI results — please report ITK CI, downstream (SimpleITK / BRAINSFit / ANTs) verification, and baseline outcomes here.
The three defects (all confirmed against source)
-
ScaleLogarithmicTransform Jacobian ignored the center. ScaleTransform differentiates about the center (j(dim,dim) = p[dim] - center[dim]); the Log(scale) chain rule gives scale[dim] * (p[dim] - center[dim]). The old scale[dim] * p[dim] dropped the center. Independent of the metric fixes; ships its own test; no baseline churn.
-
Marginal PDFs swapped. ComputeJointPDFPoint writes [0]=fixed, [1]=moving, so m_JointPDF(ii,jj)=p(fixed=ii,moving=jj). GetValue() reads px=m_FixedImageMarginalPDF[ii], py=m_MovingImageMarginalPDF[jj]. SetDirection(0) sums over the fixed axis → yields p_moving, which the old code stored into m_FixedImageMarginalPDF (and vice-versa). Value is wrong whenever the two intensity marginals differ; invisible for symmetric pairs (image vs a shifted/rotated copy of itself), which is why no test caught it.
-
Derivative was not an MI gradient. New per-sample weight (dJPDF/J − dMmPDF/Pm)/log2 is exactly ∂/∂b · ln(J/(pf·pm)) (the pf term drops since it is independent of the moving intensity) — the standard Viola & Wells Parzen-window form. The old log2·dMmPDF·J/Pm − dJPDF·(lnJ − lnPm) is not the differential of any MI estimate.
Coupling: the derivative threader reads m_MovingImageMarginalPDF for Pm/dMmPDF, so fix (3) requires fix (2). These two must land together; fix (1) is independent.
Sign/units confirmation (a reviewer cannot safely re-derive in isolation because ComputeValue returns −MI/log2): the corrected derivative's cosine with the independently-correct Mattes derivative is +0.95–0.99, and it converges to the true shift [2,5] (old formula ~35% off in x).
Downstream forest impact
No public signature changes — everything compiles unchanged; only numerics shift (toward the true optimum). Only workflows that explicitly select JointHistogram MI are affected; Mattes/CC/MSQ/Demons are untouched.
- SimpleITK —
SetMetricAsJointHistogramMutualInformation() maps to the affected class. Testing/Unit/sitkImageRegistrationMethodTests.cxx pins EXPECT_NEAR(-0.52624100016564002, MetricEvaluate, 2e-6); that pair is asymmetric, so the constant needs updating. Examples ImageRegistrationMethod2/BSpline3/Displacement1 shift. Follow-up SimpleITK change required after this lands.
- BRAINSTools — BRAINSFit
--costMetric MIH → the affected class. Three baseline tests: BRAINSFitTest_MIHAffineRotationMasks, BRAINSFitTest_MIHScaleSkewVersorRotationMasks, BRAINSFitTest_MIHMetricBrainToItself. Default MMI (Mattes) and NMIH (v3 class) are unaffected.
- ANTs —
antsRegistration --metric MI[...] builds the affected class (Mattes does not). ANTs' CTest suite only exercises the legacy ANTS binary (own internal MI) → not affected. Pipeline scripts (antsRegistrationSyNQuick.sh, antsBrainExtraction.sh, antsCorticalThickness.sh, antsMultivariateTemplateConstruction2.sh, antsJointLabelFusion2.sh) drift (improve); no pinned baselines.
- Not affected — elastix/ITKElastix (own
AdvancedMattesMutualInformation); teem/VTK/DCMTK; ScaleLogarithmicTransform (no optimizing consumer).
Empirical before/after on BRAINSTools MIH tests
BRAINSFit built against ITKv6 main (before) vs the same with the metric fix (after); all three affected MIH tests run single-threaded, accuracy measured against independent ground truth (a MeanSquares reference transform for the rotation pairs, identity for brain-to-itself):
| Test |
Ground truth |
BEFORE |
AFTER |
Result |
| MIHScaleSkewVersorRotationMasks |
MSE-ref TRE |
0.825 mm |
0.271 mm |
AFTER 3× more accurate |
| MIHMetricBrainToItself |
identity |
0.523° rot |
0.256° rot |
AFTER 2× more accurate |
| MIHAffineRotationMasks |
MSE-ref TRE |
0.098 mm |
0.162 mm |
both sub-voxel; negligible |
More accurate on 2 of 3 (3× and 2×), statistically tied (both sub-voxel-perfect) on the well-conditioned Affine case; no regression. The gains concentrate where the problem is hardest (ScaleSkewVersor's 9 DOF is the most gradient-sensitive). On the Affine run BEFORE stopped at iteration 11, AFTER pushed to 17 — consistent with the old derivative vanishing near the optimum. Mono-modal fixtures are the least sensitive to the marginal fix by construction; asymmetric/cross-modal problems will show larger differences.
Baseline regeneration status
itkSimpleImageRegistrationTest{Float,Double} fail against the old baselines (they encode the old metric — new vs old differs in 13,426 pixels). Regenerated locally on the metric fix:
- new-float vs new-double differ in 98 pixels (> the 8-pixel tolerance) → separate baselines: primary = double,
.1 = float; stale .2/.3 removed. Both tests then pass.
- CID computation validated against the existing content link.
- macOS arm64 outputs; Linux/Windows CI may need additional float alternates.
Actual data + content links prepared for upload to the ITK data store.
Recommended migration-guide documentation (companion entry)
The following is the proposed entry for Documentation/docs/migration_guides/, alongside the ITK v6 migration guide. It is a correctness change, not an API change, so no code must be edited to compile — but registration results computed with the affected metric change (toward the true optimum), and tests that pin the old numbers must be re-baselined.
JointHistogramMutualInformation Metric Correction — Downstream Impact
Summary. Three correctness defects are corrected: (1) the ScaleLogarithmicTransform Jacobian now uses scale[i]*(p[i]-center[i]); (2) the JointHistogramMutualInformation marginal PDFs, previously swapped, are stored on the correct axes; (3) the metric derivative now uses the standard Parzen-window (Viola & Wells) weight (dJPDF/J − dMmPDF/Pm)/log2. Fixes (2) and (3) are coupled (the derivative reads the moving marginal); fix (1) is independent.
Who is affected. Only workflows that explicitly select the JointHistogram MI metric. The Mattes MI metric, correlation, mean-squares, and demons metrics are untouched. No public signatures change.
- ITK —
itkSimpleImageRegistrationTest{Float,Double} baselines must be regenerated; the metric and convergence tests pass unchanged.
- SimpleITK —
SetMetricAsJointHistogramMutualInformation(); the pinned value in sitkImageRegistrationMethodTests.cxx and the ImageRegistrationMethod2/BSpline3/Displacement1 examples shift (follow-up SimpleITK change).
- BRAINSTools — BRAINSFit
--costMetric MIH (three baseline tests). Default MMI (Mattes) and NMIH (a v3 class) are unaffected.
- ANTs —
antsRegistration --metric MI[...] (the legacy ANTS binary and Mattes metric are unaffected). Pipeline-script outputs drift toward better alignment; no pinned baselines.
- Not affected — elastix/ITKElastix, teem/VTK/DCMTK,
ScaleLogarithmicTransform (no optimizing consumer).
Re-baselining checklist. ITK itkSimpleImageRegistrationTest{Float,Double}; SimpleITK unit constant + examples; BRAINSFit MIH{Affine,ScaleSkewVersor,BrainToItself} result images (only if tolerance is breached).
Verification note. The correction moves results toward the true optimum: the corrected derivative agrees in direction with ITK's independently-correct Mattes metric, and the metric value's argmin is unchanged (only the gradient that reached it was wrong). Downstream baselines that shift encode the previous, incorrect behavior and should be regenerated rather than treated as regressions.
The full draft lives on branch doc-jhmi-metric-behavior-change at Documentation/docs/migration_guides/joint_histogram_mutual_information_metric_correction.md (linked into migration_guides/index.md).
release-5.4 backport consideration
Higher-risk than the ScaleLogarithmicTransform fix (#6571). These are genuine correctness fixes, but backporting to release-5.4 changes registration outputs for any user of the JointHistogram MI metric on that stable line, and would require regenerating the ITK (and any pinned downstream) baselines on 5.4. The trade-off is correctness vs. reproducibility on a released branch:
- For: 5.4 users of
JointHistogramMutualInformation / BRAINSFit MIH / antsRegistration MI are currently getting a wrong gradient (and, for asymmetric pairs, a wrong value).
- Against: results shift for existing 5.4 pipelines; anyone who tuned parameters or stored results against 5.4 behavior sees a change in a patch release.
Recommendation: land on main (ITK 6) first with regenerated baselines; decide the 5.4 backport separately, and if taken, carry the baseline regeneration with it. Note the fixes are coupled — a 5.4 backport must include both the marginal and derivative commits together.
Recommendation: stronger tests for the future
The bugs survived for years because every test sits in the coincidence regime (symmetric marginals, well-initialized, easy problems). New coverage should deliberately leave that regime:
- Asymmetric marginals — register a fixed image against a non-monotonic intensity remap of itself (e.g. moving = cube of fixed), where
p_fixed ≠ p_moving. The marginal swap changes the value here; a value assertion catches it. (This is the probe used to prove the bug from GetJointPDF().)
- Cross-modal pair — a true multi-modal fixture (T1↔T2-like), the intended use of MI.
- Analytic value check — assert
GetValue() against a hand-computed MI for a tiny fixed joint histogram, independent of the internal marginal code.
- Gradient consistency — assert the derivative agrees with a finite-difference of
GetValue() (where the estimator admits it) and/or is directionally consistent with the Mattes metric on the same pair.
- Off-optimum convergence — start far from the solution with a higher-DOF transform (affine/ScaleSkewVersor), where a wrong gradient converges to the wrong place (the failure mode that only surfaced at 35% error, not the ~symmetric fixtures' near-zero error).
Checklist:
Tracking issue for the correctness fixes to
itk::JointHistogramMutualInformationImageToImageMetricv4(and the relatedScaleLogarithmicTransformJacobian) proposed in #6569, collecting the independent verification, downstream-forest impact, empirical before/after evidence, and the remaining baseline-regeneration work in one place.These are bug fixes, not enhancements. The previous metric value and derivative were correct only by coincidence — on symmetric marginals (an image registered to a shifted/rotated copy of itself, i.e. mono-modal) — and wrong for any asymmetric image pair. That coincidence is exactly why the existing tests, which all use near-symmetric fixtures, never caught it.
Status: fixes verified correct; baselines regenerated locally; downstream consumers identified; empirical improvement demonstrated on BRAINSTools. Remaining work tracked in the checklist below.
Related: #6569 (the fix) · #6571 (the independent
ScaleLogarithmicTransformhalf) · migration-guide companion docDocumentation/docs/migration_guides/joint_histogram_mutual_information_metric_correction.md.This issue is the focal point for JointHistogram-MI test/CI results — please report ITK CI, downstream (SimpleITK / BRAINSFit / ANTs) verification, and baseline outcomes here.
The three defects (all confirmed against source)
ScaleLogarithmicTransformJacobian ignored the center.ScaleTransformdifferentiates about the center (j(dim,dim) = p[dim] - center[dim]); theLog(scale)chain rule givesscale[dim] * (p[dim] - center[dim]). The oldscale[dim] * p[dim]dropped the center. Independent of the metric fixes; ships its own test; no baseline churn.Marginal PDFs swapped.
ComputeJointPDFPointwrites[0]=fixed,[1]=moving, som_JointPDF(ii,jj)=p(fixed=ii,moving=jj).GetValue()readspx=m_FixedImageMarginalPDF[ii],py=m_MovingImageMarginalPDF[jj].SetDirection(0)sums over the fixed axis → yields p_moving, which the old code stored intom_FixedImageMarginalPDF(and vice-versa). Value is wrong whenever the two intensity marginals differ; invisible for symmetric pairs (image vs a shifted/rotated copy of itself), which is why no test caught it.Derivative was not an MI gradient. New per-sample weight
(dJPDF/J − dMmPDF/Pm)/log2is exactly∂/∂b · ln(J/(pf·pm))(thepfterm drops since it is independent of the moving intensity) — the standard Viola & Wells Parzen-window form. The oldlog2·dMmPDF·J/Pm − dJPDF·(lnJ − lnPm)is not the differential of any MI estimate.Coupling: the derivative threader reads
m_MovingImageMarginalPDFforPm/dMmPDF, so fix (3) requires fix (2). These two must land together; fix (1) is independent.Sign/units confirmation (a reviewer cannot safely re-derive in isolation because
ComputeValuereturns−MI/log2): the corrected derivative's cosine with the independently-correct Mattes derivative is +0.95–0.99, and it converges to the true shift[2,5](old formula ~35% off in x).Downstream forest impact
No public signature changes — everything compiles unchanged; only numerics shift (toward the true optimum). Only workflows that explicitly select JointHistogram MI are affected; Mattes/CC/MSQ/Demons are untouched.
SetMetricAsJointHistogramMutualInformation()maps to the affected class.Testing/Unit/sitkImageRegistrationMethodTests.cxxpinsEXPECT_NEAR(-0.52624100016564002, MetricEvaluate, 2e-6); that pair is asymmetric, so the constant needs updating. ExamplesImageRegistrationMethod2/BSpline3/Displacement1shift. Follow-up SimpleITK change required after this lands.--costMetric MIH→ the affected class. Three baseline tests:BRAINSFitTest_MIHAffineRotationMasks,BRAINSFitTest_MIHScaleSkewVersorRotationMasks,BRAINSFitTest_MIHMetricBrainToItself. DefaultMMI(Mattes) andNMIH(v3 class) are unaffected.antsRegistration --metric MI[...]builds the affected class (Mattesdoes not). ANTs' CTest suite only exercises the legacyANTSbinary (own internal MI) → not affected. Pipeline scripts (antsRegistrationSyNQuick.sh,antsBrainExtraction.sh,antsCorticalThickness.sh,antsMultivariateTemplateConstruction2.sh,antsJointLabelFusion2.sh) drift (improve); no pinned baselines.AdvancedMattesMutualInformation); teem/VTK/DCMTK;ScaleLogarithmicTransform(no optimizing consumer).Empirical before/after on BRAINSTools MIH tests
BRAINSFit built against ITKv6 main (before) vs the same with the metric fix (after); all three affected MIH tests run single-threaded, accuracy measured against independent ground truth (a MeanSquares reference transform for the rotation pairs, identity for brain-to-itself):
More accurate on 2 of 3 (3× and 2×), statistically tied (both sub-voxel-perfect) on the well-conditioned Affine case; no regression. The gains concentrate where the problem is hardest (ScaleSkewVersor's 9 DOF is the most gradient-sensitive). On the Affine run BEFORE stopped at iteration 11, AFTER pushed to 17 — consistent with the old derivative vanishing near the optimum. Mono-modal fixtures are the least sensitive to the marginal fix by construction; asymmetric/cross-modal problems will show larger differences.
Baseline regeneration status
itkSimpleImageRegistrationTest{Float,Double}fail against the old baselines (they encode the old metric — new vs old differs in 13,426 pixels). Regenerated locally on the metric fix:.1= float; stale.2/.3removed. Both tests then pass.Actual data + content links prepared for upload to the ITK data store.
Recommended migration-guide documentation (companion entry)
The following is the proposed entry for
Documentation/docs/migration_guides/, alongside the ITK v6 migration guide. It is a correctness change, not an API change, so no code must be edited to compile — but registration results computed with the affected metric change (toward the true optimum), and tests that pin the old numbers must be re-baselined.JointHistogramMutualInformation Metric Correction — Downstream Impact
Summary. Three correctness defects are corrected: (1) the
ScaleLogarithmicTransformJacobian now usesscale[i]*(p[i]-center[i]); (2) theJointHistogramMutualInformationmarginal PDFs, previously swapped, are stored on the correct axes; (3) the metric derivative now uses the standard Parzen-window (Viola & Wells) weight(dJPDF/J − dMmPDF/Pm)/log2. Fixes (2) and (3) are coupled (the derivative reads the moving marginal); fix (1) is independent.Who is affected. Only workflows that explicitly select the JointHistogram MI metric. The Mattes MI metric, correlation, mean-squares, and demons metrics are untouched. No public signatures change.
itkSimpleImageRegistrationTest{Float,Double}baselines must be regenerated; the metric and convergence tests pass unchanged.SetMetricAsJointHistogramMutualInformation(); the pinned value insitkImageRegistrationMethodTests.cxxand theImageRegistrationMethod2/BSpline3/Displacement1examples shift (follow-up SimpleITK change).--costMetric MIH(three baseline tests). DefaultMMI(Mattes) andNMIH(a v3 class) are unaffected.antsRegistration --metric MI[...](the legacyANTSbinary andMattesmetric are unaffected). Pipeline-script outputs drift toward better alignment; no pinned baselines.ScaleLogarithmicTransform(no optimizing consumer).Re-baselining checklist. ITK
itkSimpleImageRegistrationTest{Float,Double}; SimpleITK unit constant + examples; BRAINSFitMIH{Affine,ScaleSkewVersor,BrainToItself}result images (only if tolerance is breached).Verification note. The correction moves results toward the true optimum: the corrected derivative agrees in direction with ITK's independently-correct Mattes metric, and the metric value's argmin is unchanged (only the gradient that reached it was wrong). Downstream baselines that shift encode the previous, incorrect behavior and should be regenerated rather than treated as regressions.
The full draft lives on branch
doc-jhmi-metric-behavior-changeatDocumentation/docs/migration_guides/joint_histogram_mutual_information_metric_correction.md(linked intomigration_guides/index.md).release-5.4 backport consideration
Higher-risk than the
ScaleLogarithmicTransformfix (#6571). These are genuine correctness fixes, but backporting torelease-5.4changes registration outputs for any user of the JointHistogram MI metric on that stable line, and would require regenerating the ITK (and any pinned downstream) baselines on 5.4. The trade-off is correctness vs. reproducibility on a released branch:JointHistogramMutualInformation/ BRAINSFitMIH/antsRegistration MIare currently getting a wrong gradient (and, for asymmetric pairs, a wrong value).Recommendation: land on
main(ITK 6) first with regenerated baselines; decide the 5.4 backport separately, and if taken, carry the baseline regeneration with it. Note the fixes are coupled — a 5.4 backport must include both the marginal and derivative commits together.Recommendation: stronger tests for the future
The bugs survived for years because every test sits in the coincidence regime (symmetric marginals, well-initialized, easy problems). New coverage should deliberately leave that regime:
p_fixed ≠ p_moving. The marginal swap changes the value here; a value assertion catches it. (This is the probe used to prove the bug fromGetJointPDF().)GetValue()against a hand-computed MI for a tiny fixed joint histogram, independent of the internal marginal code.GetValue()(where the estimator admits it) and/or is directionally consistent with the Mattes metric on the same pair.Checklist:
ScaleLogarithmicTransformJacobian fix separately (no baseline churn); keep the coupled JHMI marginal+derivative fixes togetheritkSimpleImageRegistrationTest{Float,Double}baselines to the ITK data store; add Linux/Windows float alternates if CI needs themsitkImageRegistrationMethodTests.cxxJHMI constant and refresh the affected examples (follow-up SimpleITK PR)MIHtests in CI on the new metric; regenerate*.result.nii.gzif tolerance is breachedrelease-5.4backport (higher-risk; changes outputs + requires baseline regen on 5.4; both coupled commits together)