Skip to content

Remove non-finite SSFT noise in StochasticNoise#2400

Merged
mo-AliceLake merged 4 commits into
masterfrom
fix_sporadic_unit_test
Jun 19, 2026
Merged

Remove non-finite SSFT noise in StochasticNoise#2400
mo-AliceLake merged 4 commits into
masterfrom
fix_sporadic_unit_test

Conversation

@mo-AliceLake

Copy link
Copy Markdown
Contributor

Addresses https://github.com/metoppv/mo-blue-team/issues/1220

Stochastic noise generation can occasionally produce non-finite values (NaN/Inf), which then propagate into output. This leads to sporadic failures in stochastic-noise tests (see e.g. example 1, example 2).

To fix this, updated improver/calibration/stochastic_noise.py to guard against non-finite values:

  • In _from_dB: treat non-finite linear values as below-threshold by setting them to 0.0 before threshold masking.
  • In _process_single_realization: replace any remaining non-finite values in noise_linear with 0.0 before scaling etc.

Updated improver_tests/calibration/stochastic_noise/test_StochasticNoise.py:

  • Added test_process_non_finite_noise_is_sanitized
  • Test forces do_fft output to include NaN/Inf and verifies:
    • Non-positive-region output is finite
    • Positive-region values remain unchanged
    • With scaling enabled, non-positive-region values are <= 0

Testing:

  • Ran tests and they passed OK
  • Added new tests for the new feature(s)

MoseleyS
MoseleyS previously approved these changes Jun 18, 2026

@MoseleyS MoseleyS left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good

@brhooper brhooper left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @mo-AliceLake

This change should solve our issues with non-finite values being generated and the new functional is covered by a new unit test. I've added one comment to fix a typo, then I think that this can be merged.

Comment thread improver/calibration/stochastic_noise.py Outdated
Co-authored-by: Ben Hooper <114418734+brhooper@users.noreply.github.com>

@brhooper brhooper left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. Thanks @mo-AliceLake .

@mo-AliceLake mo-AliceLake merged commit f594a07 into master Jun 19, 2026
7 checks passed
@mo-AliceLake mo-AliceLake deleted the fix_sporadic_unit_test branch June 19, 2026 09:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants