Fix zero-valued LER handling in validation metrics

[huaweil-nv]

Summary

Preserve valid logical error rate values of 0.0 during validation metric extraction.

Previously, LER aliases were selected using boolean or chains. Because 0.0 is falsy in Python, a perfect LER could be skipped and ultimately treated as missing.

Impact

When both X and Z validation LERs were 0.0:

• validation reported that it could not extract an LER;
• the epoch summary omitted the LER;
• checkpoint selection fell back to validation loss;
• checkpoint metadata still recorded using_ler=True.

This could prevent a zero-error model from being selected using the configured LER metric.

Validation

Verified through the documented local_run.sh training path on an NVIDIA A100:
Use a fresh experiment name to avoid loading artifacts from an earlier run:

  CUDA_VISIBLE_DEVICES=0 \
  GPUS=1 \
  FRESH_START=1 \
  EXPERIMENT_NAME=zero_ler_repro \
  PREDECODER_PYTHON=.venv/bin/python \
  PREDECODER_DECODE_OUTPUT_DIR=outputs/zero_ler_repro/decode_outputs \
  PREDECODER_TRAIN_EPOCHS=1 \
  PREDECODER_TRAIN_SAMPLES=4096 \
  PREDECODER_VAL_SAMPLES=64 \
  PREDECODER_TEST_SAMPLES=64 \
  PREDECODER_INFERENCE_NUM_WORKERS=0 \
  PREDECODER_SDR_NUM_WORKERS=0 \
  PREDECODER_DECODE_MODE=pymatching_only \
  PREDECODER_DISABLE_SDR=1 \
  PREDECODER_TORCH_COMPILE=0 \
  TORCH_COMPILE=0 \
  PREDECODER_LER_FINAL_ONLY=0 \
  bash code/scripts/local_run.sh

Before

 X: 64 shots, 0 errors, LER=0.0
 Z: 64 shots, 0 errors, LER=0.0
 [LER Validation] Warning: Could not extract LER value from result
 Best=0.654520
 checkpoint: best_vloss=0.654520, using_ler=True

After

  X: 64 shots, 0 errors, LER=0.0
  Z: 64 shots, 0 errors, LER=0.0
  [LER Validation] Logical error rate: 0.000000
  LER=0.000000
  Best=0.000000
  checkpoint: best_vloss=0.0, using_ler=True

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[ivanbasov]

Reviewed — the fix is correct and well-scoped.

What I checked:

• The _first_present_metric helper and the rewritten X/Z basis selection correctly preserve 0.0 while keeping the original key-priority order and the both-bases averaging semantics. The top-level key scan is also strictly better than before: a key explicitly set to None no longer shadows a later valid alias.
• All 13 tests in test_metrics_extras.py pass locally, plus the checkpoint-resume and training test suites.
• Swept the rest of the codebase for the same falsy-zero pattern (or-chains and truthiness checks on LER values): _extract_reduction_factor, the multi-pair aggregation and all validation_ler consumers in train.py, the CI log parsers, and inference.py are all clean (is not None throughout). The > 0 guards in threshold_plot.py are intentional (log-scale plotting).

One symptom from the description was left unfixed, so I pushed f0ea8b6 to this branch rather than requesting changes: checkpoint metadata recorded using_ler=True even on epochs where LER extraction failed and early stopping fell back to validation loss. Since resume uses that flag to decide whether the saved best_vloss is comparable, a loss value could later be restored and compared against LER values. The commit records the metric that actually produced best_vloss, and makes the resume-side check aware of PREDECODER_LER_FINAL_ONLY=1 (where the per-epoch metric is always loss) so resume in that mode keeps restoring correctly.

[ivanbasov]

/ok to test f0ea8b6

[ivanbasov]

Thanks a lot for this contribution, @huaweil-nv! Great catch on the falsy-zero or-chain — and the detailed repro with before/after logs made the review straightforward. The regression tests are much appreciated too. We'll merge once CI is green.

[ivanbasov]

LGTM. The zero-LER extraction fix is correct and complete, the codebase sweep found no other instances of the falsy-zero pattern, and the checkpoint-metadata follow-up (f0ea8b6) closes the last symptom from the description. All affected test suites pass locally.