Skip to content

test(vector/hnsw): stabilize incremental_append_recall_matches_fresh_build with best-of-3 (#886)#899

Merged
mosuka merged 1 commit into
mainfrom
test/886-hnsw-flaky-recall-gate
Jul 26, 2026
Merged

test(vector/hnsw): stabilize incremental_append_recall_matches_fresh_build with best-of-3 (#886)#899
mosuka merged 1 commit into
mainfrom
test/886-hnsw-flaky-recall-gate

Conversation

@mosuka

@mosuka mosuka commented Jul 25, 2026

Copy link
Copy Markdown
Owner

Summary

  • incremental_append_recall_matches_fresh_build (the perf(vector/hnsw): incremental append onto a small base uses a low-level entry point, degrading concurrent-build quality (seed-then-bulk-load) #872 base>=ef_construction regression gate) occasionally failed on low-core CI runners because the incremental regime runs a structurally different code path from the fresh build (incremental append via promoted_ep vs. one-shot construction), so their variance doesn't cancel the way the rebuild regime (base=1) does.
  • Parametrized assert_append_matches_fresh with cold_trials and take the max self-recall@10 across independent seed_then_append builds, applied to the incremental regime only (cold_trials=3); the rebuild regime keeps cold_trials=1 (unchanged — not observed to be flaky).

Why best-of-N, not median or a wider tolerance

Full investigation and design rationale posted to #886 as comments before this PR.

Verification

Test plan

Closes #886
Refs #872, #885

…build with best-of-3 (#886)

The incremental regime (base=200) runs a structurally different code
path from the fresh build (incremental append via promoted_ep vs.
one-shot construction), so their variance doesn't cancel like the
rebuild regime (base=1) does. Rare unlucky parallel-insertion orderings
under parallel_build could dip a single build's self-recall@10 into a
tail below the fixed tolerance on low-core CI runners.

Parametrize assert_append_matches_fresh with cold_trials and take the
max self-recall@10 across independent seed_then_append builds, applied
to the incremental regime only (cold_trials=3); the rebuild regime
keeps cold_trials=1 (unchanged, not observed to be flaky).

Verified the mitigation cannot mask a real regression: disabling the
#872 promoted_ep fix under RAYON_NUM_THREADS=2 reproduces an identical
failing self-recall@10 (~0.83) on every trial, since the pre-#872
defect is a deterministic structural funneling through old_ep, not a
scheduling fluke. With the fix enabled, 20 consecutive release-mode
runs under RAYON_NUM_THREADS=2 all pass.

Closes #886
Refs #872, #885
@mosuka
mosuka merged commit 876bcaf into main Jul 26, 2026
22 checks passed
@mosuka
mosuka deleted the test/886-hnsw-flaky-recall-gate branch July 26, 2026 01:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test(vector/hnsw): incremental_append_recall_matches_fresh_build is flaky on low-core CI runners

1 participant