Skip to content

test: eliminate flaky time.sleep() tests with deterministic coordination#115

Merged
27Bslash6 merged 1 commit into
mainfrom
test/eliminate-flaky-sleep-tests
May 16, 2026
Merged

test: eliminate flaky time.sleep() tests with deterministic coordination#115
27Bslash6 merged 1 commit into
mainfrom
test/eliminate-flaky-sleep-tests

Conversation

@27Bslash6
Copy link
Copy Markdown
Contributor

@27Bslash6 27Bslash6 commented May 16, 2026

Summary

  • Adds ThreadGate helper to tests/utils/timing_helper.py — named synchronization points for deterministic thread coordination
  • Replaces 26 flaky time.sleep() calls across 5 test files with deterministic primitives
  • Fixes test_version_mismatch_aborts_refresh_concurrent flake blocking release-please PR chore(main): release 0.6.1 #112

Changes by file

File Before After
tests/unit/test_l1_swr.py patch("time.time") + sleep-based thread ordering time-machine + threading.Event
tests/unit/test_load_control.py time.sleep(0.05) "let thread acquire" ThreadGate + wait_for_condition
tests/critical/test_backpressure_load_control.py time.sleep(0.05) "let it acquire" ThreadGate + wait_for_condition
tests/unit/test_metrics_collection.py time.sleep(0.1) "allow time for processing" collector.flush() (existing API)
tests/critical/conftest.py backpressure tests required Redis setup skipped (they don't use Redis)

What was NOT changed

  • time.sleep inside thread callbacks simulating real work (valid usage)
  • Integration tests waiting for real Redis/memcached TTL expiry
  • Performance tests measuring real elapsed time

Test plan

  • All 1400 unit tests pass
  • All 11 critical backpressure tests pass
  • test_version_mismatch_aborts_refresh_concurrent passes 50/50 in tight loop
  • Ruff lint + format clean
  • basedpyright passes
  • CI passes on this PR
  • Re-run release-please PR chore(main): release 0.6.1 #112 checks after merge

Summary by CodeRabbit

  • Tests
    • Improved test reliability by replacing timing-dependent sleeps with deterministic thread coordination across backpressure control, cache revalidation, and metrics collection tests.
    • Reduced flakiness in critical load control and race condition testing scenarios.
    • Enhanced test infrastructure with new utilities for more robust concurrent testing.

Review Change Stack

Replaces 26 flaky time.sleep() calls across 5 test files with
deterministic synchronization primitives:

- ThreadGate: named sync points for thread coordination (new helper)
- TimingHelper.wait_for_condition: poll observable state instead of
  hoping sleep durations are sufficient
- collector.flush(): use existing drain API instead of sleeping
- time-machine: replace patch("time.time") for thread-safe clock
  control (test_l1_swr.py)

Fixes the test_version_mismatch_aborts_refresh_concurrent flake that
blocks the release-please PR (#112).
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 16, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 95480ddb-8120-4054-89c2-81b90bcebabd

📥 Commits

Reviewing files that changed from the base of the PR and between f51d3b1 and 5c5ee92.

📒 Files selected for processing (6)
  • tests/critical/conftest.py
  • tests/critical/test_backpressure_load_control.py
  • tests/unit/test_l1_swr.py
  • tests/unit/test_load_control.py
  • tests/unit/test_metrics_collection.py
  • tests/utils/timing_helper.py

📝 Walkthrough

Walkthrough

This PR reduces test timing flakiness by replacing sleep-based delays with deterministic synchronization. A new ThreadGate utility enables explicit thread coordination in load control tests; time_machine controls time in stale-while-revalidate tests; and collector.flush() drains background queues in metrics tests.

Changes

Test Timing Determinism

Layer / File(s) Summary
Thread coordination infrastructure
tests/utils/timing_helper.py
New ThreadGate class manages named synchronization gates via signal(name) and wait(name, timeout). TimingHelper.wait_for_condition adds configurable polling interval parameter.
Backpressure and load control test determinism
tests/critical/conftest.py, tests/critical/test_backpressure_load_control.py, tests/unit/test_load_control.py
Backpressure controller tests in both critical and unit suites replace sleep-based blocking with ThreadGate signaling and TimingHelper.wait_for_condition polling. Redis fixtures are skipped for backpressure tests. Tests coordinate permit acquisition and queue formation deterministically.
L1 Cache stale-while-revalidate test determinism
tests/unit/test_l1_swr.py
SWR tests use time_machine.travel() for deterministic time control instead of mocking time.time. Critical race condition test adds threading.Event coordination between refresher and invalidator threads to order operations deterministically.
Metrics collection test determinism
tests/unit/test_metrics_collection.py
Metrics tests replace sleep delays with collector.flush() calls to ensure background queue draining before assertions, removing timing dependencies across counter, histogram, gauge, and Prometheus integration tests.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Poem

🐰 Sleeps and timers, gone away,
Gates and signals lead the way,
Time machines spin backward, forward bright,
Tests now dance without the fight!
Determinism blooms, no more delay—
Flakiness has had its day! 🎉

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely describes the main change: eliminating flaky time.sleep() tests by implementing deterministic thread coordination.
Description check ✅ Passed The description covers motivation, detailed changes by file, scope clarification, and test results. All major required sections from the template are addressed with substantive content.
Docstring Coverage ✅ Passed Docstring coverage is 82.98% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch test/eliminate-flaky-sleep-tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@27Bslash6 27Bslash6 merged commit ad8b048 into main May 16, 2026
32 checks passed
@27Bslash6 27Bslash6 deleted the test/eliminate-flaky-sleep-tests branch May 16, 2026 05:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant