fix(runtime): poll batches during ingestion

[AnsonDev42]

Summary

• reconcile active provider batches at durable ingestion chunk boundaries when the monotonic provider cadence is due
• reuse capacity released by completed batches before full source materialization
• preserve pause, cancel, retry-backoff, and checkpoint invariants
• bump the package metadata to 0.0.6 and update runtime documentation

Test plan

• uv run ty check src
• uv run pytest -q
• uv run mkdocs build --strict
• uv build
• focused ingestion, polling, submission, and fake-provider integration tests

Proof

Source SHA: d6a796c

Commit-bound criterion evidence will be added after the independent proof pass.

[AnsonDev42]

Proof

Tested immutable source SHA: d6a796ccbd544789253bfb8d9f960e23237c7d19 (H)

Baseline: 602be52ebe758baac53fc5fb8e322dd2eab57330

Exact deterministic flows and results

1. Focused head tests:

uv run pytest -q --no-cov \
  tests/unit/test_batchor_runtime_ingestion.py::test_ingest_job_items_polls_on_cadence_before_chunk_submission \
  tests/unit/test_batchor_runtime_ingestion.py::test_ingest_job_items_does_not_poll_without_active_batches \
  tests/unit/test_batchor_runtime_ingestion.py::test_ingest_job_items_stops_checkpointed_ingestion_when_poll_changes_control \
  tests/unit/test_batchor_runtime_ingestion.py::test_ingest_job_items_stops_checkpointed_ingestion_when_poll_records_backoff \
  tests/integration/test_batchor_runner.py::test_ingestion_poll_releases_capacity_before_source_is_fully_materialized

Result: exit 0; six cases passed (the control-state case is parameterized for pause and cancellation).

2. Bug reproduction against BASE and H: the PR's integration-test scaffold was applied without the runtime fix to a detached BASE checkout. To avoid depending on the new injectable-clock seam, the temporary test-only harness used the same 1.05-second boundary delay and the same behavioral assertion in both checkouts: a second provider batch must be created before all 2,001 prompts are materialized.

uv run pytest -q --no-cov \
  tests/integration/test_batchor_runner.py::test_ingestion_poll_releases_capacity_before_source_is_fully_materialized

BASE result: exit 1; only batch_0 was created, so the required pre-completion second submission was absent.

H result: exit 0; batch_0 and batch_1 were created, with the second creation occurring before prompt 2,001.

3. Repository gates at H:

uv run pytest -q
uv run ty check src
uv run mkdocs build --strict
uv build --out-dir <temporary-output-directory>
git diff --check 602be52ebe758baac53fc5fb8e322dd2eab57330...d6a796ccbd544789253bfb8d9f960e23237c7d19

Results:

• full pytest: exit 0; 89.89% total coverage (85% required)
• type check: exit 0, all checks passed
• strict MkDocs build: exit 0
• package build: exit 0; produced batchor-0.0.6.tar.gz and batchor-0.0.6-py3-none-any.whl
• diff check: exit 0

GitHub CI is green on H: secret scan, lint, Python 3.12/3.13/3.14 tests, package build, docs, and the Postgres contract job all completed successfully. CI checks on the immutable head

Criterion → evidence

1. Due reconciliation precedes chunk submission: the focused cadence unit test asserts the call order submit → poll → submit → submit.
2. Terminal work releases local enqueue capacity before full ingestion: the portable BASE-vs-H integration reproduction fails on BASE and passes on H; the committed deterministic-clock integration test also passes.
3. Cadence/no request per fast chunk/monotonic contract: deterministic clock test proves one poll only when elapsed time reaches the positive configured interval; the no-active-batch test proves no unnecessary poll.
4. Pause, cancellation, and retry-backoff safety: three parameterized/focused cases prove no subsequent submission, checkpoint remains incomplete, and the durable index remains at the last 1,000-row boundary. The full suite passes.
5. Docs and 0.0.6 packaging without schema change: strict docs and package builds pass; README and runtime/provider/storage design docs describe the behavior; the diff contains no storage implementation or schema change.
6. Coverage/types/runtime/provider contracts: local coverage is 89.89%, types pass, focused runtime/fake-provider evidence passes, and GitHub's multi-version plus Postgres contract checks are green on H.

Public-safety and retention

Sanitized before posting: machine-local checkout paths, virtual-environment paths, pytest temporary paths, local scratch-directory names, and local run identifiers are omitted. No credentials, DSNs, tokens, source data, or personal data were used or retained.

This PR comment is the durable evidence summary; retention owner is the repository/GitHub PR history. Temporary local test checkouts and build artifacts are disposable and are not required to interpret the retained evidence.