fix(toc_extractor): LLM refiner returning empty tuple silently demotes document to Scenario C with no log #249

Summary

When the LLM refiner returns an empty tuple, _derive_page_ranges([]) returns () (falsy), and the Scenario B path falls through to Scenario C (fallback_only=True) with no exception, no WARNING log, and no sections persisted. Any refiner bug that returns an empty result causes silent data loss — the document appears indexed but returns zero results.

Root Cause

ingestion/toc_extractor.py around line 307:

raw = llm_refiner.refine(raw, ...)
heuristic = _derive_page_ranges(raw, total_pages)  # () when raw is []
if heuristic:        # ← False when refiner returns empty
    return TocResult(entries=heuristic, scenario='B', fallback_only=False)
# falls through to Scenario C silently:
return TocResult(entries=(), scenario='C', fallback_only=True)

There is no log entry when a non-empty raw list is passed to the refiner and an empty result comes back — a complete structural erasure is indistinguishable from a document that genuinely had no structure.

Failure Scenario

1. LLM refiner has a bug and returns () for a document with 15 clear heuristic headers.
2. _derive_page_ranges([], total_pages) returns ().
3. if heuristic: → False.
4. TocResult(entries=(), scenario='C', fallback_only=True) returned.
5. upsert_document() called with fallback_only=True, no section rows written.
6. 15 sections of content permanently invisible in search. No error in logs.

Proposed Fix

raw_after_refine = llm_refiner.refine(raw, ...)
if not raw_after_refine and raw:
    logger.warning(
        "LLM refiner erased all %d heuristic entries; falling back to Scenario C",
        len(raw),
        extra={"doc_id": doc_id, "original_count": len(raw)},
    )
heuristic = _derive_page_ranges(raw_after_refine, total_pages)

Add a metric counter toc_refiner_empty_result_total to make this visible in Prometheus dashboards.