fix(pipeline): no-retention upload NULLs content_hash, permanently breaking dedup for that document #241

Summary

When a document is re-uploaded with no_retention=True, upsert_document() is called with content_hash_value=None, which writes NULL over the previously stored SHA-256 hash. Any future normal upload of the same bytes can no longer find its dedup hash, causing a full re-ingestion and orphaned Qdrant points.

Root Cause

ingestion/pipeline.py, the else branch for no_retention=True (around line 488):

# no_retention path — skips hash lookup AND clears the stored hash:
await upsert_document(
    ...,
    content_hash_value=None,   # ← overwrites existing SHA-256 with NULL
    ...
)

The no_retention flag means "don't store the hash for future dedup on THIS upload". It does not mean "erase the hash from a previously indexed version of this document". The current code conflates these two meanings.

Failure Scenario

1. Tenant uploads file.pdf normally → content_hash=sha256(file) stored, sections indexed, Qdrant populated.
2. Same tenant re-uploads file.pdf with no_retention=True → pipeline executes the else branch, calls upsert_document(..., content_hash_value=None), NULLing the row's content_hash.
3. On a third upload (normal), find_doc_id_by_hash(sha256(file)) returns None (no hash to match).
4. Pipeline treats it as a brand-new document, assigns a new doc_id, re-writes all sections, re-upserts chunk points Qdrant already holds → dedup invariant permanently broken for that document row.

Proposed Fix

# Only write content_hash_value when no_retention is False:
content_hash_to_store = content_hash_value if not doc.no_retention else KEEP_EXISTING

# Or: always preserve the existing hash, just skip the lookup for no_retention:
await upsert_document(
    ...,
    content_hash_value=content_hash_value if not doc.no_retention else None,
    # But pass update_hash=False so the SQL uses COALESCE(new, existing):
    ...
)

The upsert_document SQL should use COALESCE(:new_hash, content_hash) so a NULL input preserves the existing value rather than overwriting it.