[gabrielspadon]

Summary

Two pairs of different papers by the same team were scored as one work, both on disk and in enrichment, so a refresh with slightly different metadata could delete one of each pair:

• Bodorik: Liu2021-BlockchainPrivacy vs Liu2022-AutomatingSmart
• Matwin: Ferreira2022-SemiSupervised vs Ferreira2022-SemiSupervisedGeometric

Root cause

compute_dedup_score gives 0.25 for author overlap and 0.10 for the same year, or 0.05 for one year apart (citeforge/text_utils.py:654-664). A co-authored pair from one year therefore starts at 0.30 to 0.35 of the 0.60 threshold with no title evidence at all.

• Liu: title 0.654 × 0.40 = 0.262, plus authors 0.25, year 0.05 and venue 0.08, gives 0.641.
• Ferreira: 0.262 + 0.25 + 0.10 gives 0.612.

The composite paths only required a title score of 0.55 to 0.6 before the composite could decide. In enrichment the floor was 0.35.

Fix

_titles_agree (citeforge/identity.py:106) accepts titles that score at least SIM_TITLE_SIM_MIN (0.8), or where one normalized title occurs whole inside the other, because Scholar truncates long titles from both ends. When neither record carries an arXiv id, the following now require it:

• the disk preprint-pair, same-key, composite and relaxed-preprint paths (:250-255);
• the enrichment composite gate (:208).

Equal arXiv ids, the one-sided arXiv rule from #61, and DOI, DOI-version, external-id, high-title and truncated-title matches are unchanged.

Effect on the committed corpus

Every same-directory pair evaluated under DISK_SURVIVOR before and after: three pairs flip from "same work" to "distinct".

• Bodorik Liu 2021/2022 and Matwin Ferreira 2022. These are distinct papers and are now kept.
• Orji, D2025-DevelopingInteractive.bib (medRxiv 10.1101/2025.10.07.25337102) vs Etienne2025-DevelopingInteractive.bib (JMIR Human Factors 10.2196/86283). It is the same author list, retitled, at a title score of 0.773. Crossref has no relation between the two DOIs, so the preprint is now kept instead of removed as superseded.

Reilly's Porcino2022 pair is unchanged: both files are kept.

Verification

• pytest -m 'not live' --cov=citeforge --cov=main --cov-fail-under=68 on Python 3.12, macOS: 2083 passed, 2 skipped (the documented cutover contracts), 7 deselected, coverage 84.14%. ruff check and mypy are clean.

• test_shared_authors_and_year_never_merge_different_titles uses the committed Liu and Ferreira records. On main both cases fail, with one file left on disk. Both pass with the fix, and enrichment also rejects both pairs.

• TestStrongAuthorDedupGate::test_same_authors_moderate_title_sim_matches still passes. Alhasani 2025 scores 0.737, and its Scholar title is a both-ends truncation of the published one.

• Enrichment-context verdicts on every same-directory corpus pair: main accepts 11 pairs and this branch accepts 2. Besides Liu, Ferreira and Orji, main also accepted six pairs of different Neethirajan papers at title scores of 0.48 to 0.53, with a preprint/published split opening the gate. The 2 that remain are prefix truncations (TRUNCATED_TITLE).

• Refresh replay on main plus this change: the real run_all over a copy of the committed corpus, with live Crossref, arXiv, OpenAlex and DOI enrichment of shared-DOI records (Scholar and Gemini not used), then finalize_run. 9 authors were forced, 0 DOI conflicts remained, and finalization completed with 3691 files and 1105 in a2i2. Against the committed corpus, 0 of 58 directories changed file count or publication set. Liu, Ferreira, Porcino, Orji and VISLA are all present.