Modular PDF & full-text fetching framework#48

Merged

cmungall merged 25 commits into

mainfrom

feature/modular-pdf-fulltext-fetching

Jun 17, 2026

Commits on Jun 12, 2026

docs: design spec for modular PDF & full-text fetching framework

cmungall
and
claude
committed

Commits on Jun 13, 2026

docs: implementation plan for modular PDF & full-text fetching

cmungall
and
claude
committed
feat: add ReferenceIdentifiers, FullTextLocation, and ReferenceContent provenance fields

cmungall
and
claude
committed
feat: add full-text fetching config fields and files cache dir

cmungall
and
claude
committed
feat: add Extractor base class and ExtractorRegistry

cmungall
and
claude
committed
feat: add HTML and JATS/XML extractors

cmungall
and
claude
committed
feat: add pluggable PDF extractor with pypdf default backend

cmungall
and
claude
committed
feat: add ContentAcquirer with size cap and format resolution
cmungall
committed
feat: add FullTextProvider base class and registry
cmungall
committed
feat: add UnpaywallProvider
cmungall
committed
feat: add OpenAlexProvider
cmungall
committed
feat: add PMCFullTextProvider; route PMID full text through the chain

cmungall
and
claude
committed
feat: add identifier crosswalk for full-text providers
cmungall
committed
feat: wire full-text provider chain into ReferenceFetcher with PDF download+extract
cmungall
committed
feat: persist full-text provenance fields in the disk cache
cmungall
committed
test: isolate fetcher unit tests from full-text enrichment default

cmungall
and
claude
committed
feat: extract text from PDFs served at a url: reference
cmungall
committed
feat: add declarative custom full-text providers and YAML loader

cmungall
and
claude
committed
feat: register custom full-text providers at init and add --no-full-text CLI flag

cmungall
and
claude
committed
test: end-to-end full-text chain integration (metadata -> chain -> PDF)

cmungall
and
claude
committed
docs: how-to for full-text and PDF fetching

cmungall
and
claude
committed
style: remove unused imports in full-text test modules

cmungall
and
claude
committed

Commits on Jun 14, 2026

Merge branch 'main' into feature/modular-pdf-fulltext-fetching
cmungall
authored

Commits on Jun 17, 2026