Skip to content

fix(fetcher): respect docs page URLs#140

Merged
chaliy merged 1 commit into
mainfrom
codex/fix-docs-site-root-llms
Jul 4, 2026
Merged

fix(fetcher): respect docs page URLs#140
chaliy merged 1 commit into
mainfrom
codex/fix-docs-site-root-llms

Conversation

@chaliy

@chaliy chaliy commented Jul 4, 2026

Copy link
Copy Markdown
Contributor

What

Fix DocsSiteFetcher so origin-level llms.txt / llms-full.txt is only used for direct index requests or root docs-site URLs.

Specific docs page URLs now fetch the requested page instead of returning the whole-site LLM index.

Why

A page request like https://docs.tenzir.com/reference/operators/where/ was returning the site-wide llms.txt index, dropping the page content the caller requested.

How

  • Added a root-docs URL check before probing origin-level llms.txt.
  • Kept direct /llms.txt and /llms-full.txt behavior unchanged.
  • Added regression tests for root docs index behavior and specific docs page behavior.
  • Updated specs/fetchers.md to match the new contract.

Risk

  • Low
  • Changes only the docs-site fetcher path.
  • Main compatibility impact: callers that relied on specific docs pages returning site-wide llms.txt now receive the requested page content.

Checklist

  • Unit tests are passed
  • Smoke tests are passed
  • Documentation/spec is updated
  • Specs are up to date and not in conflict
  • Security review completed

@chaliy chaliy marked this pull request as ready for review July 4, 2026 21:31
@chaliy chaliy merged commit 119ce24 into main Jul 4, 2026
11 checks passed
@chaliy chaliy deleted the codex/fix-docs-site-root-llms branch July 4, 2026 21:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant