Skip to content

Docs accuracy + register sweep: 45 pages verified against source#30

Open
DrAlexHarrison wants to merge 4 commits into
mainfrom
docsweep
Open

Docs accuracy + register sweep: 45 pages verified against source#30
DrAlexHarrison wants to merge 4 commits into
mainfrom
docsweep

Conversation

@DrAlexHarrison

Copy link
Copy Markdown
Member

Six Opus agents swept every page on docs.saturday.fit for two things at once: factual accuracy against the shipped code, and compliance with the AI-writing ruleset captured 2026-07-23.

The coaching tab had one light register pass in #26. The API Guides tab, 25 pages and ~4,700 lines, had never had an accuracy pass at all.

What a partner hit before this

Every code sample sent a literal sk_test_abc123def456 to api.saturday.fit, which returns 401. Sandbox keys only work against a base URL issued with the key, so the copy-paste path failed for both audiences at once. The quickstart's worked totals were also 33% high, so a partner checking the math against their own output would have concluded the engine was wrong. That is the first ninety seconds of every partner evaluation. All 33 occurrences across eight pages now use an environment variable, and every Python block is syntax-checked.

Invented API surface, removed

The AI coach SSE format did not exist: real events are message_start, text_delta, tool_call, tool_result, safety_warning, error, message_end, all JSON, with no [DONE] sentinel. The tool-budget table named three tools that do not exist. batch-operations documented a metadata wrapper, a scenario label, and a top-level athlete_id, none of which exist, and promised external_id deduplication that is not implemented, so a partner retrying a failed import duplicates their roster.

Corrections with direct customer impact

  • The Coach API gate is Pro Coach and above, not Business. The docs were telling paying subscribers to upgrade for access they already had. Both SDKs already had it right; the docs were the outlier.
  • Coach webhook registration succeeds on any tier but only delivers on Business and Enterprise. Now stated, including that nothing on the endpoint object says so.
  • Rate limits apply per account, not per key. X-RateLimit-Reset is delta seconds, not a Unix timestamp.
  • X-Saturday-Data-Training does not exist and is removed from two pages. X-Saturday-Data-License is now quoted exactly.
  • deployment.mdx said the docs site is hosted at api.saturday.fit, making every llms.txt, openapi.yaml, and mcp URL on it wrong.
  • Both attribution.mdx examples used the dark-background asset on a light page, so the copy-paste example rendered invisible.

Coaching docs

The pages described a Coach Portal that has since moved. ?view=manage deep-links are inert, /admin/billing redirects to Stripe Connect payouts, roles live on Access & Roles. Assistants start at Pro Coach, not Head Coach. A "senior assistant" preset role was documented and does not exist. Org-paid coverage has no tier gate at all. Assignment does not grant visibility: a newly activated assistant sees the head coach's whole roster read-only immediately.

Nutrition: eco-mode is a five-position spectrum, not a toggle; there is no concentration dial; and the gluc:fruc ratio sets the blend rather than raising the carb ceiling.

AI coach guide

The endpoint returns a 404 byte-identical to an unknown route so its existence cannot be detected, while a public guide announced it. Moved to drafts/, which .mintignore excludes wholesale, and removed from nav and from the introduction feature table. Content preserved for whenever the stage moves.

Verification

Nav matches disk in both directions, every internal link resolves, 24 Python and 32 JSON blocks parse, zero em dashes, zero dead brand, no secrets or internal paths.

Not included, needs your ruling

data-policy.mdx promises a 30-day retention grace and GDPR export window that nothing implements, and DELETE is an immediate hard delete so the export window cannot exist. Left unedited: you ruled to build the purge instead. safety.mdx drops a worked example whose numbers are not producible; the teaching intent is sound and it needs real values.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Y9Yok49jHDcx2DvirP6EXT

…trition dials against source

Verified every claim against fuel-backend, coach-portal, and the app rather
than the pages' own assertions.

Routing: /admin/billing now redirects to Stripe Connect payouts, so coverage
links repoint to the roster row menu; roles move to Access & Roles; the four
inert ?view=manage deep-links are removed; /admin/groups is documented.

Tiers and permissions: assistants start at Pro Coach (1) not Head Coach;
extra seats are a $10/seat/mo add-on that stacks rather than forcing an
upgrade; custom-role builder and org hierarchy are Head Coach and above;
audit log is Business and Enterprise; the 'senior assistant' preset role did
not exist and is replaced with the eight real roles. Org-paid coverage has no
tier gate at all: it needs an org you administer, the billing-coverage
permission, and a paid plan on the org beyond its included seats.

Assignment does not grant visibility. A newly activated assistant sees the
head coach's whole roster read-only immediately; assignment writes ownership.

Billing: free Coach tier funds zero paid seats; per-bracket prices stated to
the cent; invites can commit coverage; the receiving coach redeems a transfer,
not the athlete; trial-end handoff already fires before the coach is asked.

Nutrition: eco-mode is a five-position spectrum, not a toggle; there is no
concentration dial; the claim that the dials enforce the app's ranges was
false. The gluc:fruc ratio sets the blend and does not raise the carb ceiling,
which comes from the athlete profile.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y9Yok49jHDcx2DvirP6EXT

Ci-From: linux
Ci-Session: e18f7ca8-5c6f-4788-81ff-6fe566401959
…nable samples

The API Guides tab had never had an accuracy pass. Every endpoint, field,
enum, limit, and error code is now checked against the Go handlers in
fuel-backend, with live production probes where a value could drift.

Samples now run. Every code sample sent a literal sk_test_abc123def456 to
api.saturday.fit, which returns 401; sandbox keys only work against a base URL
issued with the key. All 33 occurrences across eight pages now use a
SATURDAY_API_KEY environment variable, and every Python block is
syntax-checked. The quickstart's worked totals were 33% high.

Invented surface removed. The AI coach SSE format did not exist: real events
are message_start, text_delta, tool_call, tool_result, safety_warning, error,
message_end, all JSON, with no [DONE] sentinel. The tool-budget table named
three tools that do not exist. batch-operations documented a metadata wrapper,
a scenario label field, and a top-level athlete_id, none of which exist, and
promised external_id deduplication that is not implemented, so a partner
retrying a failed import duplicates their roster.

Corrections with customer impact: the Coach API gate is Pro Coach and above,
not Business, so the docs were telling paying subscribers to upgrade for
access they had. Coach webhook registration succeeds on any tier but only
delivers on Business and Enterprise, which is now stated. Rate limits apply
per account, not per key. X-RateLimit-Reset is delta seconds, not a timestamp.
404 emits resource_not_found; 409 emits invalid_request; there is no 422
safety_limit. X-Saturday-Data-Training does not exist and is removed from two
pages; X-Saturday-Data-License is now quoted exactly.

oauth2 documented pre-fix refresh behavior and omitted both the offline_access
scope and the subscriber wall that gates every integration.

deployment.mdx said the docs site is hosted at api.saturday.fit, making every
llms.txt, openapi.yaml, and mcp URL on it wrong. The OpenAPI spec it described
is not served. attribution.mdx's two examples used the dark-background asset
on a light page, so the copy-paste example rendered invisible.

Feature stages are presented as shipped defaults with a pointer to GET /v1/,
because production config overrides code defaults on two of five gates in
opposite directions. Beta notice now states 14 days, matching the policy
decision rather than the old header text.

safety.mdx drops a worked example whose numbers are not producible; the
teaching intent is sound and it needs real values.

Also in this commit: introduction.mdx loses its AI Coach row (see the
following commit).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y9Yok49jHDcx2DvirP6EXT

Ci-From: linux
Ci-Session: e18f7ca8-5c6f-4788-81ff-6fe566401959
…STEALTH stage

The endpoint returns a 404 byte-identical to an unknown route, before
authentication, so its existence cannot be detected. A public guide in the
nav announcing the feature and how to request access cancelled that out.

Moved to drafts/, which .mintignore excludes wholesale, so the page leaves the
published site rather than sitting unlinked at a guessable URL. Removed from
docs.json nav and from the introduction.mdx feature table, which was the last
surface naming it.

Content is preserved for whenever the stage moves.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Y9Yok49jHDcx2DvirP6EXT

Ci-From: linux
Ci-Session: e18f7ca8-5c6f-4788-81ff-6fe566401959
llms.txt still sent the literal `sk_test_abc123def456`, the exact key this
sweep replaced in 33 other places because it 401s. This file is the one LLMs
ingest to generate partner code, so it propagates the broken sample furthest.
Now uses $SATURDAY_API_KEY, matching the convention used everywhere else.

rate-limiting.mdx corrected X-RateLimit-Reset in the example block but kept
Limit 60 / Remaining 42 from the pre-sweep version, which contradicts the
self-serve numbers this same commit added eight lines above. Source is
pkg/partner/individual.go (RequestsPerSecond 2, BurstSize 5) and
pkg/gateway/ratelimit.go (rpm = int(rps*60), remaining = limiter.Tokens()),
so a self-serve account emits 120, and Remaining can never exceed 5.

Ci-From: linux
Ci-Session: 33b3e323-f21c-417e-af46-f8bcf4ace105
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant