feat: finish v1 leftovers — OTLP tracing, rate-limit rule, verified citations

.github/workflows/ci.yml

@@ -97,6 +97,9 @@ jobs:
       - uses: astral-sh/setup-uv@v5
         with:
           enable-cache: true
+          # The post-job cache prune has failed the whole job with exit code 2 on runs
+          # where every real step passed. A cache that is not pruned is merely larger.
+          prune-cache: false
       - run: uv sync --python 3.12 --frozen
       - name: ruff
         run: uv run ruff check .
@@ -133,6 +136,9 @@ jobs:
         if: steps.probe.outputs.have == 'true'
         with:
           enable-cache: true
+          # The post-job cache prune has failed the whole job with exit code 2 on runs
+          # where every real step passed. A cache that is not pruned is merely larger.
+          prune-cache: false
       - if: steps.probe.outputs.have == 'true'
         run: uv sync --python 3.12 --frozen
       - name: compose up

.github/workflows/conformance.yml

@@ -43,6 +43,9 @@ jobs:
       - uses: astral-sh/setup-uv@v5
         with:
           enable-cache: true
+          # The post-job cache prune has failed the whole job with exit code 2 on runs
+          # where every real step passed. A cache that is not pruned is merely larger.
+          prune-cache: false
       - run: uv sync --python 3.12 --frozen
 
       - name: bring up the broker
@@ -209,6 +212,9 @@ jobs:
       - uses: astral-sh/setup-uv@v5
         with:
           enable-cache: true
+          # The post-job cache prune has failed the whole job with exit code 2 on runs
+          # where every real step passed. A cache that is not pruned is merely larger.
+          prune-cache: false
       - run: uv sync --python 3.12 --frozen
 
       # MEASUREMENTS.md is committed, and stale numbers are worse than none —

.github/workflows/registry-scan.yml

@@ -51,6 +51,11 @@ jobs:
           path: |
             scan/summary.json
             scan/summary.md
+      - name: show the aggregate as an annotation
+        run: |
+          # Aggregate only. Readable through the checks API without downloading an artifact.
+          msg="$(sed ':a;N;$!ba;s/%/%25/g;s/\r//g;s/\n/%0A/g' scan/summary.md)"
+          echo "::notice title=registry-scan aggregate::${msg}"
       - if: ${{ inputs.per_server }}
         uses: actions/upload-artifact@v4
         with:

.pre-commit-hooks.yaml

@@ -14,7 +14,7 @@
 # version pinned by `rev`.
 - id: sentinel-scan
   name: sentinel MCP conformance scan
-  description: Grade a locally-running MCP server against MCP 2026-07-28 and fail on a MUST violation.
+  description: Grade a locally-running MCP server against the current MCP spec and fail on a MUST violation.
   entry: ghcr.io/patsypppe/sentinel:latest scan --gate must --no-color
   language: docker_image
   pass_filenames: false

CHANGELOG.md

@@ -4,6 +4,89 @@
 
 The harness stops being a one-shot grader and becomes something a team runs.
 
+### Measured: the whole MCP registry
+
+`registry-scan` run on GitHub Actions: 15,451 endpoints, 14.6% on the new revision. A 4,000-endpoint
+sample with corrected rules: 16% of new-revision servers pass every verifiable required rule (the first
+run's 13 of 2,263 was inflated by two false-positive classes, since fixed). Aggregate in
+`benchmarks/REGISTRY_SCAN.md`. The workflow also prints the
+aggregate as an annotation so it can be read without downloading an artifact.
+
+### Measured: precision and a production comparison
+
+`benchmarks/PRECISION.md`: hand-checked random samples (parameters: 3 of 12 real gaps before the rule was
+refined, 7 of 12 after, on a fresh draw; specification findings: 15 of 15 correct). The rule now treats
+enums, paging names, flag booleans and unit-suffixed numbers as self-explanatory and accepts quoted
+parameter names in prose. Headline: 11% of parameters flagged, an estimated 6 to 7% genuinely missing.
+`benchmarks/compare_cisco.py`: Sentinel against Cisco's `mcp-scanner` on a labelled poisoning corpus.
+
+### Fixed: the parameter-documentation rule over-reported
+
+`SENTINEL/STYLE/parameters-are-described` counted only schema descriptions. It now also accepts a
+parameter documented in the tool description (`name: ...`, as SDK docstring `Args:` blocks produce), an
+object whose fields are all described, and self-evident plain-string names (`path`, `url`, ...).
+The benchmark moved from 28% to 17% of parameters as a result.
+
+### Changed: lint shares one server process
+
+The lint rules only read, so they share one child process instead of starting the server per rule:
+4.4x to 6.7x faster with identical results. `benchmarks/chart.py` draws the 27-server result as an SVG.
+
+### Added: `sentinel lint` and stdio servers
+
+- `sentinel lint --command "npx -y <server>"` runs the manifest rules against a server started as a
+  child process (newline-delimited JSON over pipes), or `--endpoint` for HTTP. Most published MCP
+  servers are stdio programs, which Sentinel could not reach before.
+- `benchmarks/popular_servers.py`: lints 29 published servers; 28 started, 362 tools, 1,349
+  parameters, 17% documented nowhere.
+
+### Added: legacy-session mode
+
+Against a server that has not migrated to the current protocol, the rules that only read the tool
+manifest (names, descriptions, injection, schema validity, Origin) now open a previous-revision
+session and run, instead of every rule reporting the server unreachable.
+
+### Added: benchmarks against servers this project did not write
+
+- `benchmarks/official_negatives.py`: runs the broken fixtures from the official conformance repo and
+  reports which labelled violations Sentinel catches (7 of 7 gradable; one fixture is unusable and one violation has no rule, both listed).
+- `benchmarks/sdk_readiness.py`: scans servers built with the official SDKs.
+- Reports now say which protocol generation answered, and a scan of a legacy-only server says so at
+  the top instead of presenting a wall of failures as independent bugs.
+- The stub-description threshold is 12 characters, not 20 ("Add two numbers." is a description).
+
+### Added: `sentinel demo` and tool-description lint
+
+- `sentinel demo`: starts two bundled practice servers (one broken, one correct), scans both and
+  prints the verdicts. Works from a plain `uvx` install because the practice servers ship in the wheel.
+- `SENTINEL/STYLE/description-states-purpose` and `SENTINEL/STYLE/parameters-are-described`,
+  grounded in two 2026 studies of MCP tool descriptions (arXiv 2602.14878, 2602.18914).
+
+### Fixed: rules that were stricter than the specification
+
+Every `MCP/` rule was audited against the 2026-07-28 specification source and the official
+`modelcontextprotocol/conformance` suite. Rule ids are permanent, so a withdrawn rule is
+deprecated and points at its successor.
+
+- `MUST/discover-without-negotiated-version` withdrawn: the specification never exempts
+  `server/discover` from version negotiation, so it failed conformant servers.
+- Demoted to SHOULD (new ids): `discover-reports-server-info`, `list-changed-advertised-truthfully`
+  (now INDETERMINATE when the probe cannot tell), `malformed-json-is-parse-error`.
+- Moved out of the spec namespace to `SENTINEL/STYLE/`: `tools-have-descriptions`, and the
+  `initialize` / `ping` / `logging/setLevel` / `resources/(un)subscribe` removed-method checks. A
+  dual-era server MAY keep serving them. The `sampling` and `roots` checks were dropped: they remain
+  functional during their deprecation window.
+- Loosened to INDETERMINATE instead of FAIL where the wire cannot tell a violation from a
+  permitted behaviour: a headerless request that is served, a tampered `requestState` answered
+  with a fresh `input_required`, a populated result for an unknown resource URI, a tool list that
+  differs once between two connections, a tool list whose payload (not order) varies.
+- `missing-protocol-version-rejected` accepts `-32020` as well as `-32602`;
+  `missing-capability-error-shape` reads `requiredCapabilities` as an object, as the schema types it;
+  `x-mcp-header` rules no longer FAIL an untyped or union-typed property; the token-syntax rule uses a
+  full match; the rate-limit rule no longer treats the word "quota" as a throttle.
+- Three security rules cited a specification page that does not exist; every `MCP/` rule now cites a
+  heading that does.
+
 ### Added: operating it
 
 - **`sentinel.toml`**: endpoint, gate, timeouts, rule toggles, severity overrides, waivers.
@@ -36,7 +119,7 @@ capability agreement, `tampered-requeststate-rejected`, `input-required-carries-
 `tool-description-injection`, `wildcard-scopes`, `manifest-drift`, `tls-posture` and
 `SENTINEL/OPS/latency-budget`. A third fixture, `partial`, exercises every `NOT_APPLICABLE` branch.
 
-Eleven of the new `MCP/` rules cite a page, not a heading anchor; see `docs/CITATIONS.md`.
+Every `MCP/` rule cites a heading anchor, checked against the specification source. Three security rules had cited a page that does not exist (`basic/security_best_practices`; the page lives at `docs/2026-07-28/tutorials/security/`) and now point at the real one.
 
 ### Added: distribution
 
@@ -72,8 +155,6 @@ SBOM on each release.
 
 - `handles`, `mrtr` and the Postgres audit sink still carry their pgx code in the same packages;
   splitting them behind storage interfaces into `mcp/postgres` is the next step.
-- Citations for eleven rules are page-level (see above).
-- The OTLP exporter is not wired into the broker.
 - No release has been cut; PyPI trusted publishing and the GHCR image still need a first release.
 
 ## 0.3.0