ci: Dependabot, eval-report artifacts, and assertion-count doc fix#14
Merged
Merged
Conversation
- .github/dependabot.yml: weekly update PRs for pip (requirements-dev.txt) and the GitHub Actions used by the workflows. - quality-gate.yml: both jobs now upload evals/reports/ (including the per-run VCRM gap report) as artifacts with if: always(), so the audit trail survives the runner instead of being discarded — previously the reports were generated in CI and lost on job teardown. 30-day retention. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VBrxqChRJtxdvSpFhiUWUy
The suite has grown to 142 assertions but README (badge, features list, coverage section, checklist), ARCHITECTURE.md, and a workflow comment still said 85 — a claim dating to the initial release that went unnoticed while the suites never executed. Verified against a live deploy: 142/142. Also tightens the Tier S eval expectation (min_total_assertions 85 -> 142) so a future drop in assertion count fails the eval instead of passing silently. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VBrxqChRJtxdvSpFhiUWUy
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two follow-ups to the quality gate (PR #2):
ci: add Dependabot config and preserve eval reports as CI artifacts
.github/dependabot.yml: weekly update PRs for pip (requirements-dev.txt) and GitHub Actions versions.evals/reports/(including the per-run VCRM gap report) as artifacts withif: always(), 30-day retention — previously the reports were generated in CI and discarded on teardown.docs: correct SQL assertion count from 85 to 142
min_total_assertions85 → 142) so a future drop in assertion coverage fails the eval instead of passing silently.Test plan