Repository navigation
feat: pr-gate, the estate merge gate on GitHub #1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
b526896
feat: pr-gate, the estate merge gate as a reusable GitHub workflow
frankxai 1404c37
fix: close the false passes Codex found in review of b526896
frankxai 504fba7
fix: close the round-2 findings from Codex review of 1404c37
frankxai 6a84842
fix: round-3 findings from Codex review of 504fba7
frankxai File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| name: pr-gate tests | ||
|
|
||
| on: | ||
| pull_request: | ||
| paths: ['pr-gate/**', '.github/workflows/pr-gate*.yml'] | ||
| push: | ||
| branches: [main] | ||
| paths: ['pr-gate/**', '.github/workflows/pr-gate*.yml'] | ||
|
|
||
| concurrency: | ||
| group: pr-gate-test-${{ github.ref }} | ||
| cancel-in-progress: true | ||
|
|
||
| jobs: | ||
| test: | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 5 | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| - uses: actions/setup-node@v4 | ||
| with: | ||
| node-version: '22' | ||
| - run: node --test pr-gate/decide.test.mjs |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,62 @@ | ||
| name: pr-gate | ||
|
|
||
| # Reusable merge gate. Callers install pr-gate/caller.yml and pin this file to a commit SHA. | ||
| # It reads the PR through the API and never checks out or runs the PR's code, which is what makes | ||
| # running it from pull_request_target safe. | ||
| on: | ||
| workflow_call: | ||
| inputs: | ||
| pr-number: | ||
| description: PR to judge, as a string. Anything that is not a plain number is a no-op. | ||
| required: true | ||
| type: string | ||
| merge: | ||
| description: Squash-merge when every condition passes. False = report only. | ||
| required: false | ||
| type: boolean | ||
| default: false | ||
| gate-ref: | ||
| description: Commit of Starlight-Intelligence/.github to load the gate from. Pin it to the same SHA as the uses line. | ||
| required: false | ||
| type: string | ||
| default: main | ||
| required-checks: | ||
| description: Comma-separated check run names that must pass on the head commit, e.g. "test, build". | ||
| required: true | ||
| type: string | ||
| wait-minutes: | ||
| description: How long to wait for running CI when it is the only thing outstanding. | ||
| required: false | ||
| type: number | ||
| default: 8 | ||
|
|
||
| jobs: | ||
| decide: | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 15 | ||
| permissions: | ||
| contents: write | ||
| pull-requests: write | ||
| issues: write | ||
| checks: read | ||
| steps: | ||
| - name: Load the gate | ||
| uses: actions/checkout@v4 | ||
| with: | ||
| repository: Starlight-Intelligence/.github | ||
| ref: ${{ inputs.gate-ref }} | ||
| path: .pr-gate | ||
| persist-credentials: false | ||
|
|
||
| - uses: actions/setup-node@v4 | ||
| with: | ||
| node-version: '22' | ||
|
|
||
| - name: Decide | ||
| env: | ||
| GITHUB_TOKEN: ${{ github.token }} | ||
| PR_NUMBER: ${{ inputs.pr-number }} | ||
| PR_GATE_MERGE: ${{ inputs.merge }} | ||
| PR_GATE_WAIT_MINUTES: ${{ inputs.wait-minutes }} | ||
| PR_GATE_REQUIRED_CHECKS: ${{ inputs.required-checks }} | ||
| run: node .pr-gate/pr-gate/run.mjs |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| ## Code Review Rules | ||
|
|
||
| Codex reviews here as the second provider: most PRs in this repo are written by Claude. Flag as P0 or P1: | ||
|
|
||
| - A claim in code, copy or docs with no evidence behind it: invented metrics, counts, testimonials, "verified" or "trusted by" without a source. | ||
| - Anything that fakes or bypasses CI: posting commit statuses by hand, skipping tests, `--admin`, `--no-verify`, tests rewritten to match broken behavior, graders a do-nothing run can pass. | ||
| - Secrets, tokens or keys in code, logs or tests; `.env` values committed. | ||
| - Money, auth, env or dependency changes, and new or deleted public pages, without a linked ruling from Frank. | ||
| - A checkout, buy button or install command for something that is not purchasable or published yet. Unlaunched products use the waitlist. | ||
| - Unbounded loops, retries or agent fan-out with no cap. | ||
| - On web surfaces: `100vh` in shells, hover styles outside `(hover: hover) and (pointer: fine)`, inputs under 16px, zoom disabled, motion with no `prefers-reduced-motion` path. | ||
|
|
||
| Do not flag style preferences already enforced by the linter. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,39 @@ | ||
| # pr-gate | ||
|
|
||
| The estate merge gate, running on GitHub instead of a laptop. On the PR's current head commit it passes only when: | ||
|
|
||
| - A review from a GitHub bot identity of a different harness than the author exists for that commit: `chatgpt-codex-connector[bot]` (Codex) or `claude[bot]` (Claude). The author is read from the branch prefix, because every harness pushes as the same GitHub account. | ||
| - That review is an approval, or a review whose body says it found no major issues. Dismissed and pending reviews are ignored. A bot review mentioning P0 or P1 blocks, and so does a bot comment that names a P0 or P1 against the head. Bot comments never sign off, because they carry no commit id. | ||
| - A renamed file is judged on both its old and new path. | ||
| - Every check named in the repo's `required-checks` has a latest attempt that succeeded, no other check failed, and nothing is still running. Commit statuses never count: anything holding a token can post one, which is how 20 PRs merged on 2026-10-19 with no check having run. | ||
| - All review threads are resolved (every page read; an unreadable answer blocks) and the full list of changed files was read. | ||
| - It is not a draft, carries no `hold` / `do-not-merge` / `blocked` / `wip` label, and its body links the intent it serves (an issue, board item or ruling). | ||
| - It touches none of Frank's paths: workflows, env files, dependencies, money, auth and middleware, `vercel.json`, or a new or deleted public page. No label overrides this. Labels prove nothing about who applied them on a shared account, so Frank merges those himself. | ||
|
|
||
| In merge mode it decides again from fresh facts just before merging and merges with a head-SHA guard. | ||
|
|
||
| ## What does not count | ||
|
|
||
| Typed lines (`starlight-signoff: ...` from the estate `tools/pr-gate.mjs`, `Checker: claude-cloud · head <sha> · verdict PASS`) are shown as advisory but never pass the gate: the author can type them. The estate `pr-gate.mjs` still honours them for a merge a human runs. | ||
|
|
||
| So today, Claude-authored PRs pass once Codex reviews them. Codex- and Gemini-authored PRs pass only after a review from the Claude GitHub bot, or a human-run merge. | ||
|
|
||
| Branch prefixes: `agent/claude/`, `claude/`, `ccr-` and `factory/` (Foreman, whose implementer is an Anthropic model) are Claude; `codex/`, `agent/codex/`, `agent/gpt*` are Codex; `agent/antigravity/`, `agent/gemini/`, Jules and `sentinel-` branches are Gemini. | ||
|
|
||
| ## Install in a repo | ||
|
|
||
| 1. Copy `caller.yml` to `.github/workflows/pr-gate.yml` and `poke.yml` to `.github/workflows/pr-gate-poke.yml`. Replace both `GATE_SHA` with a commit of this repo, list the repo's CI workflow names under `workflow_run`, and its check names in `required-checks`. | ||
| 2. Turn on Codex code review and automatic reviews for the repo in Codex settings. | ||
| 3. Add the section in `CODE-REVIEW-RULES.md` to the repo's `AGENTS.md`. | ||
| 4. Start with `merge: false` (report only). Flip to `merge: true` where autonomous landing is ruled. | ||
|
|
||
| Every trigger runs the caller from the default branch. Review events go through `pr-gate-poke`, which holds no permissions, and come back as `workflow_run`. | ||
|
|
||
| ## Known limits | ||
|
|
||
| - Without branch protection nothing stops someone merging around the gate; the private repos on the free plan cannot have it. The gate is the merge path agents are told to use, not a lock. | ||
| - Between the final decision and the merge call a few milliseconds remain in which a review could be dismissed. Only a required status check under branch protection closes that. | ||
| - A same-repo PR's own workflows run with the repo token. The gate refuses to merge any PR that edits workflows, but cannot stop such a PR from posting a same-named check on itself. | ||
| - A squash merge made with the workflow token does not trigger push workflows on the base branch. Vercel's git integration still deploys. | ||
|
|
||
| Tests: `node --test pr-gate/decide.test.mjs`. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,47 @@ | ||
| # Copy to .github/workflows/pr-gate.yml in a product repo, with poke.yml beside it. | ||
| # Replace GATE_SHA (both places) with a commit of Starlight-Intelligence/.github, list the repo's CI | ||
| # workflow names under workflow_run and its check names in required-checks, and set merge to true | ||
| # only where autonomous landing is ruled. | ||
| # | ||
| # Every trigger here runs this file from the default branch, never from the PR, so a PR cannot | ||
| # rewrite the gate it is judged by. Reviews arrive through pr-gate-poke (no permissions) and come | ||
| # back here as workflow_run. | ||
| name: pr-gate | ||
|
|
||
| on: | ||
| pull_request_target: | ||
| types: [opened, synchronize, reopened, ready_for_review, labeled, unlabeled, edited] | ||
| issue_comment: | ||
| types: [created] | ||
| workflow_run: | ||
| workflows: [pr-gate-poke, CI] | ||
| types: [completed] | ||
|
|
||
| permissions: | ||
| contents: write | ||
| pull-requests: write | ||
| issues: write | ||
| checks: read | ||
|
|
||
| concurrency: | ||
| group: pr-gate-${{ github.event.pull_request.number || github.event.issue.number || github.event.workflow_run.pull_requests[0].number || github.event.workflow_run.display_title }} | ||
| cancel-in-progress: true | ||
|
|
||
| jobs: | ||
| pr-gate: | ||
| # Comments re-run the gate only for a result from an allowlisted reviewer bot, or an exact | ||
| # `/gate` from an owner or org member. A comment can only trigger a re-read, never a sign-off. | ||
| if: >- | ||
| (github.event_name == 'pull_request_target') || | ||
| (github.event_name == 'workflow_run') || | ||
| (github.event_name == 'issue_comment' && github.event.issue.pull_request && ( | ||
| contains(fromJSON('["chatgpt-codex-connector[bot]","claude[bot]"]'), github.event.comment.user.login) || | ||
| (github.event.comment.body == '/gate' && contains(fromJSON('["OWNER","MEMBER"]'), github.event.comment.author_association)))) | ||
| uses: Starlight-Intelligence/.github/.github/workflows/pr-gate.yml@GATE_SHA | ||
| with: | ||
| # Reusable inputs are typed; the number is passed as a string. A relayed review with no PR | ||
| # attached falls back to the poke run title, which is the PR number. | ||
| pr-number: ${{ format('{0}', github.event.pull_request.number || github.event.issue.number || github.event.workflow_run.pull_requests[0].number || github.event.workflow_run.display_title) }} | ||
| required-checks: test | ||
| merge: false | ||
| gate-ref: GATE_SHA | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In this caller template, every newly created issue comment starts a workflow with the same PR concurrency group, including ordinary human comments and comments from untrusted users. Because cancellation is configured at workflow scope but the allowlist is evaluated only in the job's
if, an ineligible comment can cancel an active gate run and then skip its own gate job. If the original run was waiting for CI or had not reached the merge call, the PR is left without an active evaluation until another qualifying event occurs. Move the concurrency policy onto the filtered job so comments that cannot run the gate cannot cancel it.Useful? React with 👍 / 👎.