Skip to content

feat: pr-gate, the estate merge gate on GitHub - #1

Merged
frankxai merged 4 commits into
mainfrom
agent/claude/pr-gate-action
Oct 27, 2026
Merged

frankxai merged 4 commits into
mainfrom
agent/claude/pr-gate-action

Conversation

@frankxai

@frankxai frankxai commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Intent: Frank, 2026-10-27, in session: "build all and lead and create autonomously" (follow-up to the overnight cloud teams plan: move the merge gate from the laptop to GitHub so reviewed work lands while the laptop sleeps).

What

  • pr-gate/decide.mjs: pure decision. Cross-harness review on the exact head (author from branch prefix), CI = check runs only (commit statuses excluded: the 2026-10-19 forged-status class), no unresolved threads, intent link, hold labels, and Frank's paths (workflows, env, deps, money, auth/middleware, vercel.json, new or deleted public pages) need frank-approved.
  • pr-gate/run.mjs: fetches facts with the workflow token, waits up to 8 min when CI is the only thing outstanding, keeps one sticky comment, squash-merges with a sha guard only when the caller sets merge: true.
  • .github/workflows/pr-gate.yml: reusable workflow; never checks out PR code, so pull_request_target callers are safe.
  • pr-gate/caller.yml, CODE-REVIEW-RULES.md (Codex ## Code Review Rules section), README with known limits.
  • Codex counts only from its bot identity; Foreman factory/ branches count as Claude-authored (implementer is Anthropic).

Verification

node --test pr-gate/decide.test.mjs: 14/14 pass locally (cases: self-review, stale sha, human-typed Codex text, Codex comment before head, checker lines, short sha, statuses-only CI, failing/running CI, threads, drafts, labels, missing intent, changes requested, protected paths, page add/delete).

Not done / limits

  • Not yet exercised against a live PR; rollout PRs in ladder repos follow in report-only mode first.
  • A PR editing workflows could post a same-named check; only an org ruleset requiring this workflow closes that (Frank's setting).
  • Needs Codex review (Claude-authored).

🤖 Generated with Claude Code


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

frankxai and others added 4 commits October 27, 2026 04:19
Moves maker != checker from the laptop-only tools/pr-gate.mjs onto GitHub so
reviewed work can land overnight. Counts only check runs as CI (no commit
statuses, the 2026-10-19 forgery class), binds every review to the head SHA,
accepts Codex only from its bot identity, blocks unresolved threads and
Frank's paths. Report-only by default; merge mode squash-merges with a sha guard.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- drop pull_request_review from the privileged caller; reviews relay through
  a no-permission pr-gate-poke workflow and return as workflow_run
- typed sign-off lines are advisory only; only exact GitHub bot identities count
- dismissed/pending bot reviews ignored; only approval or a clean result passes;
  P0/P1 text blocks
- bot comments must name the head commit
- repos declare required checks; skipped or missing required checks block;
  latest attempt only
- paginate review threads, fail on GraphQL errors, block on truncated file lists
- no label overrides Frank's paths
- re-decide from fresh facts right before merging
- /gate only from OWNER/MEMBER/COLLABORATOR

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- renames judged on old and new path; a page rename is a delete plus a publish
- bot comments never sign off; a P0/P1 bot comment on the head blocks;
  severity match is case-insensitive
- relayed reviews without an attached PR resolve it from the head commit
  (single open PR in this repo with that exact head, else no-op)
- REST pagination fails closed instead of returning a short list
- /gate only from the repo owner as a leading command; comment triggers only
  from allowlisted reviewer bots

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- pass pr-number as a typed string via format(); reusable input is string
- relay carries the PR number in the poke run title (run-name); the gate
  validates it as a plain number and re-reads that PR, so a forged title can
  only trigger a re-read; head-sha resolver removed
- one concurrency group per PR across all events
- /gate must be the exact comment, from OWNER or MEMBER

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@frankxai

Copy link
Copy Markdown
Contributor Author

Codex review (GPT, codex exec --profile review), round 4 of 4 on 6a84842

Rounds: b526896 FAIL (1 P0, 7 P1, 2 P2) → 1404c37 FAIL (3 P1, 2 P2) → 504fba7 FAIL (2 P1, 3 P2) → 6a84842 PASS. Every reproduced false pass is now a regression test (17 tests).

Relayed verbatim by the author's session (Claude ran the Codex CLI); the reviewer is a different provider, the relay is not. Weigh accordingly.

Round 4 output

TL;DR: VERDICT: PASS for 6a84842. All Round 3 findings are fixed; I found no new P0, P1, or P2 finding.

Round 3 finding Status Evidence
Reusable input number versus string FIXED caller.yml:44 uses format('{0}', …) for the string input in pr-gate.yml:9. GitHub documents matching input types.
Merge-commit SHA fallback FIXED poke.yml:7 sets the run name to the PR number. caller.yml:44 reads its display_title; run.mjs:18 rejects nonnumeric titles, then run.mjs:59 fetches the PR afresh. GitHub defines run-name as the displayed run name.
Resolver pagination FIXED The SHA resolver is gone; no commit-to-PR lookup remains.
Split concurrency groups FIXED caller.yml:27 uses the same PR number, or the numeric relay title, across event paths.
/gate prefix and OWNER-only check FIXED caller.yml:39 requires the whole command and accepts OWNER or MEMBER. GitHub comparisons are case insensitive, so /GATE also triggers a re-read; suffixes such as /gate-now do not.

A forged poke title or a CI run with an empty pull_requests array and a numeric title can select an existing PR for a fresh API evaluation. It cannot supply a review, check result, or head SHA; the merge path re-evaluates and uses a head-SHA guard at run.mjs:104. A nonnumeric commit title exits without a decision.

node --test pr-gate/decide.test.mjs: 17 passed, 0 failed. GitHub reports pr-gate tests successful for 6a84842. That workflow tests decide.mjs; the relay behavior above is verified from code and GitHub documentation, not from a live poke run.

Findings: P0 none · P1 none · P2 none.

@frankxai
frankxai marked this pull request as ready for review October 27, 2026 04:02
@frankxai

Copy link
Copy Markdown
Contributor Author

starlight-signoff: harness=codex verdict=pass sha=6a84842bf5b3e85d96d5c1b8d11046fe4680b276

Codex review round 4 PASS at 6a84842 (P0/P1/P2 none); full rounds in the PR comments.

Bound to 6a84842 — a new push invalidates this sign-off. /gate

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-27T04:06:05.902836Z 6a84842 Draft marked ready
🔒 Security Review ✅ Completed 2026-10-27T04:12:16.652601Z 6a84842 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6a84842bf5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread pr-gate/decide.mjs
}

const blocks = signoffs.filter((s) => (s.verdict === 'block' || s.verdict === 'fail') && onHead(s) && s.harness !== author);
const valid = signoffs.filter((s) => s.verdict === 'pass' && onHead(s) && s.harness !== author);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Block automatic merges when the author harness is unknown

For an unrecognized branch such as fix/bug or agent/custom/bug, harnessOfBranch() returns unlabeled or agent-other, so either allowlisted reviewer bot satisfies this inequality. A Codex-authored PR on such a branch therefore passes with only a Codex review, allowing self-review to authorize an automatic merge. Reject unknown author identities rather than treating them as a distinct provider; this violates the maker/checker separation required by AGENTS.md.

AGENTS.md reference: AGENTS.md:L22-L22

Useful? React with 👍 / 👎.

Comment thread pr-gate/caller.yml
Comment on lines +26 to +28
concurrency:
group: pr-gate-${{ github.event.pull_request.number || github.event.issue.number || github.event.workflow_run.pull_requests[0].number || github.event.workflow_run.display_title }}
cancel-in-progress: true

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Apply comment filtering before cancellation can affect the gate

In this caller template, every newly created issue comment starts a workflow with the same PR concurrency group, including ordinary human comments and comments from untrusted users. Because cancellation is configured at workflow scope but the allowlist is evaluated only in the job's if, an ineligible comment can cancel an active gate run and then skip its own gate job. If the original run was waiting for CI or had not reached the merge call, the PR is left without an active evaluation until another qualifying event occurs. Move the concurrency policy onto the filtered job so comments that cannot run the gate cannot cancel it.

Useful? React with 👍 / 👎.

@frankxai
frankxai merged commit e3faf85 into main Oct 27, 2026
1 check passed
@frankxai
frankxai deleted the agent/claude/pr-gate-action branch October 27, 2026 12:20
@frankxai
frankxai restored the agent/claude/pr-gate-action branch October 27, 2026 12:21
frankxai added a commit to Starlight-Intelligence/marketing-intelligence that referenced this pull request Oct 28, 2026
…github main

Addresses the Codex P1 on c78485c: the `test` job passed on a tree with no
tests. It now fails when required files are missing, when no test files
exist, when a test file defines no tests, or when no test passes. Adds
tests/repo.test.mjs with real assertions on README and the gate wiring
(pin consistency, report-only, required check, poke permissions).

Repins the reusable pr-gate from 6a84842 to e3faf85, the merge of
Starlight-Intelligence/.github#1 on main, in both the uses line and gate-ref.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015JZDKkwqzo2YxGP29vbs17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant