Skip to content

agent-audience: a mentioned member may stay silent, so a PR can sit for rounds with nobody owning the verdict #911

Closed

Description

@Birfy

Reported by the owner, 2026-10-24: "this @Reviewer got no response — file an issue to fix it."

The reported symptom is not that a member ignored a request. Reviewer answered every time, correctly: it said "not mine, #907 is Reviewer2's" and stopped. What broke is that nothing in the repository notices a misaddressed mention, so a pull request can sit for rounds with the address pointing at someone who is right to decline.

What actually happened (#907, verified through the API)

  • The PR body has no Owner: and no Reviewer: line at all — its first line is Written-by: Dev3. Under the routing rule the group runs on, a Reviewer that cannot see those two lines must not judge at all.
  • Seven tips were pushed between 00:53Z and 01:27Z (19e1d83f → 0c7b33a7); four verdicts were posted on four of them, all by Reviewer2.
  • Dev3 addressed @Reviewer on three of those tips (19e1d83f, a65d109e, 973e0624). Reviewer declined each time, in the group, in one line. Correct behaviour, and still nobody picked up the next step, because the only place that knew who owned the verdict was a message in a chat.

Measured against the live record (2026-10-24, head b8b582bd) — this corrects what the check may claim. Run against #907 itself, the check produces one report, not two: "no Owner:/Reviewer: line". The three @Reviewer mentions were typed in the group chat and never became GitHub comments, so the API holds no record of them and the mention rule has no input for this incident. Both consequences stay in this issue and in the PR body:

  • The missing-lines rule is what would have flagged #907 on its first run — a pull request that names no reviewer is unassigned by definition, and that alone is enough. #907's body still has no such line at b8b582bd.
  • The mention rule only covers @s that live on GitHub (PR comments, review bodies). Chat-side routing is as invisible to CI as "did anyone read it" — the round driver owns both, and no check should be written as if it did.

The replay fixture (mention text fed straight into the evaluator) is the right unit test for the rule — the logic must be able to go red — but it is a replay, never evidence that the live incident would have been flagged twice. Say exactly that in the PR body.

So the defect has two halves, and only one of them can be a check:

  1. Checkable: the body names an owner and a reviewer, and the people actually mentioned on a tip agree with that line.
  2. Not checkable: whether a mentioned member read the message and acted. No repository artifact records a read. That half is a routing rule and belongs in the round driver, not in CI.

Scope — a report-only check, built the way this repo already builds them

There is an established shape for exactly this: .github/scripts/review-gate.mjs + .github/workflows/review-gate.yml (report only, continue-on-error: true, read-only token, writes $GITHUB_STEP_SUMMARY and one ::warning, never labels/comments/merges). Follow it; do not invent a second shape.

New: .github/scripts/agent-audience.mjs + .github/scripts/agent-audience.test.mjs + .github/workflows/agent-audience.yml (pull_request plus issue_comment, permissions: contents: read, pull-requests: read, issues: read, continue-on-error: true).

The membership list already exists and must be the source of truth — .github/agent/review-policy.json: writers is the set of Bot names a report may name, reviewers/maintainers are the human logins. Do not hard-code a roster in the script, and do not add a second file listing the same names.

The check reports, per pull request:

  • the body's first block has no Owner: line, or no Reviewer: line — one line each;
  • a named owner or reviewer is not in writers/reviewers/maintainers — one line, naming the unknown name;
  • a member was mentioned (@Name) in a comment created after the current head commit's committedDate and that name is neither the body's Reviewer: nor its Owner:, and not Chief (Chief is the documented escalation target, so addressing it is always legitimate) — one line per (member, count), naming the head SHA.

Output must be readable by the round driver without opening the workflow: the first line of each report is #<n> @<head-sha7>: <sentence>, and the whole set goes into $GITHUB_STEP_SUMMARY. Exit 0 on every path. This is not a gate: it must not be added to branch protection or to tests-ok's needs, and it must not post comments (the same reasoning as review-verdict.yml's header: a job that turns red is a job people learn to ignore, and a comment needs a write token).

Names are text, not logins. Every Bot shares one GitHub identity (see review-policy.json's own note, #627, and #633), so an @Reviewer in a body is plain text and cannot be resolved against a login. The check compares text names from writers; a mention of a login in reviewers/maintainers is compared literally as well. Say this in the script's header comment so the next person does not try to make it authoritative.

Acceptance — each item is a fixture test in agent-audience.test.mjs

  • The #907 record replays as a red case: a body with no Owner:/Reviewer: lines plus three comments containing @Reviewer produces two reports — "no reviewer named" and "@Reviewer mentioned N times while the body names none". Today no such script exists, so the test fails before the script does.
  • A body naming Reviewer: Reviewer2 with comments mentioning @Reviewer2 produces no report. This is the negative control that keeps the check from crying wolf.
  • A comment mentioning @Chief produces no report; a comment mentioning @Dev4 on a Reviewer: Reviewer2 PR produces one report naming Dev4.
  • A mention that happened before the head commit's committedDate produces no report — the check judges the current tip, not the PR's history.
  • An unknown name (Owner: Nobody) is reported once and does not crash the run; a body with the two lines and no comments yields a clean summary and exit 0.
  • The workflow file passes the repository's own workflow test (scripts/test-workflow.test.mjs) and does not appear in merge-gate-ruleset.json or in tests-ok's needs.
  • The run leaves the pull request untouched: no label, no comment, no review, no approval.

What this issue does not do

  • It does not detect silence. A mention of the correct reviewer that goes unanswered is invisible to CI; that is the round driver's job, and the rule "a member that is @'d but is not the owner answers one line or reports Chief" is already in the group notice. Do not add a new process section here: if that rule needs to live in the repo, it becomes one more line in #906's version of the contract, not a third copy.
  • It does not make the body's names authoritative for judging. A verdict still belongs to the person the body names; this check only says when the addressing disagrees with that line.

Related

#907 — the pull request this came from; #906 — the contract the routing rule belongs to; #633 / #644 — why a written name is a claim and not an identity; #909 / #904 — the other inbound-trust work in flight, unrelated except that they are the PRs this check would have flagged.

Activity

  1. added
    enhancementNew feature or request
    agent-readyMaintainer-approved spec: an agent may take it. Only maintainers set this; never in a template
    area:automationScheduled tasks and automation
    on Oct 24, 2026
  2. self-assigned this
    on Oct 24, 2026
  3. added
    agent-workingAn agent has claimed this issue (the claim comment names the branch)
    and removed
    agent-readyMaintainer-approved spec: an agent may take it. Only maintainers set this; never in a template
    on Oct 24, 2026
  4. Birfy commented on Oct 24, 2026

    @Birfy
    CollaboratorAuthor

    Written-by: Dev2

    Claimed on branch chore/911-agent-audience (repo branch, base cbad5e58). PR #912 with Fixes #911.

  5. removed
    agent-workingAn agent has claimed this issue (the claim comment names the branch)
    on Oct 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area:automationScheduled tasks and automationenhancementNew feature or request

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions