Integrate Oz changelog skill into create_release workflow#10831
Integrate Oz changelog skill into create_release workflow#10831vikvang wants to merge 11 commits into
Conversation
Replaces the TypeScript orchestrator in services/changelog-draft/ with a skill-first architecture: - .agents/skills/changelog-draft/SKILL.md: 7-step workflow for generating reviewable changelog drafts from release PRs - scripts/fetch_prs.py: collects PRs in a release range, extracts CHANGELOG markers from PR bodies - scripts/classify_contributors.py: checks warpdotdev org membership via gh api - scripts/extract_feature_flags.py: parses RELEASE/PREVIEW/DOGFOOD flag lists from warp_features/src/lib.rs - .agents/skills/classify-changelog-pr/SKILL.md: inline classification guidance for unmarked PRs - .github/workflows/changelog_draft.yml: workflow_dispatch workflow using oz-agent-action Co-Authored-By: Oz <oz-agent@warp.dev>
- fetch_prs.py now falls back to all first-parent commits when no merge commits are found (handles squash-merge workflow used by warp repo) - Added examples/changelog-draft-example.md generated from real data: v0.2026.04.29.08.56.stable_00 → v0.2026.05.06.09.12.stable_00 (211 PRs, 57 with explicit markers, 154 unmarked) Co-Authored-By: Oz <oz-agent@warp.dev>
- fetch_prs.py: include changed_files per PR so the agent can detect FeatureFlag references and apply channel visibility rules - classify_contributors.py: add 'unknown' bucket for auth failures instead of misclassifying as external - SKILL.md: add guidance for feature-flag detection via changed_files and handling unknown contributors - changelog_draft.yml: pin oz-agent-action to commit SHA ce1621abf6a8ed8afdd4e4cc994545ede8fe1c6f Co-Authored-By: Oz <oz-agent@warp.dev>
- fetch_prs.py: remove merge-commit fallback (warp uses squash merges only); add CHANGELOG-NONE opt-out marker support - SKILL.md: fix release range logic to diff against previous release *cut* (different date _00 tag), not just previous tag — handles RC tags like _01, _02 correctly - SKILL.md: use markdown PR links [#N](url) in output examples; remove IMAGE from category list; add CHANGELOG-NONE to skipped output - classify-changelog-pr: remove IMAGE category (configured manually); add CHANGELOG-NONE as first exclusion rule - Example: convert all PR refs to markdown links; add ✨ unicode emoji to external contributor attribution - Workflow: add comment clarifying workflow_dispatch access restriction Co-Authored-By: Oz <oz-agent@warp.dev>
Co-Authored-By: Oz <oz-agent@warp.dev>
…ngelog-draft-skill
…an output - Add CHANGELOG-NONE to PR template for deterministic changelog exclusion - Remove 'Needs Review' and 'Skipped PRs' from markdown output (kept in JSON) - Add 'Community' section with Contributors and Issue Reporters subsections - New fetch_issue_reporters.py script to credit users who reported fixed issues - Update fetch_prs.py to extract linked issues from PR bodies (Fixes/Closes) - Update SKILL.md workflow: new Step 5 for issue reporters, 8 steps total Co-Authored-By: Oz <oz-agent@warp.dev>
For commits that aren't squash-merges (no #PR in subject), walk the merge commit's second-parent lineage to find PRs brought in via merge commit (e.g. security-patches branch). Each discovered PR gets its own fetch_pr_data call so none are missed. Co-Authored-By: Oz <oz-agent@warp.dev>
fetch_issue_reporters.py now accepts --org flag to check each reporter's org membership via the GitHub API, filtering out internal members so they aren't misattributed as external community reporters. Co-Authored-By: Oz <oz-agent@warp.dev>
- fetch_prs.py: strip HTML comments before scanning for CHANGELOG markers so template placeholders aren't parsed as real opt-outs - fetch_prs.py: match trailing (#N) for PR number extraction to avoid grabbing issue numbers from squash titles - classify_contributors.py: only return external on explicit 404, treat all other failures as unknown - changelog_draft.yml: check out default branch instead of release tag so skill files are always available for older releases - example: remove Needs Review and Skipped sections to match output contract; add NONE to classify skill categories, remove IMAGE Co-Authored-By: Oz <oz-agent@warp.dev>
Replace the warpdotdev/generate-changelog action in the generate_changelogs
job with oz-agent-action, which invokes the changelog-draft skill to produce
a release-compatible JSON artifact (changelog-release.json).
Changes:
- SKILL.md Step 8: add third output file (changelog-release.json) with the
{newFeatures, improvements, bugFixes, images, oz_updates} schema consumed
by the Slack payload builder and in-app changelog.json steps.
- create_release.yml: replace GitHub App token + generate-changelog steps
with oz-agent-action + a bridge step that reads changelog-release.json
and exposes it as outputs.changelog for downstream compatibility.
- Fix .image -> .images key in the changelog.json builder step to match
the new schema.
Co-Authored-By: Oz <oz-agent@warp.dev>
|
I'm starting a first review of this pull request. You can view the conversation on Warp. I completed the review and no human review was requested for this pull request. Comment Powered by Oz |
![oz-for-oss[bot]](https://avatars.githubusercontent.com/in/3450139?s=60&v=4){kind=small}
There was a problem hiding this comment.
Overview
This PR adds an Oz changelog-draft skill, helper scripts, a manual changelog workflow, and replaces the release workflow's generate-changelog action with an Oz-generated changelog-release.json.
Concerns
- The Oz action steps assert that
ghis authenticated, but the workflows no longer pass a GitHub token into the action even though the new scripts depend ongh pr view,gh issue view, andgh api. - PR metadata fetch failures are silently dropped, so the audit invariant that every PR is accounted for can pass over an incomplete input set.
- Issue reporter attribution can include internal users when org-membership checks fail unexpectedly or when the same internal reporter appears on more than one linked issue.
Security
- The issue reporter filtering path can misattribute internal reporters as community reporters; treat org membership failures conservatively and cache internal/external state separately.
Verdict
Found: 0 critical, 5 important, 0 suggestions
Request changes
Comment /oz-review on this pull request to retrigger a review (up to 3 times on the same pull request).
Powered by Oz
|
|
||
| After writing the output files, print the full contents of changelog-draft.md to stdout so it appears in the workflow log. | ||
|
|
||
| You are running in a GitHub Actions workflow. The repo is checked out at the default branch (HEAD) with full history. Use the release_tag input as the git range endpoint — do NOT check out the release tag. `gh` is authenticated. Do not commit, push, or create PRs. |
There was a problem hiding this comment.
WARP_API_KEY while the skill runs gh pr view, gh issue view, and gh api; export a GitHub token with the needed permissions to the action or release changelog generation will fail or produce empty data.
|
|
||
| After writing the output files, print the full contents of changelog-draft.md to stdout so it appears in the workflow log. | ||
|
|
||
| You are running in a GitHub Actions workflow. The repo is checked out at the default branch (HEAD). Use the release_tag input as the git range endpoint — do NOT check out the release tag. `gh` is authenticated. Do not commit, push, or create PRs. |
There was a problem hiding this comment.
gh is authenticated, but the workflow never exposes GH_TOKEN/GITHUB_TOKEN to the Oz action; pass an authenticated token into the step so the helper scripts can read PRs, issues, and org membership.
| """Fetch and record a single PR by number.""" | ||
| data = fetch_pr_data(args.repo, pr_num) | ||
| if data is None: | ||
| return |
There was a problem hiding this comment.
gh pr view fails or returns invalid JSON, this silently omits the PR from prs, so the later accounting check cannot detect the missing PR; fail the script or emit an explicit fetch error instead of returning.
| # Auth failure — be conservative, treat as internal | ||
| if "403" in stderr or "401" in stderr or "saml" in stderr: | ||
| return True | ||
| return False |
There was a problem hiding this comment.
external; only an explicit 404 should return False, and all other failures should be conservative to avoid public-crediting internal reporters.
| continue | ||
| username = info["reporter"] | ||
| # Skip internal org members when --org is provided | ||
| if org and username not in seen_reporters and is_org_member(org, username): |
There was a problem hiding this comment.
seen_reporters and skipped only for the first issue; subsequent issues from the same internal user bypass this condition and are appended as community reporters, so cache membership status separately or continue for already-known internal reporters.
1 participant
Description
Replace the
warpdotdev/generate-changelogGitHub Action in thegenerate_changelogsjob with the Oz changelog-draft skill (oz-agent-action). This makes the release changelog generation AI-powered, using the same skill that was landed in #10280.What changed
changelog-release.jsonwith the{newFeatures, improvements, bugFixes, images, oz_updates}schema expected by the Slack payload builder and in-app changelog.json steps.generate_changelogsjob now:oz-agent-actionwith a prompt to follow the changelog-draft skill workflowchangelog-release.jsonand setsoutputs.changelogso all downstream steps (Slack post, GCS upload) work unchanged.imageto.imageskey in the changelog.json builder for schema consistencyHow it works
The Oz agent runs the full skill workflow (fetch PRs, classify contributors, extract feature flags, classify unmarked PRs, assemble draft) and writes three files:
changelog-draft.md- human-reviewable markdownchangelog-draft.json- machine-readable audit artifactchangelog-release.json- release-pipeline-compatible JSON (new)The bridge step loads
changelog-release.jsonintosteps.generate_changelog.outputs.changelog, maintaining the same interface the downstream Slack and GCS steps expect.Linked Issue
Testing
Validated YAML structure (1788 lines, all key references intact)
Generated a real
changelog-release.jsonfrom the latest stable release range (v0.2026.05.06 to v0.2026.05.13): 2 new features, 10 improvements, 27 bug fixes, 1 imageRan the exact downstream jq transforms (Slack payload builder + changelog.json builder) against the generated JSON - all produce valid output
Full end-to-end test will occur on the next release cut
I have manually tested my changes locally with
./script/runAgent Mode
Warp Conversation | Plan: Oz-based changelog draft generator
Co-Authored-By: Oz oz-agent@warp.dev