Skip to content

Commit 8a186dd

Browse files
Add correction tracking workflow and TypeScript scripts (Phase 2)
- track-correction.yml: GitHub Action triggered on label/comment events - Detects when humans correct AI triage classifications - Records corrections as JSON files on triage-corrections branch - Opens/reuses a single PR for review - Concurrency control per issue to prevent race conditions - Maintainer-only permission guard (write/maintain/admin) - scripts/corrections/: TypeScript package with full test coverage - track-correction.ts: correction/confirmation detection logic - write-correction.ts: branch/file/PR management - update-context-comments.ts: late justification comment appender - 37 tests (unit + integration) covering all scenarios - justfile: Added install-corrections, test-corrections, lint-corrections Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent e4a008e commit 8a186dd

19 files changed

Lines changed: 4056 additions & 1 deletion
Lines changed: 63 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,63 @@
1+
name: "Track Classification Corrections"
2+
3+
on:
4+
issues:
5+
types: [labeled, unlabeled]
6+
issue_comment:
7+
types: [created]
8+
9+
concurrency:
10+
group: triage-corrections-${{ github.event.issue.number }}
11+
cancel-in-progress: false
12+
13+
permissions:
14+
contents: write
15+
pull-requests: write
16+
issues: read
17+
18+
jobs:
19+
track-correction:
20+
name: "Track correction or confirmation"
21+
if: github.event_name == 'issues'
22+
runs-on: ubuntu-latest
23+
steps:
24+
- uses: actions/checkout@v4
25+
26+
- uses: actions/setup-node@v4
27+
with:
28+
node-version: 22
29+
30+
- run: npm ci
31+
working-directory: scripts/corrections
32+
33+
- run: npm run build
34+
working-directory: scripts/corrections
35+
36+
- uses: actions/github-script@v7
37+
with:
38+
script: |
39+
const script = require('./scripts/corrections/dist/track-correction.js')
40+
await script({github, context, core})
41+
42+
update-context-comments:
43+
name: "Append late justification comment"
44+
if: github.event_name == 'issue_comment'
45+
runs-on: ubuntu-latest
46+
steps:
47+
- uses: actions/checkout@v4
48+
49+
- uses: actions/setup-node@v4
50+
with:
51+
node-version: 22
52+
53+
- run: npm ci
54+
working-directory: scripts/corrections
55+
56+
- run: npm run build
57+
working-directory: scripts/corrections
58+
59+
- uses: actions/github-script@v7
60+
with:
61+
script: |
62+
const script = require('./scripts/corrections/dist/update-context-comments.js')
63+
await script({github, context, core})

evals/corrections/.gitkeep

Whitespace-only changes.

evals/results/.gitkeep

Whitespace-only changes.

justfile

Lines changed: 16 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -72,7 +72,7 @@ test-dotnet:
7272
@cd dotnet && dotnet test test/GitHub.Copilot.SDK.Test.csproj
7373

7474
# Install all dependencies across all languages
75-
install: install-go install-python install-nodejs install-dotnet
75+
install: install-go install-python install-nodejs install-dotnet install-corrections
7676
@echo "✅ All dependencies installed"
7777

7878
# Install Go dependencies and prerequisites for tests
@@ -100,6 +100,21 @@ install-test-harness:
100100
@echo "=== Installing test harness dependencies ==="
101101
@cd test/harness && npm ci --ignore-scripts
102102

103+
# Install triage agent correction tracking dependencies
104+
install-corrections:
105+
@echo "=== Installing correction tracking dependencies ==="
106+
@cd scripts/corrections && npm ci
107+
108+
# Run triage agent correction tracking tests
109+
test-corrections:
110+
@echo "=== Testing correction tracking ==="
111+
@cd scripts/corrections && npm test
112+
113+
# Typecheck triage agent correction tracking scripts
114+
lint-corrections:
115+
@echo "=== Typechecking correction tracking ==="
116+
@cd scripts/corrections && npm run typecheck
117+
103118
# Run interactive SDK playground
104119
playground:
105120
@echo "=== Starting SDK Playground ==="

scripts/corrections/.gitignore

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
node_modules/
2+
dist/

0 commit comments

Comments
 (0)