Skip to content

Feat/v2 agent scanner pr diff annotation#57

Merged
CalebAbhulimhen merged 6 commits into
mainfrom
feat/v2-agent-scanner-pr-diff-annotation
Apr 13, 2026
Merged

Feat/v2 agent scanner pr diff annotation#57
CalebAbhulimhen merged 6 commits into
mainfrom
feat/v2-agent-scanner-pr-diff-annotation

Conversation

@CalebAbhulimhen

Copy link
Copy Markdown
Collaborator

⏺ V2 Agent Scanner + PR Diff-Aware Analysis

Summary

This PR introduces a production-ready V2 scanning codepath powered by Pydantic-AI, alongside a smarter PR review mode that focuses the agent's attention on the
lines that actually changed.


What's new

V2 Agent Scanner (core/runner_v2.py, core/code_scanner/agent_scanner.py)

  • Scans files one at a time using a Pydantic-AI Agent, returning structured FileScanResult output instead of a raw markdown string
  • Streams results to stdout progressively as each file is scanned
  • Supports all three scan modes: full directory, local git diff (--changes_only), and GitHub PR

GitHub PR Integration (core/utils/github_integration.py)

  • Posts findings as inline review comments directly on the relevant line in the PR diff
  • Falls back to a regular issue comment if the line isn't in the diff (e.g. architectural findings with no specific line)

PR Diff Annotation (core/utils/file_extractor.py)

  • Parses unified diff patches from the GitHub API (get_pr_changed_line_numbers) and local git diff output (get_local_changed_line_numbers) to extract the exact
    new-file line numbers that changed
  • When scanning in PR or --changes_only mode, changed lines are marked with [CHANGED] in the prompt sent to the agent (e.g. 14: [CHANGED] def foo():)
  • Full file content is still sent — context is preserved for accurate data flow analysis — but the agent is instructed to prioritise [CHANGED] lines

Eval harness (evals/)

  • 5 fixture files covering auth, data pipelines, file handling, mobile backend, and web APIs
  • expected_findings.json splits findings into standard (required from any model) and advanced (required from gpt-4o only) tiers
  • 80% threshold per fixture; exit code 0 = all pass
  • Verified: all 5 fixtures pass at 100% both with and without [CHANGED] markers

CLAUDE.md

  • Documents all commands: install, test, lint, build, evals, and CLI usage
  • Architecture overview of both V1 and V2 codepaths

- File-by-file scanning via Pydantic-AI Agent with structured FileScanResult output
- GitHub integration for inline PR review comments
- [CHANGED] line markers when scanning PR diffs or local git changes
- Diff parsing helpers (get_pr_changed_line_numbers, get_local_changed_line_numbers)
- Agent prompt updated to prioritise [CHANGED] lines while retaining full file context
- Eval harness and 5 fixture files for regression testing
@CalebAbhulimhen CalebAbhulimhen merged commit 1ef6156 into main Apr 13, 2026
9 checks passed
@CalebAbhulimhen CalebAbhulimhen deleted the feat/v2-agent-scanner-pr-diff-annotation branch April 13, 2026 04:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant