Replace Ask ChatGPT with provider-agnostic AI Review#47
Open
jgamblin wants to merge 1 commit intoCERTCC:mainfrom
Open
Replace Ask ChatGPT with provider-agnostic AI Review#47jgamblin wants to merge 1 commit intoCERTCC:mainfrom
jgamblin wants to merge 1 commit intoCERTCC:mainfrom
Conversation
d03168a to
3c74fa7
Compare
d9598b9 to
3c74fa7
Compare
Contributor
Author
4 tasks
Replace the single-provider "Ask ChatGPT" button with a transparent, provider-agnostic "AI Review" workflow: - New modal shows the full prompt text before sending, so users can review exactly what data will be shared with the AI provider - Dropdown to select provider: ChatGPT, Claude, or Gemini - Copies prompt to clipboard and opens the selected provider - Improved prompt with structured 6-point CVE quality review covering description quality, affected block, schema compliance, CWE classification, references, and completeness - Adds JSON parse error handling and fixes "inut" typo Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
3c74fa7 to
88ae31c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Why
The previous implementation silently opened
chat.openai.comwith the entire CVE record in the URL — no transparency about what was being sent, no provider choice, and a weak prompt. For a security tool, users should see exactly what leaves the app before it does.How it works
Before / After
Before: Single "Ask ChatGPT" button
After: "AI Review" button
After: AI Review modal with prompt preview and provider selection
Prompt design and testing
The old prompt was a single sentence: "I have this CVE record and want help improve it especially the affected block." This produced generic, surface-level feedback.
The new prompt was iteratively designed and tested against Claude using a sample CVE record (
api/cve/CVE-9999-0000) from the repo. It uses a structured 6-point review framework:versionType,lessThan/lessThanOrEqual, anddefaultStatusinstead of just listing exact versionscweIdfield usage (not just CWE in description text)advisory,patch,vendor-advisoryWhen tested, the prompt produced a detailed review that correctly identified 5 issues in the sample record:
versionTypecweIdfieldThe AI response included a corrected version of the complete JSON with all improvements applied — exactly what a CNA needs to quickly improve their record before publication.
Test plan
🤖 Generated with Claude Code