Skip to content

Replace Ask ChatGPT with provider-agnostic AI Review#47

Open
jgamblin wants to merge 1 commit intoCERTCC:mainfrom
jgamblin:feature/ai-review-provider
Open

Replace Ask ChatGPT with provider-agnostic AI Review#47
jgamblin wants to merge 1 commit intoCERTCC:mainfrom
jgamblin:feature/ai-review-provider

Conversation

@jgamblin
Copy link
Copy Markdown
Contributor

@jgamblin jgamblin commented Apr 1, 2026

Summary

  • Replaces the single-provider "Ask ChatGPT" button with a transparent, provider-agnostic AI Review workflow
  • New modal displays the full prompt text before sending so users can review exactly what CVE data will be shared
  • Supports ChatGPT, Claude, and Gemini via a dropdown selector
  • Improved prompt with a structured 6-point CVE quality review (description, affected block, schema compliance, CWE, references, completeness)

Why

The previous implementation silently opened chat.openai.com with the entire CVE record in the URL — no transparency about what was being sent, no provider choice, and a weak prompt. For a security tool, users should see exactly what leaves the app before it does.

How it works

  1. User clicks AI Review in the CVE modal footer
  2. A new modal shows the full prompt (read-only) with their CVE JSON included
  3. User selects their preferred AI provider from the dropdown
  4. User clicks Copy & Open AI — prompt is copied to clipboard, provider opens in a new tab
  5. User pastes the prompt into the AI chat

Before / After

Before: Single "Ask ChatGPT" button

Before

After: "AI Review" button

After button

After: AI Review modal with prompt preview and provider selection

After modal

Prompt design and testing

The old prompt was a single sentence: "I have this CVE record and want help improve it especially the affected block." This produced generic, surface-level feedback.

The new prompt was iteratively designed and tested against Claude using a sample CVE record (api/cve/CVE-9999-0000) from the repo. It uses a structured 6-point review framework:

  1. Description Quality — checks if the description follows the standard pattern: "[Vulnerability type] in [component] in [product] [version] allows [attacker type] to [impact] via [vector]"
  2. Affected Block — evaluates whether version ranges use versionType, lessThan/lessThanOrEqual, and defaultStatus instead of just listing exact versions
  3. Schema Compliance — validates against CVE JSON 5.x schema guidance
  4. CWE Classification — checks for proper cweId field usage (not just CWE in description text)
  5. References — verifies URLs are complete and tagged with types like advisory, patch, vendor-advisory
  6. Completeness — flags missing recommended fields like CVSS metrics, timeline, and additional affected products

When tested, the prompt produced a detailed review that correctly identified 5 issues in the sample record:

  • Description was a meta-statement ("is a Test CVE Record") rather than describing the actual vulnerability
  • Version fields used only exact versions with no ranges or versionType
  • CWE was embedded in description text without the machine-readable cweId field
  • Reference URL was incomplete and lacked type tags
  • Missing CVSS metrics and timeline fields

The AI response included a corrected version of the complete JSON with all improvements applied — exactly what a CNA needs to quickly improve their record before publication.

Test plan

  • Click "AI Review" with valid CVE JSON — modal shows with full prompt
  • Click "AI Review" with invalid JSON — error dialog appears
  • Select each provider (ChatGPT, Claude, Gemini) and click "Copy & Open AI" — correct URL opens, prompt is in clipboard
  • Click Cancel — modal closes, nothing is copied or opened
  • Verify prompt text is read-only in the textarea

🤖 Generated with Claude Code

@jgamblin jgamblin force-pushed the feature/ai-review-provider branch from d03168a to 3c74fa7 Compare April 2, 2026 18:31
jgamblin added a commit to jgamblin/cveClient that referenced this pull request Apr 2, 2026
Since PRs CERTCC#46 (schema) and CERTCC#47 (AI review) will also be merged,
update the README to reference those features.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@jgamblin jgamblin force-pushed the feature/ai-review-provider branch from d9598b9 to 3c74fa7 Compare April 2, 2026 18:36
@jgamblin
Copy link
Copy Markdown
Contributor Author

jgamblin commented Apr 2, 2026

Note: PR #46 should be merged before this one. It includes a test fix for getadp/deleteadp methods that were removed in PR #45. Once #46 is on main, this PR will need a rebase to pick up that fix and pass CI.

@jgamblin jgamblin mentioned this pull request Apr 2, 2026
4 tasks
Replace the single-provider "Ask ChatGPT" button with a transparent,
provider-agnostic "AI Review" workflow:

- New modal shows the full prompt text before sending, so users can
  review exactly what data will be shared with the AI provider
- Dropdown to select provider: ChatGPT, Claude, or Gemini
- Copies prompt to clipboard and opens the selected provider
- Improved prompt with structured 6-point CVE quality review covering
  description quality, affected block, schema compliance, CWE
  classification, references, and completeness
- Adds JSON parse error handling and fixes "inut" typo

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@jgamblin jgamblin force-pushed the feature/ai-review-provider branch from 3c74fa7 to 88ae31c Compare April 2, 2026 19:00
jgamblin added a commit to jgamblin/cveClient that referenced this pull request Apr 2, 2026
Since PRs CERTCC#46 (schema) and CERTCC#47 (AI review) will also be merged,
update the README to reference those features.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant