Skip to content

Add NIST NVD as a vulnerability source alongside OSV#2

Open
webchick wants to merge 1 commit into
mainfrom
add-nvd-vulnerability-source
Open

Add NIST NVD as a vulnerability source alongside OSV#2
webchick wants to merge 1 commit into
mainfrom
add-nvd-vulnerability-source

Conversation

@webchick

@webchick webchick commented Jun 5, 2026

Copy link
Copy Markdown
Contributor

Why

Scout's vulnerability checks query OSV only. OSV is an aggregator — for PyPI it mirrors the GitHub Advisory Database (GHSA) and the PyPA Advisory Database, and has no independent analysts of its own. CVEs that arrive through a different pipeline (for example a huntr.dev disclosure that lands in NIST NVD without a corresponding GHSA or PyPA advisory ever being filed) never reach OSV's feeds.

That's a coverage gap, not a timing lag — no amount of waiting closes it, because nothing upstream of OSV ever describes the vulnerability. Querying NVD directly is a structurally different source that catches these.

What

A new NVD check that runs in parallel with the existing OSV check:

  • checks/nvd.py — new activities.nvd.check. Queries the NVD REST API v2.0 and lets NVD do server-side version-range matching via virtualMatchString (CPE product = package name, vendor wildcarded). An optional NVD_API_KEY raises the rate limit from 5→50 requests/30s; without it the check still works (graceful degradation, consistent with the project's zero-keys principle). A package whose CPE product name doesn't line up with the registry name yields an empty result — a miss, never a false positive.
  • models/ — new NVDChecks model and nvd field on PackageChecks.
  • classifiers/ — NVD vulnerabilities force a hard RED, unioned and deduped with OSV so a hit in either feed is sufficient. The LLM classifier surfaces the new signal through the existing model_dump path; prompt label updated to name NVD.
  • workflows/ — appended to the check registry (append-only, replay-safe).

Trade-off

NVD's CPE-based matching is looser than GHSA's curated PyPI version ranges, so it can surface more false-positive version matches than OSV alone. The dedup with OSV and the hard-RED-only-on-known-CVE behavior keep this contained, but it's the reason this is additive coverage rather than a replacement for OSV.

Tests

  • 5 new NVD activity tests (tests/test_activities.py), 100% coverage of checks/nvd.py.
  • Replay fixtures regenerated for the new activity in the schedule sequence.
  • Full suite passes (1414 tests); ruff and mypy clean.

🤖 Generated with Claude Code

OSV aggregates GitHub Security Advisories (GHSA) and the PyPA Advisory
Database for PyPI; it has no independent analysts. CVEs that arrive
through other pipelines — e.g. a huntr.dev disclosure that lands in NVD
without a corresponding GHSA/PyPA advisory ever being filed — never reach
OSV's feeds, so an OSV-only check has a structural blind spot rather than
just a timing lag.

This adds a parallel NVD check that closes that gap:

- checks/nvd.py: new `activities.nvd.check` activity. Queries the NVD
  REST API v2.0 and lets NVD do server-side version-range matching via
  virtualMatchString (CPE product = package name, vendor wildcarded).
  Optional NVD_API_KEY raises the rate limit 5→50 req/30s; absent, it
  still works (graceful degradation). A package whose CPE product name
  doesn't match the registry name yields an empty result — a miss, never
  a false positive.
- models: new NVDChecks model and `nvd` field on PackageChecks.
- classifiers: NVD vulnerabilities now force a hard RED, unioned and
  deduped with OSV so a hit in either feed is sufficient; the LLM prompt
  surfaces the nvd signal via the existing model_dump path.
- workflows: appended to the check registry (append-only, replay-safe).
- tests: 5 new NVD activity tests (100% coverage of checks/nvd.py);
  replay fixtures regenerated for the new activity in the schedule
  sequence. Full suite passes; ruff and mypy clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant