Add NIST NVD as a vulnerability source alongside OSV#2
Open
webchick wants to merge 1 commit into
Open
Conversation
OSV aggregates GitHub Security Advisories (GHSA) and the PyPA Advisory Database for PyPI; it has no independent analysts. CVEs that arrive through other pipelines — e.g. a huntr.dev disclosure that lands in NVD without a corresponding GHSA/PyPA advisory ever being filed — never reach OSV's feeds, so an OSV-only check has a structural blind spot rather than just a timing lag. This adds a parallel NVD check that closes that gap: - checks/nvd.py: new `activities.nvd.check` activity. Queries the NVD REST API v2.0 and lets NVD do server-side version-range matching via virtualMatchString (CPE product = package name, vendor wildcarded). Optional NVD_API_KEY raises the rate limit 5→50 req/30s; absent, it still works (graceful degradation). A package whose CPE product name doesn't match the registry name yields an empty result — a miss, never a false positive. - models: new NVDChecks model and `nvd` field on PackageChecks. - classifiers: NVD vulnerabilities now force a hard RED, unioned and deduped with OSV so a hit in either feed is sufficient; the LLM prompt surfaces the nvd signal via the existing model_dump path. - workflows: appended to the check registry (append-only, replay-safe). - tests: 5 new NVD activity tests (100% coverage of checks/nvd.py); replay fixtures regenerated for the new activity in the schedule sequence. Full suite passes; ruff and mypy clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Scout's vulnerability checks query OSV only. OSV is an aggregator — for PyPI it mirrors the GitHub Advisory Database (GHSA) and the PyPA Advisory Database, and has no independent analysts of its own. CVEs that arrive through a different pipeline (for example a huntr.dev disclosure that lands in NIST NVD without a corresponding GHSA or PyPA advisory ever being filed) never reach OSV's feeds.
That's a coverage gap, not a timing lag — no amount of waiting closes it, because nothing upstream of OSV ever describes the vulnerability. Querying NVD directly is a structurally different source that catches these.
What
A new NVD check that runs in parallel with the existing OSV check:
checks/nvd.py— newactivities.nvd.check. Queries the NVD REST API v2.0 and lets NVD do server-side version-range matching viavirtualMatchString(CPE product = package name, vendor wildcarded). An optionalNVD_API_KEYraises the rate limit from 5→50 requests/30s; without it the check still works (graceful degradation, consistent with the project's zero-keys principle). A package whose CPE product name doesn't line up with the registry name yields an empty result — a miss, never a false positive.models/— newNVDChecksmodel andnvdfield onPackageChecks.classifiers/— NVD vulnerabilities force a hard RED, unioned and deduped with OSV so a hit in either feed is sufficient. The LLM classifier surfaces the new signal through the existingmodel_dumppath; prompt label updated to name NVD.workflows/— appended to the check registry (append-only, replay-safe).Trade-off
NVD's CPE-based matching is looser than GHSA's curated PyPI version ranges, so it can surface more false-positive version matches than OSV alone. The dedup with OSV and the hard-RED-only-on-known-CVE behavior keep this contained, but it's the reason this is additive coverage rather than a replacement for OSV.
Tests
tests/test_activities.py), 100% coverage ofchecks/nvd.py.🤖 Generated with Claude Code