VEX (Vulnerability Exploitability eXchange) capability

[dasiths]

VEX (Vulnerability Exploitability eXchange) capability

Description

Introduce an end-to-end VEX (Vulnerability Exploitability eXchange) capability to HVE Core, enabling machine-readable, signed documentation of per-CVE vulnerability impact and reachability. A scanner reports "you have N CVEs," but most are not exploitable in this product: the vulnerable function is never called, the code path is unreachable, or the dependency is only used at build time. VEX records, per CVE, why a vulnerability does or does not affect HVE Core. It turns scanner noise into accountable answers and lets VEX-aware scanners (Trivy, Grype) suppress findings the maintainers have assessed as not_affected.

The capability spans two complementary tracks that converge on an AI-assisted, human-reviewed drafting loop:

• Workflow track (plumbing): a signed VEX document published and attested with each release, a scheduled detection workflow, and an AI drafting workflow.
• Tooling track (the brain): an OpenVEX skill, a VEX Generator agent with a CVE Analyzer subagent, and /vex-scan + /vex-triage prompts.

The trust model is AI drafts, human merges: the agent performs every step except the merge click; the merge-commit author is the accountable author of record, and the agent is forbidden from asserting not_affected at low confidence. The PR also includes extensive validation and a clear licensing posture.

The most important changes are:

Workflow and Foundation

• Added a signed OpenVEX document (security/vex/hve-core.openvex.json) published with each release and attested twice via the attest-and-upload-vex job in release-stable.yml: a build-provenance attestation of the VEX file, and an in-toto attestation that binds the VEX document as the predicate over the dependency SBOM subject (the OpenVEX encapsulating format).
• Integrated a detection workflow (.github/workflows/vex-detect.yml) that scans dependencies with OSV-Scanner, diffs findings against the VEX document, and files a single deduplicated triage issue for untriaged or status-drifted CVEs.
• Implemented an AI-powered drafting workflow (.github/workflows/vex-draft.md and compiled vex-draft.lock.yml, gh-aw engine copilot) that opens one pull request with an updated VEX document, ensuring only human-reviewed changes are merged.

Tooling and Agent Intelligence

• Introduced the OpenVEX skill and references (.github/skills/security/openvex-spec/: SKILL, schema, status-logic, and CVE-data-source references) plus the vex-standards and vex-generation instructions.
• Added the VEX Generator agent (.github/agents/security/vex-generator.agent.md) and a dedicated CVE Analyzer subagent (.github/agents/security/subagents/cve-analyzer.agent.md) for deep, evidence-based exploitability analysis.
• CVE enrichment resolves each finding across OSV.dev, NVD, and the GitHub Advisory Database, collecting every identifier and querying each source by the id it is keyed on (GHSA-/PYSEC- for OSV and GitHub, CVE- for NVD) with alias fallback, so records are resolved regardless of which identifier a scanner reports.
• Added /vex-scan and /vex-triage prompts for interactive triage and evidence collection, with collection registration and a new VEX triage PR template that standardizes evidence requirements and reviewer checklists.

Integration and Environment

• Updated the devcontainer setup (.devcontainer/scripts/on-create.sh) and copilot-setup-steps.yml to install and verify a SHA-pinned osv-scanner for local and CI-based vulnerability scanning, ensuring reproducible and secure builds.

Documentation and Compliance

• Added docs/security/vex-verification.md (consumer guide) and docs/agents/security/vex-generator.md (agent docs); added control SC-9 to docs/security/security-model.md; linked the guide from docs/security/README.md; and updated SECURITY.md with the .openvex.json artifact row, a VEX verification subsection, and a signed-artifacts row.

Vocabulary and Codebase Hygiene

• Expanded the spell-check and acronym dictionaries (PYSEC, deprioritize, and other VEX/CVE terms) to improve codebase consistency and reduce false-positive linter warnings.

These changes collectively deliver a robust, auditable, and automatable VEX workflow, raising the security assurance and transparency of HVE Core.

Artifact inventory

Area	Artifacts
Foundation	security/vex/hve-core.openvex.json (OpenVEX v0.2.0 document); attest-and-upload-vex job in release-stable.yml (provenance + VEX-as-predicate-over-SBOM attestations)
Skill + instructions	.github/skills/security/openvex-spec/ (SKILL + schema, status-logic, and CVE-data-source references); vex-standards and vex-generation instructions
Agent	.github/agents/security/vex-generator.agent.md + subagents/cve-analyzer.agent.md
Prompts	/vex-scan and /vex-triage; collection registration; .github/PULL_REQUEST_TEMPLATE/vex-triage.md
Detection workflow	.github/workflows/vex-detect.yml (OSV-Scanner scan, diff against the VEX doc, file a triage issue)
Drafting workflow	.github/workflows/vex-draft.md + compiled vex-draft.lock.yml (gh-aw, engine: copilot)
Environment	osv-scanner install in .devcontainer/scripts/on-create.sh and copilot-setup-steps.yml
Documentation	docs/security/vex-verification.md, docs/agents/security/vex-generator.md; updates to docs/security/security-model.md (control SC-9), docs/security/README.md, and SECURITY.md

Architecture

flowchart TB
    subgraph Tooling["Tooling track (the brain)"]
        Skill["openvex-spec skill"]
        Agent["vex-generator agent + cve-analyzer subagent"]
        Prompts["/vex-scan + /vex-triage"]
        Skill --> Agent --> Prompts
    end
    subgraph Workflow["Workflow track (the plumbing)"]
        Foundation["VEX document + release attestation"]
        Detect["vex-detect.yml"]
        Draft["vex-draft.md (gh-aw)"]
        Foundation --> Detect --> Draft
    end
    Agent -. powers .-> Draft
    Workflow --> Docs["Documentation"]
    Tooling --> Docs

Loading

How it works once live

1. Release publishes the signed VEX document alongside the SBOM, attested via Sigstore (a provenance attestation of the VEX file plus a VEX-as-predicate attestation over the dependency SBOM).
2. Detection (vex-detect.yml) re-scans on a schedule and after each release, diffs OSV-Scanner findings against the VEX document, and files a single deduplicated triage issue for untriaged or non-terminal-status CVEs.
3. AI drafting (vex-draft.md) triggers off the detection run, invokes the VEX Generator agent to enrich each CVE, analyze reachability, route by confidence, and opens one pull request with an updated OpenVEX document.
4. Human review merges the PR. AI drafts, human merges; the merge commit author is the accountable author of record.
5. Consumers download the VEX document, verify its attestation, and apply it with trivy --vex / grype --vex to suppress not_affected findings.

Related Issue(s)

#1220
#1221

Type of Change

Select all that apply:

Code & Documentation:

• Bug fix (non-breaking change fixing an issue)
• New feature (non-breaking change adding functionality)
• Breaking change (fix or feature causing existing functionality to change)
• Documentation update

Infrastructure & Configuration:

• GitHub Actions workflow
• Linting configuration (markdown, PowerShell, etc.)
• Security configuration
• DevContainer configuration
• Dependency update

AI Artifacts:

• Reviewed contribution with prompt-builder agent and addressed all feedback
• Copilot instructions (.github/instructions/*.instructions.md)
• Copilot prompt (.github/prompts/*.prompt.md)
• Copilot agent (.github/agents/*.agent.md)
• Copilot skill (.github/skills/*/SKILL.md)

Note for AI Artifact Contributors:

• Agents: Research, indexing/referencing other project (using standard VS Code GitHub Copilot/MCP tools), planning, and general implementation agents likely already exist. Review .github/agents/ before creating new ones.
• Skills: Must include both bash and PowerShell scripts. See Skills.
• Model Versions: Only contributions targeting the latest Anthropic and OpenAI models will be accepted. Older model versions (e.g., GPT-3.5, Claude 3) will be rejected.
• See Agents Not Accepted and Model Version Requirements.

Other:

• Script/automation (.ps1, .sh, .py)
• Other (please describe):

Sample Prompts (for AI Artifact Contributions)

User Request:

Scan our dependencies for vulnerabilities and draft a VEX document for the ones that do not affect us.

This invokes the VEX Generator agent through the /vex-scan prompt (Mode 1). /vex-triage report=osv-results.json runs Mode 2 against an existing scan report instead.

Execution Flow:

1. The agent runs a Trivy scan over the scope (or reads the provided report in Mode 2) and captures the findings.
2. For each finding it collects every identifier (CVE plus GHSA-/PYSEC- aliases) and enriches from OSV.dev, NVD, and the GitHub Advisory Database, querying each source by the id it is keyed on and falling back across aliases.
3. It delegates per-CVE reachability analysis to the CVE Analyzer subagent, which traces the import path, confirms dead code, or identifies a mitigation, with file and line citations.
4. It routes each finding by confidence band and drafts a status. Where reachability cannot be determined, it draws under_investigation and never not_affected.
5. It writes the updated OpenVEX document plus a triage report (executive summary, per-CVE technical findings, OpenVEX JSON).

Output Artifacts:

The updated security/vex/hve-core.openvex.json, for example:

{
  "@context": "https://openvex.dev/ns/v0.2.0",
  "@id": "https://github.com/microsoft/hve-core/security/vex/2026-06-18",
  "author": "Microsoft HVE Core Maintainers",
  "version": 2,
  "statements": [
    {
      "vulnerability": { "name": "CVE-2026-53550", "aliases": ["GHSA-h67p-54hq-rp68"] },
      "products": [{ "@id": "pkg:npm/js-yaml@3.14.2" }],
      "status": "not_affected",
      "justification": "vulnerable_code_not_in_execute_path"
    }
  ]
}

Success Indicators:

The document validates against the OpenVEX v0.2.0 schema; every not_affected statement carries a justification code and code-citation evidence; ambiguous findings are under_investigation; and the triage report lists the human-touch items. The PR touches only the VEX document.

For detailed contribution requirements, see:

• Common Standards: docs/contributing/ai-artifacts-common.md - Shared standards for XML blocks, markdown quality, RFC 2119, validation, and testing
• Agents: docs/contributing/custom-agents.md - Agent configurations with tools and behavior patterns
• Prompts: docs/contributing/prompts.md - Workflow-specific guidance with template variables
• Instructions: docs/contributing/instructions.md - Technology-specific standards with glob patterns
• Skills: docs/contributing/skills.md - Task execution utilities with cross-platform scripts

Testing

The capability was validated across seven layers: static re-validation, OpenVEX schema conformance, a local run of the detection workflow against the real dependency set, a bounded run of the agent pipeline, the integration seams, the release-attestation wiring (static), and a live end-to-end run of the drafting workflow.

Live drafting run (gh-aw, GitHub Copilot engine)

A real run of the VEX Drafting workflow on the fork, dispatched against a seeded detection issue, drafted a schema-valid OpenVEX update as a pull request:

• Detection issue (work list): VEX detection: untriaged vulnerabilities found dasiths/hve-core#20
• Workflow run (Actions): https://github.com/dasiths/hve-core/actions/runs/27705375100
• Drafted output (PR): VEX: draft status for untriaged findings dasiths/hve-core#22

The run confirmed the full pipeline end to end: the activation guard fired correctly (no spurious noop), the agent ran under the Copilot engine, CVE enrichment resolved advisories across OSV.dev, NVD, and the GitHub Advisory Database, and the safe-output PR touched only security/vex/hve-core.openvex.json. The agent produced evidence-backed not_affected determinations (each with a justification code and CVSS / codebase citations), and the forbidden-transition guard held (no unevidenced not_affected).

Validation results

The following all pass on this branch: gh aw compile (0 errors / 0 warnings), lint:yaml (actionlint), lint:md, lint:frontmatter, lint:md-links, spell-check, lint:dependency-pinning, lint:permissions, validate:skills, and lint:ai-artifacts.

Checklist

Required Checks

• Documentation is updated (if applicable)
• Files follow existing naming conventions
• Changes are backwards compatible (if applicable)
• Tests added for new functionality (if applicable)

AI Artifact Contributions

• Used /prompt-analyze to review contribution
• Addressed all feedback from prompt-builder review
• Verified contribution follows common standards and type-specific requirements

Required Automated Checks

The following validation commands must pass before merging:

• Markdown linting: npm run lint:md
• Spell checking: npm run spell-check
• Frontmatter validation: npm run lint:frontmatter
• Skill structure validation: npm run validate:skills
• Link validation: npm run lint:md-links
• PowerShell analysis: npm run lint:ps
• Plugin freshness: npm run plugin:generate
• Docusaurus tests: npm run docs:test

The lint:md, spell-check, lint:frontmatter, validate:skills, and lint:md-links checks were run on this branch and pass (see Testing above). Re-run all checks in CI before merging and tick each box once green.

Security Considerations

CVE advisory data is used under per-source licenses: NVD (US Government public domain) for CVSS and CWE; OSV.dev (mixed, routed by record prefix); GitHub Advisory Database (CC-BY-4.0) for identifiers, factual metadata, and reference URLs only. The agent writes original impact and remediation prose and does not quote or closely paraphrase advisory text. OpenVEX template text is CC0. Workflows declare least-privilege permissions, pin actions to SHA, and verify the osv-scanner download by checksum.

• This PR does not contain any sensitive or NDA information
• Any new dependencies have been reviewed for security issues
• Security-related scripts follow the principle of least privilege

Additional Notes

Maturity and known follow-ups:

• All new artifacts ship at experimental maturity. Promotion to stable follows validation across three or more codebases with a 5% or lower false-positive rate on not_affected determinations.
• Autonomous PR identity: the drafting workflow uses gh-aw's GH_AW_GITHUB_TOKEN seam (falling back to GITHUB_TOKEN); populate it from a GitHub App so drafted PRs trigger downstream checks.

• Reviewed and validated by a qualified human reviewer

[dasiths]

Implementation tracking epic (on the contributor fork): dasiths/hve-core#10 — breaks the work into the eight delivered phases across the workflow and tooling tracks, with per-phase PRs and the live end-to-end drafting-workflow validation run.

All CI checks on this PR are currently green (table-format and ms.date freshness were fixed in the latest commit).

[bindsi]

Approved: the VEX detection workflow is scoped with appropriate permissions and robust handling of OSV/OpenVEX data. I did not find actionable issues.

[katriendg]

Thanks @dasiths, I really appreciate the depth of thinking and iteration that went into this. It clearly builds on the prior discussions we've been having offline, the iterations you've done, the care in validating some of the harder aspects of this space, especially around non‑determinism, validation, and end-to-end workflow design.

The way you've structured the draft-and-merge accountability model, introduced the human-in-the-loop validation, and handled the forbidden-transition guard all give a lot of confidence that this isn't just a conceptual experiment, but something that's been exercised with real care. The licensing posture across OSV/NVD/GHSA and the release-time Sigstore attestation are also exactly in line with the direction we want to go.

I also really value the effort you've put into actually validating the pipeline — the live gh-aw drafting run makes a big difference in demonstrating that this holds together in practice. Shipping this in experimental maturity and learning from the initial runs within HVE Core ourselves, and with the AW workflow, feels like the right and responsible call given where we are.

My feedback is mostly about hardening the trust story before we lean on this, not about the core design. Leaving some inline comments below, plus a few general items.

• Correctness/completeness: I'd like an eval loop / labeled CVE corpus before we trust not_affected at scale. The repo now has first-class eval support, so let's track that as a post-merge backlog issue and make it the gate for promotion to stable.
• Human-in-the-loop: love the pattern; let's make the CODEOWNERS gate enforced (branch protection) rather than convention, and close out the autonomous-PR-identity follow-up so drafted PRs trigger required checks.
• RAI/limitations: the AI-generated disclaimer is well covered in the docs; one small gap is embedding provenance in the published JSON itself.
• A couple of convention nits inline (skill-by-name references).

---

Track eval coverage with a post-merge backlog issue

This capability has no eval coverage yet, and the repo now has first-class support for evals that new AI artifacts are expected to wire up — see docs/contributing/evals-ci.md and the "Adding New Evals" section in evals/README.md. Given the known variability in LLM outputs and the multi-step reachability reasoning here, please open a post-merge backlog / tracking issue to add a vally eval suite for the VEX agents before this leaves experimental. Concretely: an evals/agent-behavior/ (and/or skill-quality) suite with a labeled CVE corpus (known reachable vs. unreachable cases) that asserts the forbidden-transition guard and measures the not_affected false-positive rate, using the copilot-sdk executor with runs: 3+. That eval becomes the machine-checkable gate for the documented promotion criteria (≤5% FP on not_affected across 3+ codebases). Landing as experimental now is fine; the eval issue should be tracked as the promotion blocker.

Enforce the human-in-the-loop gate

The draft-and-merge model and forbidden-transition guard are excellent. Two reinforcements so the human gate is enforced rather than conventional: (a) add branch protection requiring CODEOWNERS review on /security/vex/** (the CODEOWNERS entry is in place, but review enforcement needs the branch rule); (b) resolve the open autonomous-PR-identity follow-up (the GH_AW_GITHUB_TOKEN / GitHub App seam) so drafted PRs trigger required downstream checks — a PR authored by the default GITHUB_TOKEN may not.

Source provenance / licensing confirmation

The openvex-spec skill frontmatter lists metadata.skill_based_on: https://github.com/dasiths/ai_generated_vex (a personal repo). This may not be something we want to keep, regarding licensing and supportability — do we really need it, since you're the git contributor for the PR anyway? Everything else on licensing (OpenVEX Apache-2.0/CC0, NVD public domain, GHSA URL-only) looks well-handled.

Embed AI-assisted provenance in the published JSON

The AI-generated/human-reviewed disclaimer is well covered in vex-verification.md and the agent docs. One gap: a consumer reading the raw hve-core.openvex.json outside the docs sees no provenance marker. Consider a documented convention (a doc-level field or per-statement status_notes) recording that statements are AI-drafted and human-reviewed.

Broader supply-chain integration (optional)

Integration is strong at the release/attestation layer (SC-9, SBOM pairing, SECURITY.md). VEX isn't yet referenced from the supply-chain-security skill or sssc-planner instructions, though — since VEX is a natural extension of the SBOM/Scorecard/SLSA story, wiring it in there would be a logical follow-up. Worth considering a backlog item to keep track of that.

[katriendg]

Repo convention is to reference a skill by its name (e.g., the powerpoint skill, the telemetry-foundations shared skill), rather than via a #file: walk-up path into the skills tree — see other agents under .github/agents/. Skill paths into skills/ are also more fragile across distributed contexts. The instruction references via relative #file: paths (Lines 35–36) are fine and can stay.

Suggested change

	* #file:../../skills/security/openvex-spec/SKILL.md — OpenVEX v0.2.0 schema, status definitions, justification codes, and document structure.
	* #file:../../skills/security/openvex-spec/references/vex-status-logic.md — decision tree, evidence requirements, confidence bands, and forbidden transitions.
	* Use the skill `openvex-spec` for OpenVEX v0.2.0 schema, status definitions, justification codes, and document structure.
	* Use the file `references/vex-status-logic.md` from the `openvex-spec` skill for decision tree, evidence requirements, confidence bands, and forbidden transitions.

[dasiths]

Done in e935ec9 — the agent now references the openvex-spec skill and its references/vex-status-logic.md by name; the instruction #file: references are left as-is per your note. Now included in this PR via the fork's main.

[katriendg]

Same skill-by-name convention here. The line already names the openvex-spec skill, so just drop the #file: skill path and keep the in-skill reference name. (The instruction #file: path later in the line is fine to leave.)

Suggested change

The authoritative decision tree, per-status evidence requirements, justification codes, confidence-routing bands, and forbidden transitions are defined in the `openvex-spec` skill reference #file:../../../skills/security/openvex-spec/references/vex-status-logic.md. Read it in full during setup and follow it exactly. Agent behavioral constraints are defined in #file:../../../instructions/security/vex-generation.instructions.md. Do not duplicate or paraphrase those tables here; treat the skill reference as the single source of truth.

The authoritative decision tree, per-status evidence requirements, justification codes, confidence-routing bands, and forbidden transitions are defined in the `openvex-spec` skill reference `references/vex-status-logic.md`. Read it in full during setup and follow it exactly. Agent behavioral constraints are defined in #file:../../../instructions/security/vex-generation.instructions.md. Do not duplicate or paraphrase those tables here; treat the skill reference as the single source of truth.

[dasiths]

Done in e935ec9 — dropped the #file: skill path; the line now names the in-skill reference references/vex-status-logic.md. The instruction #file: path later in the line is kept.

[katriendg]

Same fix in the Setup step — reference the openvex-spec skill by name rather than the walk-up path. (Line 45's instruction #file: reference is acceptable as-is.)

Suggested change

	1. Read the canonical skill reference #file:../../../skills/security/openvex-spec/references/vex-status-logic.md end-to-end to load the decision tree, evidence requirements, justification codes, confidence bands, and forbidden transitions.
	1. Read the `openvex-spec` skill reference `references/vex-status-logic.md` end-to-end to load the decision tree, evidence requirements, justification codes, confidence bands, and forbidden transitions.

[dasiths]

Done in e935ec9 — the Setup step now reads the openvex-spec skill reference references/vex-status-logic.md by name. Line 45's instruction #file: reference is unchanged.

[dasiths]

Thanks for the thorough and generous review, @katriendg. I've pushed a round addressing the actionable items; the commits are now in this PR via the fork's main (e935ec9a).

Done in this PR

• Convention nits (skill-by-name): all three inline threads addressed — the VEX generator agent and the CVE analyzer subagent now reference the openvex-spec skill (and its in-skill references/vex-status-logic.md) by name. The instruction #file: references are left as-is, as you noted.
• RAI / embedded provenance: added the OpenVEX document-level tooling field to the published security/vex/hve-core.openvex.json so provenance (AI-drafted, human-reviewed, Sigstore-attested) now travels inside the document itself, not only in the external attestation. The document mutation contract (vex-standards), the generation output requirements (vex-generation), and the schema example were updated to require/illustrate it; the foundation document is bumped to version 2.

Deferred to the HVE Core maintainers (post-merge backlog)

These are yours to own and prioritise, as you suggested:

• Eval loop / labeled CVE corpus as the gate for promotion to stable.
• CODEOWNERS enforcement via branch protection — the CODEOWNERS file already owns /security/vex/ and /.github/; the remaining piece is the branch-protection rule.
• Autonomous-PR identity (GitHub App token for gh-aw's GH_AW_GITHUB_TOKEN) so drafted PRs trigger required checks.

I've intentionally not opened these as issues so the core team can scope them in your backlog. Happy to help implement any of them once filed.

[dasiths]

Proposed post-merge plan and backlog

Consolidating the follow-ups from the review into one place so we can track them after merge. The skill-by-name nits and B4 (embedded provenance) are already done in this PR; everything below is forward-looking.

Post-merge activities (operational setup, one-time)

Activity	Why	Likely owner
Enable branch protection requiring CODEOWNERS review on /security/vex/ (and the VEX .github/ paths)	Makes the human-in-the-loop a required control rather than convention. CODEOWNERS already assigns ownership; this is just the enforcement toggle.	Core team / repo admin
Provision a GitHub App and populate gh-aw's GH_AW_GITHUB_TOKEN	So agent-drafted PRs run as a non-GITHUB_TOKEN identity and therefore trigger required checks. On the org, the Copilot engine uses copilot-requests: write (no PAT needed, unlike the personal-fork validation run).	Core team / repo admin
First live VEX Detection -> VEX Draft run inside HVE Core	Exercise the pipeline on the real repo and learn from the initial runs, as you suggested for experimental.	Maintainers + me
First real release exercises attest-and-upload-vex	The attestation jobs only run on release_created; they are statically validated today and will be runtime-validated on the next stable release.	Release process

Backlog items (tracked work / promotion gates)

Item	Notes	Owner
Eval loop / labeled CVE corpus (B1)	The gate for promoting from experimental to stable. Builds on the repo's new first-class eval support.	Core team (as you proposed)
Promotion criteria	Proposed bar: validated across 3+ codebases with a 5% or lower false-positive rate on not_affected determinations.	Core team
Autonomous-PR identity (B3)	Same as the GitHub App activity above; tracking it as a backlog item if the App provisioning is not immediate.	Core team

Already addressed in this PR

Item	Result
Skill-by-name convention (3 inline threads)	Done (e935ec9a)
Embedded provenance in the published JSON (B4)	Done: OpenVEX tooling field + mutation-contract/generation/schema updates (e935ec9a)

@katriendg — does this capture the right set of post-merge activities and backlog items, with sensible owners and sequencing? If anything is missing or you'd reframe a gate, I'll adjust. I've intentionally not opened these as issues so the core team can file/scope them, but I'm happy to draft any of them on request.

[dasiths]

Heads-up: the one red check is a transient infra flake (no action needed on the diff)

Plugin Validation / Validate Plugins is the only failing check. It failed in the prerequisite module-install step, before any plugin or collection content is validated:

Install-Module -Name PowerShell-Yaml -RequiredVersion 0.4.7 ...
Install-Package: Object reference not set to an instance of an object.
Process completed with exit code 1.

That Install-Package: Object reference not set to an instance of an object is a known intermittent PowerShell Gallery / NuGet provider failure, not a validation error.

It is unrelated to this PR:

• The diff touches no collections/ or plugins/ files.
• npm run plugin:generate on the current head produces zero drift, and the generated plugin copies already reflect the latest VEX content.
• The other 65 checks pass, and this same check passed on the earlier rounds.

Could a maintainer re-run the failed job (run 27766610829)? I don't have Actions permissions on the upstream repo to re-run it myself. Thanks @katriendg!

[rezatnoMsirhC]

Nitpick: the "File or update detection issue" step is gated on count != '0', so when a later run finds zero untriaged vulnerabilities (all CVEs triaged), any previously opened detection issue is never closed. It stays open indefinitely.

A companion step closes the loop:

- name: Close detection issue when all vulnerabilities are triaged
  if: ${{ steps.diff.outputs.count == '0' }}
  env:
    GH_TOKEN: ${{ github.token }}
    REPO_SLUG: ${{ github.repository }}
    ISSUE_TITLE: 'VEX detection: untriaged vulnerabilities found'
  run: |
    existing=$(gh issue list \
      --repo "${REPO_SLUG}" \
      --state open \
      --label automated \
      --search "in:title \"${ISSUE_TITLE}\"" \
      --json number,title \
      --jq "map(select(.title == \"${ISSUE_TITLE}\")) | .[0].number // empty")
    if [[ -n "${existing}" ]]; then
      gh issue comment "${existing}" \
        --repo "${REPO_SLUG}" \
        --body "All previously untriaged vulnerabilities have been resolved or suppressed by the VEX document. Closing."
      gh issue close "${existing}" \
        --repo "${REPO_SLUG}"
    fi

[dasiths]

Done in 1c8a741 — added a companion step gated if: count == '0' that finds the open detection issue (same title + automated search the file step uses), comments, and closes it. The two gates are mutually exclusive and no new permissions are needed (issues: write already declared). Now in this PR via the fork's main.

[rezatnoMsirhC]

The findings dict is keyed by vulnerability ID. When the same CVE affects multiple packages, each loop iteration overwrites the prior entry, so the detection issue table reflects only the last package encountered for that CVE. VEX triage is per-CVE so correctness is unaffected, but maintainers lose visibility into the full blast radius.

Accumulating affected packages instead preserves that detail:

if vid in findings:
    findings[vid]['package'] += f', {name}@{version}'
else:
    findings[vid] = {
        'id': vid,
        'aliases': ', '.join(a for a in aliases if a) or '-',
        'package': f'{name}@{version}',
        'source': source,
        'severity': sev or '-',
        'status': vex_status or 'none',
    }

[dasiths]

Done in 1c8a741 — findings now accumulates all affected packages per CVE (deduped and sorted) instead of overwriting, so the table preserves the full blast radius. Triage stays per-CVE, so VEX status correctness is unchanged. Thanks for the catch!

[dasiths]

@rezatnoMsirhC — thanks for the sharp review. Both of your vex-detect.yml comments are addressed in 1c8a741a (now in this PR via the fork's main):

• Detection issue lifecycle: added a companion step (if: steps.diff.outputs.count == '0') that finds the open dedup detection issue, comments, and closes it once the backlog clears — so it no longer stays open indefinitely. The two gates are mutually exclusive and it reuses the same title + automated search; no new permissions (the job already has issues: write).
• Multi-package CVEs: findings now accumulates all affected packages per CVE (deduped and sorted) instead of overwriting, so the table preserves the full blast radius. Triage remains per-CVE, so VEX status correctness is unchanged.

Replies are on both inline threads. Would you mind taking another look and letting me know if the approach matches what you had in mind? Happy to adjust.

(Unrelated heads-up: the two currently-red checks are not from this change — Plugin Validation is a transient Install-Module flake, and npm Security Audit is a repo-wide cheerio > undici advisory with no dependency change on our side.)

[katriendg]

Thanks for the fast turnaround, @dasiths — re-reviewed against the latest head (1c66d2dc). Everything actionable from the first pass has landed: the three skill-by-name nits (e935ec9a), the embedded JSON provenance via the document-level tooling field (foundation doc bumped to version 2), and @rezatnoMsirhC's two vex-detect.yml fixes (1c8a741a). The broader supply-chain integration (referencing VEX from the supply-chain-security skill / sssc-planner) I'm happy to treat as a genuinely optional, post-merge nicety — no need to hold the PR for it.

I've filed the two deferred follow-ups so they don't get lost:

• feat(evals): establish VEX promotion gate with eval loop and labeled CVE corpus #2132 — VEX promotion gate (eval loop + labeled CVE corpus). Post-merge; this is the machine-checkable gate for experimental → stable.
• ci(security): enforce VEX human-in-the-loop gate via branch protection and autonomous-PR identity #2133 — Enforce the human-in-the-loop gate (CODEOWNERS branch protection + autonomous-PR identity). I'd like this completed before we merge, since the "AI drafts, human merges" trust model is the core of this capability and is currently convention rather than an enforced control. Happy to pair on the branch-protection rule and the GitHub App / GH_AW_GITHUB_TOKEN identity.

Two things still open on my side before approval:

1. Merge conflict — the branch is currently dirty against main; needs a rebase so what we review is what merges.
2. skill_based_on: dasiths/ai_generated_vex — the openvex-spec skill frontmatter still points provenance at a personal repo. Can we re-point it to an owned/upstream source, drop it, or confirm in writing it's licensed to persist? Quick to resolve but I'd like it settled before merge.

@WilliamBerryiii — could you do a pass on this PR and the overall approach, and ack/approve the direction? In particular the draft-and-merge accountability model, the not_affected confidence-gating / forbidden-transition guard, the release-time Sigstore attestation (VEX-as-predicate-over-SBOM), and the experimental-with-eval-gated-promotion posture. It's an important capability with a non-trivial trust surface, so your input on the design before we land it would be really valuable.

[dasiths]

Thanks for the thorough re-review, @katriendg, and for the quick turnaround. Much appreciated. Good news: both pre-approval blockers are sorted, and the branch is current with main.

Blocker A (branch conflict): re-synced the fork's main with upstream main (5537c5b8), resolved the collection/plugin conflicts, and regenerated the plugin and extension outputs from the unioned manifests (lint:collections-metadata green, zero drift). The PR now reports MERGEABLE against the current main, so what you review is exactly what merges.

Blocker B (skill_based_on provenance): good catch. I dropped the metadata.skill_based_on pointer to the personal prototype repo from openvex-spec (c9c517ed). I removed it rather than re-pointing, since content_based_on already records the genuine source (openvex/spec), so there's nothing ambiguous to carry forward. Frontmatter and AI-artifact validation pass.

#2133 (human-in-the-loop enforcement): completely agree, this is the heart of the trust model and worth getting right before we lean on it. The CODEOWNERS file already declares ownership of /security/vex/ and /.github/ in this PR; the remaining pieces (the branch-protection rule requiring CODEOWNERS review, and the GitHub App / GH_AW_GITHUB_TOKEN identity so drafted PRs trigger required checks) live on the microsoft/hve-core admin/infra side. I'd be glad to help get these over the line: I can draft the exact branch-protection ruleset and the App permission set so it's a quick apply on your end. Happy to touch base offline whenever works for you, or we can hash it out on #2133.

#2132 (eval gate): sounds right. Acknowledged as the post-merge experimental to stable gate; nothing needed from me on this PR.

(For context, the earlier red checks were both non-content: a transient Install-Module PowerShell-Yaml PSGallery flake that's been hopping between PowerShell jobs, and a since-cleared cheerio > undici advisory. Local lint:ai-artifacts, lint:collections-metadata, frontmatter, and spell-check all pass on this head.)