Skip to content

test: extend cross-collection maturity-conflict coverage#2219

Open
PratikWayase wants to merge 2 commits into
microsoft:mainfrom
PratikWayase:test/extend-cross-collection-maturity-conflict-coverage
Open

test: extend cross-collection maturity-conflict coverage#2219
PratikWayase wants to merge 2 commits into
microsoft:mainfrom
PratikWayase:test/extend-cross-collection-maturity-conflict-coverage

Conversation

@PratikWayase

Copy link
Copy Markdown
Contributor

Description

Follow-up bundle from the owasp-docker removal / maturity: removed refactor. This PR extends the validation logic and test coverage to prevent silent maturity drift as more themed collections adopt the maturity: removed marker at the collection YAML level.

Test Coverage (Validate-Collections.Tests.ps1):

  • Added scenario for maturity conflicts across N>2 themed collections.
  • Added specific pair conflicts (removed vs stable, removed vs deprecated, deprecated vs experimental) with clear error message assertions.
  • Added regression guard to confirm the canonical hve-core-all collection is explicitly excluded from the new collection-level conflict detection.

Implementation (Validate-Collections.ps1):

  • Added logic to fall back to collection-level maturity when item-level maturity is not explicitly defined.
  • Added cross-themed collection maturity conflict detection.
  • Safeguarded existing canonical vs. themed conflict check to only trigger on explicitly defined item maturities, allowing themed collections to tombstone items without conflicting with the aggregate collection.

Related Issue(s)

Closes #1446

Type of Change

Select all that apply:

Code & Documentation:

  • Bug fix (non-breaking change fixing an issue)
  • New feature (non-breaking change adding functionality)
  • Breaking change (fix or feature causing existing functionality to change)
  • Documentation update

Infrastructure & Configuration:

  • GitHub Actions workflow
  • Linting configuration (markdown, PowerShell, etc.)
  • Security configuration
  • DevContainer configuration
  • Dependency update

AI Artifacts:

  • Reviewed contribution with prompt-builder agent and addressed all feedback
  • Copilot instructions (.github/instructions/*.instructions.md)
  • Copilot prompt (.github/prompts/*.prompt.md)
  • Copilot agent (.github/agents/*.agent.md)
  • Copilot skill (.github/skills/*/SKILL.md)
  • Copilot hook (.github/hooks/*/*.json)
  • Eval spec added/updated for changed AI artifacts (evals/)

Other:

  • Script/automation (.ps1, .sh, .py)
  • Other (please describe):

Testing

Ran the Pester test suite locally using Invoke-Pester -Path "scripts/tests/collections/Validate-Collections.Tests.ps1". All 63 tests pass successfully, including:

  • The 5 new cross-collection maturity conflict tests (N>2 collections, specific maturity pairs, and the hve-core-all exclusion regression guard).
  • The pre-existing item-level maturity conflict test to ensure backwards compatibility.

Checklist

Required Checks

  • Documentation is updated (if applicable)
  • Files follow existing naming conventions
  • Changes are backwards compatible (if applicable)
  • Tests added for new functionality (if applicable)

AI Artifact Contributions

  • Used /prompt-analyze to review contribution
  • Addressed all feedback from prompt-builder review
  • Verified contribution follows common standards and type-specific requirements

Required Automated Checks

The following validation commands must pass before merging:

  • Markdown linting: npm run lint:md
  • Spell checking: npm run spell-check
  • Frontmatter validation: npm run lint:frontmatter
  • Skill structure validation: npm run validate:skills
  • Link validation: npm run lint:md-links
  • PowerShell analysis: npm run lint:ps
  • Eval spec schema and coverage (if AI artifacts changed): npm run eval:lint:schema
  • Plugin freshness: npm run plugin:generate
  • Docusaurus tests: npm run docs:test

Security Considerations

  • This PR does not contain any sensitive or NDA information
  • Any new dependencies have been reviewed for security issues
  • Security-related scripts follow the principle of least privilege

Additional Notes

None.

@PratikWayase PratikWayase requested a review from a team as a code owner June 28, 2026 10:43
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 81.38%. Comparing base (3d4dbad) to head (bbdb314).
⚠️ Report is 6 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #2219      +/-   ##
==========================================
+ Coverage   81.29%   81.38%   +0.09%     
==========================================
  Files         129      119      -10     
  Lines       19068    19016      -52     
  Branches       12        0      -12     
==========================================
- Hits        15501    15477      -24     
+ Misses       3564     3539      -25     
+ Partials        3        0       -3     
Flag Coverage Δ
docusaurus ?
pester 86.03% <100.00%> (+0.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
scripts/collections/Validate-Collections.ps1 94.48% <100.00%> (+0.57%) ⬆️

... and 11 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test: extend cross-collection maturity-conflict coverage in Validate-Collections.Tests.ps1

3 participants