Repository Quality: MSTest Analyzer `Category.Performance` Deficit and MSTEST0067 Calibration

[Evangelink]

🎯 Repository Quality Improvement — MSTest Analyzer Performance Category Deficit

Analysis Date: 2026-06-26 Focus Area: analyzer-performance-category-deficit (Custom)

Executive Summary

Only 1 of 68 MSTest analyzer rules (1.5%) uses Category.Performance, despite multiple rules that directly impact test-suite execution performance. The lone Performance rule is MSTEST0001 (UseParallelizeAttribute). Two rules that cause measurable test-suite slowness are miscategorized, and the most comprehensive blocking-call rule is disabled by default.

Category distribution today:

• Category.Performance: 1 rule (1.5%)
• Category.Design: 14 rules (20.6%)
• Category.Usage: 52 rules (76.5%)

Misclassified rules:

Rule	Current Category	Correct Category	Severity	Enabled
MSTEST0067 AvoidThreadSleepAndTaskWaitInTests	Usage	Performance	Info	false
MSTEST0045 UseCooperativeCancellationForTimeout	Design	Performance	Info	true

MSTEST0067 covers Thread.Sleep, Task.Wait, Task.WaitAll, Task.WaitAny, and Task<T>.Result across all 8 MSTest lifecycle attributes plus test methods — the framework's most comprehensive lifecycle-aware rule — yet teams receive no signal unless they explicitly opt in. No tests enforce category correctness, so misclassifications go undetected.

---

Suggested Improvement Tasks

Task 1: Reclassify MSTEST0067 to Category.Performance

Priority: High | Effort: Small

In src/Analyzers/MSTest.Analyzers/AvoidThreadSleepAndTaskWaitInTestsAnalyzer.cs:

// Before
Category.Usage,
// After
Category.Performance,

MSTEST0067 is disabled by default, so this is non-breaking. Teams with category-performance.severity = warning will immediately benefit.

---

Task 2: Enable MSTEST0067 by default

Priority: High | Effort: Small

// Before
isEnabledByDefault: false);
// After
isEnabledByDefault: true);

Thread.Sleep/Task.Wait inside test lifecycle methods is almost universally replaceable with await Task.Delay(...)/await. The rule fires exclusively inside MSTest-annotated methods, keeping false-positive rates very low. Enabling at Info severity also auto-promotes it to Warning in Recommended mode via the existing ruleset-mode matrix (DecideForModeRecommended escalates enabled-by-default Info → Warning).

---

Task 3: Reclassify MSTEST0045 to Category.Performance

Priority: Medium | Effort: Small

// Before — UseCooperativeCancellationForTimeoutAnalyzer.cs
Category.Design,
// After
Category.Performance,

Cooperative cancellation lets timed-out tests observe CancellationToken and release thread-pool threads promptly rather than running to completion. This is a resource/performance concern, not a design preference.

⚠️ Note: MSTEST0045 is enabled by default. Teams with category-Design.severity = none would start seeing this rule. Target for next minor/major release with a changelog entry.

---

Task 4: Add category validation tests

Priority: Medium | Effort: Small

Add AnalyzerCategoryGovernanceTests.cs to test/UnitTests/MSTest.Analyzers.UnitTests/ with lightweight property-access tests verifying that MSTEST0001, MSTEST0045, and MSTEST0067 use "Performance" as their DiagnosticDescriptor.Category. These tests compile with no Roslyn workspace overhead and act as a living specification of the taxonomy.

---

Task 5: Document Category.Performance decision criteria in Category.cs

Priority: Low | Effort: Small

Expand the XML doc comments in src/Analyzers/MSTest.Analyzers/Helpers/Category.cs to explain when each category should be used (e.g., "Performance: blocking calls, missing parallelism, non-cooperative timeouts"; "Usage: API misuse, incorrect signatures"; "Design: structural/organizational patterns"). This prevents future contributors from defaulting new performance-anti-pattern rules to Usage.

---

Historical Context

Previous runs (19 total, 89.5% custom): treenode-filter-completeness, itestdatasource-extensibility-quality, testconfig-json-schema-and-discoverability, publicapi-versioning-hygiene, legacy-assertion-structured-message-gap, analyzer-code-fix-coverage-gap, localization-pipeline-health, target-framework-support-matrix.

Next analysis: 2026-06-27

🤖 Automated content by GitHub Copilot. Posted via a maintainer's GitHub token, so it appears under their account — the account owner did not write or approve this content personally. Generated by the Repository Quality Improver workflow. · 752.2 AIC · ⌖ 25.9 AIC · ⊞ 45.8K · [◷]( · ◷)

Add this agentic workflows to your repo

To install this agentic workflow, run

gh aw add githubnext/agentics/workflows/repository-quality-improver.md@main

• expires on Jun 28, 2026, 10:50 PM UTC

[Evangelink]

Partially addressed in #9487: MSTEST0067 reclassified to Category.Performance (non-breaking, disabled by default), Category.cs taxonomy documented, and governance tests added (Tasks 1, 4, 5). Tasks 2 (enable MSTEST0067 by default) and 3 (reclassify MSTEST0045) change behavior for already-enabled rules and are intentionally deferred for a maintainer decision / changelog, as the issue itself notes for MSTEST0045.

[github-actions]

This issue was automatically closed because it expired on 2026-06-28T22:50:57.626Z.

Closed by Workflow