[efficiency-improver] Add server-mode (JSON-RPC) performance scenario to nightly perf pipeline

[Evangelink]

Background

The nightly perf-timing-nightly.yml workflow currently measures only plain-process scenarios — it launches the test host as a child process and records wall-clock + CPU time for 100 classes × 100 methods. This leaves the dotnet test server-mode (JSON-RPC) path entirely unmeasured.

In server mode, the test host communicates with the MTP parent via a JSON-RPC protocol over named pipes. This path goes through:

• TestResultMessagesSerializer / DiscoveredTestMessagesSerializer / other IPC serializers
• DotnetTestDataConsumer.UpdateTestNode (PropertyBag walks)
• Named pipe I/O and message framing

Several recent Efficiency Improver PRs targeted exactly this path (PRs #9274, #9300, #9353, #9380, #9408, #9436). Without a server-mode perf scenario, there is no CI-level signal that these improvements actually moved the needle, and no way to detect future regressions.

Proposed Changes

1. Add a server-mode test scenario to MSTest.Performance.Runner

Add Scenario2_ServerMode_PlainProcess in Program.cs that:

• Runs the existing 100×100 test asset via dotnet test (which triggers server mode / JSON-RPC)
• Records the same metrics: wall-clock time and Process.TotalProcessorTime
• Saves results to Result.json for artifact upload

2. Update perf-timing-nightly.yml to include the new scenario

The existing workflow filters on *PlainProcess* — add *ServerMode* (or broaden the filter) so the new results are captured.

3. Add trend tracking (stretch goal)

Currently, nightly results are uploaded as artifacts for 90 days with no comparison or alerting. Consider:

• Adding a step that downloads the previous night's Result.json and computes % change
• Failing the workflow (or posting a comment) when regression exceeds a threshold (e.g., +5% CPU time)
• Storing a rolling baseline in a workflow artifact or repository file

Acceptance Criteria

• A Scenario2_ServerMode_PlainProcess (or equivalent name) runs successfully in the nightly workflow
• Results for both scenarios appear as artifacts for the same run
• (Stretch) A regression detection step compares current vs. previous night and flags regressions

Relevant Files

• test/Performance/MSTest.Performance.Runner/Program.cs — pipeline scenario definitions
• test/Performance/MSTest.Performance.Runner/Steps/PlainProcess.cs — process runner step
• .github/workflows/perf-timing-nightly.yml — nightly CI workflow

🤖 Automated content by GitHub Copilot. Posted via a maintainer's GitHub token, so it appears under their account — the account owner did not write or approve this content personally. Generated by the Efficiency Improver workflow. · 2.3K AIC · ⌖ 24.9 AIC · ⊞ 58.8K · [◷]( · ◷)

Add this agentic workflows to your repo

To install this agentic workflow, run

gh aw add githubnext/agentics/workflows/efficiency-improver.md@main

[Evangelink]

A server-mode (dotnet test / JSON-RPC) scenario for the perf runner is being added in #9486.