Skip to content

chore: Reduce CI execution time by changing some tests to use InProcessToolchain#3126

Open
filzrev wants to merge 1 commit into
dotnet:masterfrom
filzrev:chore-reduce-ci-execution-time
Open

chore: Reduce CI execution time by changing some tests to use InProcessToolchain#3126
filzrev wants to merge 1 commit into
dotnet:masterfrom
filzrev:chore-reduce-ci-execution-time

Conversation

@filzrev
Copy link
Copy Markdown
Contributor

@filzrev filzrev commented May 12, 2026

This PR intended to reduce CI execution time by using InProcessToolchain when possible.

What's changed in this PR

1. Add shared configs to TestConfigs.cs
Add following test configs that used for faster test execution.

  • SingleRunInProcessConfig
  • SingleRunOutOfProcessConfig

2. Add Helpers/CaptureConsoleHelper.cs
This helper class is used to capture console logs that are outputted by benchmark.

It's required because StandardOutput is not recorded to Summary/Report instance. when running benchmark with InProcessToolchain.

As a side effect, captured log contains benchmark execution logs also.
Though, It's not affects test results when assert contents line by line.

3. Modify some test code to use InProcessToolchain

Modify following tests to use SingleRunInProcessConfig when possible

  • AsyncBenchmarksTests
  • AttributesTests
  • BaselineRatioColumnTest
  • BenchmarkSwitcherTest (Partially)
  • ParamsTests

4. Modify some test code to use SingleRunOutOfProcessConfig
Some BenchmarkSwitcherTest tests can't convert to use InProcessToolchain by following reason.

  • Test that using [DryJob] attribute.
  • Test that pass --job command line argument.
  • Test that using --resume option (It seems not works because InProcessToolchain don't output logs for resume)

So these test need to use SingleRunOutOfProcessConfig instead. (It's basically same behavior as original code)

Comment thread tests/BenchmarkDotNet.IntegrationTests/BenchmarkSwitcherTest.cs
@filzrev filzrev marked this pull request as ready for review May 12, 2026 21:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant