The Operation will be canceled. The next steps may not contain expected logs.

[tarekgh]

Build

https://dev.azure.com/dnceng-public/cbb18261-c48f-4abb-8651-8cdcb5474649/_build/results?buildId=696281

Build leg reported

Build / browser-wasm linux Release LibraryTests / Send to Helix

Pull Request

dotnet/runtime#103009

Known issue core information

Fill out the known issue JSON section by following the step by step documentation on how to create a known issue

 {
    "ErrorMessage" : "The Operation will be canceled. The next steps may not contain expected logs.",
    "BuildRetry": false,
    "ErrorPattern": "",
    "ExcludeConsoleLog": false
 }

@dotnet/dnceng

Release Note Category

• Feature changes/additions
• Bug fixes
• Internal Infrastructure Improvements

Release Note Description

Additional information about the issue reported

No response

Known issue validation

Build: 🔎 https://dev.azure.com/dnceng-public/public/_build/results?buildId=696281
Error message validated: [The Operation will be canceled. The next steps may not contain expected logs.]
Result validation: ✅ Known issue matched with the provided build.
Validation performed at: 6/4/2024 8:08:48 PM UTC

Report

Build	Definition	Step Name	Console log	Pull Request
2957307	dotnet-docker-tools	🟪 Clean public/dotnet/* (pruneEol > 15d)	Log
1389684	dotnet/runtime	Send to Helix	Log	dotnet/runtime#127202
1390399	dotnet/runtime	Send to Helix	Log
1389973	dotnet/runtime	Send to Helix	Log	dotnet/runtime#127117
1390433	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#127178
1390421	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#126916
1389986	dotnet/runtime	Send tests to Helix (Unix)	Log
1390414	dotnet/runtime	Send to Helix	Log	dotnet/runtime#122056
1390409	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#125180
1390402	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#127254
1390392	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#127229
1389908	dotnet/runtime	Send to Helix	Log	dotnet/runtime#127200
1389949	dotnet/runtime	Send to Helix	Log	dotnet/runtime#126972
1390344	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#127181
1390330	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#127252
1390362	dotnet/runtime	Send to Helix	Log	dotnet/runtime#127253
1390326	dotnet/runtime	Send to Helix	Log	dotnet/runtime#127182
1390315	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#126904
1390311	dotnet/runtime	Send to Helix	Log	dotnet/runtime#127246
1390268	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#126279
1390256	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#127074
1390272	dotnet/runtime	Send to Helix	Log	dotnet/runtime#127251
1390252	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#126259
1390247	dotnet/runtime	Send to Helix	Log	dotnet/runtime#127199
2957158	dotnet-runtime	Send job to Helix (Unix)	Log
1390015	dotnet/maui	Controls WebView	Log	dotnet/maui#35061
1389981	dotnet/maui	Controls WebView	Log
1390246	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#127250
1390377	dotnet/aspnetcore	Run Quarantined Tests	Log
1390355	dotnet/roslyn	Run Integration Tests	Log
1390241	dotnet/runtime	Send to Helix	Log	dotnet/runtime#127250
2956989	dotnet-maui	Controls WebView	Log
1390235	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#127210
1390223	dotnet/runtime	Send to Helix	Log	dotnet/runtime#127245
1390193	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#126853
2957131	dotnet-roslyn	Build	Log
1389425	dotnet/runtime	Send to Helix	Log	dotnet/runtime#127090
2957154	dotnet-performance	Send job to Helix (Unix)	Log
1390146	dotnet/runtime	Send to Helix	Log	dotnet/runtime#126348
1390103	dotnet/runtime	Send tests to Helix (Unix)	Log
1390138	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#127242
2957129	dotnet-roslyn	Build	Log
1390069	dotnet/runtime	Zip Build Assets	Log	dotnet/runtime#127043
1390119	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#127247
1390104	dotnet/runtime	Send to Helix	Log	dotnet/runtime#127186
1390110	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#127244
1390082	dotnet/runtime	Send to Helix	Log	dotnet/runtime#127213
1390076	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#125559
1389943	dotnet/runtime	Run Native AOT Test App Tests	Log	dotnet/runtime#127194
1390010	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#127241
1388459	dotnet/runtime	Send to Helix	Log	dotnet/runtime#126543
1388030	dotnet/runtime	Send to Helix	Log	dotnet/runtime#127033
1389987	dotnet/runtime	Send to Helix (Unix)	Log
1389963	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#127239
1389941	dotnet/runtime	Send to Helix	Log	dotnet/runtime#127194
2956988	dotnet-maui	Run Integration Tests - RunOniOS_BlazorRelease	Log
1389927	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#127028
1389923	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#127081
1389985	dotnet/aspnetcore	Run Quarantined Tests	Log
1389913	dotnet/runtime	Send to Helix	Log	dotnet/runtime#127238
1389602	dotnet/maui	Controls WebView	Log	dotnet/maui#34600
1389893	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#127195
1389876	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#126520
1389872	dotnet/runtime	Send to Helix (Unix)	Log	dotnet/runtime#127060
2957012	dotnet-dotnet	🛡 Guardian: AntiMalware Scanner (Source)	Log
2956985	dotnet-dotnet	🛡 Guardian: ARMory	Log
1389723	dotnet/runtime	Publish	Log	dotnet/runtime#127188
1389725	dotnet/runtime	Run Native AOT Test App Tests	Log	dotnet/runtime#127188
2956713	dotnet-maui	Component Detection (auto-injected by policy)	Log
1389854	dotnet/runtime	Send to Helix	Log	dotnet/runtime#126408
1389851	dotnet/runtime	Send to Helix	Log	dotnet/runtime#127236
1389844	dotnet/runtime	Send to Helix	Log	dotnet/runtime#127185
1389821	dotnet/runtime	Send to Helix	Log	dotnet/runtime#127073
1389865	dotnet/sdk	🟣 Run TestBuild Tests	Log	dotnet/sdk#53947
1389678	dotnet/runtime	Run Native AOT Test App Tests	Log	dotnet/runtime#127228
1389780	dotnet/runtime	Send to Helix	Log	dotnet/runtime#127233
2956725	dotnet-maui	Controls WebView	Log
1389616	dotnet/runtime	Zip	Log	dotnet/runtime#127229
1389762	dotnet/runtime	Send to Helix	Log	dotnet/runtime#127022
1389758	dotnet/runtime	Send to Helix	Log	dotnet/runtime#127222
1389618	dotnet/runtime	Run Trimming Tests	Log	dotnet/runtime#127229
1389354	dotnet/maui	Controls WebView	Log
2956888	dotnet-runtime	Send job to Helix (Unix)	Log
1389281	dotnet/maui	Controls Layout	Log	dotnet/maui#35056
2956892	dotnet-performance	Send job to Helix (Windows)	Log
1389405	dotnet/runtime	Send to Helix	Log	dotnet/runtime#127058
2956865	dotnet-performance	Send job to Helix (Windows)	Log
1389632	dotnet/sdk	🟣 Run TestBuild Tests	Log
2956842	dotnet-runtime	Send job to Helix (Windows)	Log
1388819	dotnet/runtime	Send tests to Helix (Unix)	Log	dotnet/runtime#126838
1389609	dotnet/runtime	Send to Helix	Log	dotnet/runtime#125397
2956866	dotnet-performance	Send job to Helix (Windows)	Log
1389270	dotnet/maui	Controls WebView	Log
1389538	dotnet/maui	Controls Shell	Log	dotnet/maui#35061
2956843	dotnet-runtime	Send job to Helix (Unix)	Log
1389550	dotnet/runtime	Build product	Log	dotnet/runtime#126385
1389371	dotnet/runtime	Publish	Log	dotnet/runtime#125559
1389391	dotnet/runtime	Set skipComponentGovernanceDetection variable	Log	dotnet/runtime#127148
2956833	dotnet-performance	Send job to Helix (Unix)	Log
1389454	dotnet/runtime	Send tests to Helix (Windows)	Log	dotnet/runtime#127074
Displaying 100 of 2938 results

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
146	722	2938

[dkurepa]

These appear to be timeout cancellations, and most of them are happening on the same runtime leg that uses OSX.1200.ARM64.Open, looking at grafana, they do indeed appear to be quite high. Will check with the team on what we can do about this

[dkurepa]

For pipelines experiencing timeouts on osx.1200.arm64.open we recommend using osx.14.arm64.open, as this queue has a lot more machines

[dkurepa]

There is also a number of Publish Using Darc timeouts, these are in 99% of cases AzDo Api issues. During the Publish Using Darc job, we queue up a Maestro Promotion pipeline, that downloads the artifacts from the build calling it, and publishes them where they need to go. While publishing is happening, the build pipeline keeps busy waiting for the promotion pipeline to be done, checking the status of it every 60 seconds. In cases where this times out, the build most likely did get promoted, the AzDo API just kept telling us the wrong thing

[ivanpovazan]

@dkurepa for context, the same timeouts are also caught with this tracking #1883
where I asked about potentially bumping the queues to osx.13.amd/arm64, would that suffice?
(bumping to osx.14 is not an option right now as we still want testing on osx.13 which is the minimum supported OS version)

[CarnaViire]

For the "known issues" folks: the timeout here is reported twice, making BA red despite the tracking issue. E.g.:

1️⃣ dotnet/runtime#117255

2️⃣ dotnet/runtime#117800

cc @AlitzelMendez