diff --git a/.github/workflows/fix-flakes-prompt.md b/.github/workflows/fix-flakes-prompt.md index 29acc7016bf0d..03eb4b328cd72 100644 --- a/.github/workflows/fix-flakes-prompt.md +++ b/.github/workflows/fix-flakes-prompt.md @@ -37,6 +37,11 @@ down your step-1 ranking, re-checking each; only stop once the whole shortlist i in that test's file since. Only move on if enough subsequent runs show the failures stopped; a browser roll alone is not evidence of a fix. +Compare the test across every bot that runs it, not only the failing bot. A clean boundary +between stable, beta, and dev browser channels often points to a browser-version regression. +Read the exact versions from the job logs and record the failing and passing versions; channel +names alone do not identify the affected behaviour. + ## 3. Reproduce on this OS Read the test and its `error_message`. @@ -77,6 +82,18 @@ Broader OS coverage comes from different agent runs on other OSes, not from you. it.fixme(browserName === 'webkit' && isLinux, 'https://github.com/microsoft/playwright/issues/NNNNN'); ``` + For browser-specific failures, prefer the narrowest observed version predicate over a channel + predicate. Channels roll forward, so `channel === 'chrome'` can keep skipping the test after + the browser fixes the bug. If stable fails while beta/dev passes, use `browserVersion` or + `browserMajorVersion` and gate only the affected version: + + ```ts + it.fixme(browserName === 'chromium' && browserMajorVersion === 150, 'https://github.com/microsoft/playwright/issues/NNNNN'); + ``` + + Do not invent a wider threshold for untested versions. Keep a channel predicate only when the + same browser version behaves differently between channels. + Link an issue if one exists or explain why the test is skipped. **Default to `fixme`** — it parks the debt and stays greppable. Use `skip` only if reproduction shows the test is genuinely mis-scoped for that config (`skip` claims "this failure is expected and correct," which a diff --git a/tests/library/browsercontext-locale.spec.ts b/tests/library/browsercontext-locale.spec.ts index ef6b65ca7e8c2..28c17871e97cf 100644 --- a/tests/library/browsercontext-locale.spec.ts +++ b/tests/library/browsercontext-locale.spec.ts @@ -225,8 +225,9 @@ it('should send user Accept-Language header', { it('should send Accept-Language header on WebSocket handshake', { annotation: [{ type: 'issue', description: 'https://github.com/microsoft/playwright/issues/23732' }], -}, async ({ browser, server, browserName }) => { +}, async ({ browser, server, browserName, browserMajorVersion }) => { it.fixme(browserName === 'firefox', 'Firefox does not send Accept-Language on WebSocket handshake'); + it.fixme(browserName === 'chromium' && browserMajorVersion === 150, 'Chromium 150 sends the browser Accept-Language instead of the emulated locale on WebSocket handshake, https://github.com/microsoft/playwright/issues/23732'); const context = await browser.newContext({ locale: 'en-GB' }); const page = await context.newPage(); await page.goto(server.EMPTY_PAGE);