From 93cca3316f64f6f1bd093dfe70ab2be3670607e7 Mon Sep 17 00:00:00 2001 From: Copilot <223556219+Copilot@users.noreply.github.com> Date: Thu, 23 Jul 2026 07:25:14 +0000 Subject: [PATCH 1/2] test: fixme WebSocket Accept-Language locale test on Chrome/Edge MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `browsercontext-locale.spec.ts › should send Accept-Language header on WebSocket handshake` fails 100% on the branded Google Chrome and Microsoft Edge channels while passing on Chromium. Branded builds send the browser's own `Accept-Language` (`en-US,en;q=0.9`) on the WebSocket handshake instead of the emulated `en-GB` locale, so the assertion always fails. Scope a `fixme` to those channels, mirroring the existing Firefox `fixme`. DB evidence (rolling window, expected_status=passed): 100% fail on every branded bot that runs it, still failing through Jul 22 — chrome-ubuntu-22.04 88/88, msedge-windows-latest 88/88, chrome-macos-latest 88/88, chrome-windows-latest 88/88. 0 failures on any chromium-* bot. Error: expect(...).toContain('en-GB'); received 'en-US,en;q=0.9' (x352). Test added Jul 21 in #41904; related issue https://github.com/microsoft/playwright/issues/23732. Emoji run history — msedge-windows-latest (22 runs, oldest first): [🟥](https://github.com/microsoft/playwright/actions/runs/29856729151/attempts/1)[🟥](https://github.com/microsoft/playwright/actions/runs/29859883032/attempts/1)[🟥](https://github.com/microsoft/playwright/actions/runs/29865460136/attempts/1)[🟥](https://github.com/microsoft/playwright/actions/runs/29866867131/attempts/1)[🟥](https://github.com/microsoft/playwright/actions/runs/29867308985/attempts/1)[🟥](https://github.com/microsoft/playwright/actions/runs/29873996100/attempts/1)[🟥](https://github.com/microsoft/playwright/actions/runs/29874748609/attempts/1)[🟥](https://github.com/microsoft/playwright/actions/runs/29874783963/attempts/1)[🟥](https://github.com/microsoft/playwright/actions/runs/29877078341/attempts/1)[🟥](https://github.com/microsoft/playwright/actions/runs/29879172510/attempts/1)[🟥](https://github.com/microsoft/playwright/actions/runs/29904950059/attempts/1)[🟥](https://github.com/microsoft/playwright/actions/runs/29915407358/attempts/1)[🟥](https://github.com/microsoft/playwright/actions/runs/29918102409/attempts/1)[🟥](https://github.com/microsoft/playwright/actions/runs/29924682554/attempts/1)[🟥](https://github.com/microsoft/playwright/actions/runs/29931173316/attempts/1)[🟥](https://github.com/microsoft/playwright/actions/runs/29932910968/attempts/1)[🟥](https://github.com/microsoft/playwright/actions/runs/29933325202/attempts/1)[🟥](https://github.com/microsoft/playwright/actions/runs/29938598979/attempts/1)[🟥](https://github.com/microsoft/playwright/actions/runs/29940740538/attempts/1)[🟥](https://github.com/microsoft/playwright/actions/runs/29945241493/attempts/1)[🟥](https://github.com/microsoft/playwright/actions/runs/29957712363/attempts/1)[🟥](https://github.com/microsoft/playwright/actions/runs/29960743816/attempts/1) Verified locally on windows-latest with msedge installed: - PWTEST_CHANNEL=msedge --project=chromium-library: reproduced the failure (received 'en-US,en;q=0.9'), then confirmed skipped after the fixme. - --project=chromium-library (no channel): still runs and passes. - npm run flint clean (pre-existing unrelated html-reporter check-deps failure reproduces without this change). CI run: https://github.com/microsoft/playwright/actions/runs/29987350696 Suggested-reviewer: yury-s --- tests/library/browsercontext-locale.spec.ts | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tests/library/browsercontext-locale.spec.ts b/tests/library/browsercontext-locale.spec.ts index ef6b65ca7e8c2..d463bd95de567 100644 --- a/tests/library/browsercontext-locale.spec.ts +++ b/tests/library/browsercontext-locale.spec.ts @@ -225,8 +225,9 @@ it('should send user Accept-Language header', { it('should send Accept-Language header on WebSocket handshake', { annotation: [{ type: 'issue', description: 'https://github.com/microsoft/playwright/issues/23732' }], -}, async ({ browser, server, browserName }) => { +}, async ({ browser, server, browserName, channel }) => { it.fixme(browserName === 'firefox', 'Firefox does not send Accept-Language on WebSocket handshake'); + it.fixme(!!channel && (channel.startsWith('chrome') || channel.startsWith('msedge')), 'Google Chrome and Microsoft Edge send the browser Accept-Language instead of the emulated locale on WebSocket handshake, https://github.com/microsoft/playwright/issues/23732'); const context = await browser.newContext({ locale: 'en-GB' }); const page = await context.newPage(); await page.goto(server.EMPTY_PAGE); From 23e46645dc543f8c3f4254f63b8e95e7d8185ef1 Mon Sep 17 00:00:00 2001 From: Simon Knott Date: Thu, 23 Jul 2026 09:43:02 +0200 Subject: [PATCH 2/2] test: scope WebSocket locale fixme to Chromium 150 Stable Chrome and Edge fail on Chromium 150, while Chrome Beta 151 and Edge Dev 152 pass. Gate on the affected version and teach the fix-flakes workflow to compare browser versions before channel names. Copilot-Session: 19d8bd17-13fd-482d-8228-94bd104679cf --- .github/workflows/fix-flakes-prompt.md | 17 +++++++++++++++++ tests/library/browsercontext-locale.spec.ts | 4 ++-- 2 files changed, 19 insertions(+), 2 deletions(-) diff --git a/.github/workflows/fix-flakes-prompt.md b/.github/workflows/fix-flakes-prompt.md index 29acc7016bf0d..03eb4b328cd72 100644 --- a/.github/workflows/fix-flakes-prompt.md +++ b/.github/workflows/fix-flakes-prompt.md @@ -37,6 +37,11 @@ down your step-1 ranking, re-checking each; only stop once the whole shortlist i in that test's file since. Only move on if enough subsequent runs show the failures stopped; a browser roll alone is not evidence of a fix. +Compare the test across every bot that runs it, not only the failing bot. A clean boundary +between stable, beta, and dev browser channels often points to a browser-version regression. +Read the exact versions from the job logs and record the failing and passing versions; channel +names alone do not identify the affected behaviour. + ## 3. Reproduce on this OS Read the test and its `error_message`. @@ -77,6 +82,18 @@ Broader OS coverage comes from different agent runs on other OSes, not from you. it.fixme(browserName === 'webkit' && isLinux, 'https://github.com/microsoft/playwright/issues/NNNNN'); ``` + For browser-specific failures, prefer the narrowest observed version predicate over a channel + predicate. Channels roll forward, so `channel === 'chrome'` can keep skipping the test after + the browser fixes the bug. If stable fails while beta/dev passes, use `browserVersion` or + `browserMajorVersion` and gate only the affected version: + + ```ts + it.fixme(browserName === 'chromium' && browserMajorVersion === 150, 'https://github.com/microsoft/playwright/issues/NNNNN'); + ``` + + Do not invent a wider threshold for untested versions. Keep a channel predicate only when the + same browser version behaves differently between channels. + Link an issue if one exists or explain why the test is skipped. **Default to `fixme`** — it parks the debt and stays greppable. Use `skip` only if reproduction shows the test is genuinely mis-scoped for that config (`skip` claims "this failure is expected and correct," which a diff --git a/tests/library/browsercontext-locale.spec.ts b/tests/library/browsercontext-locale.spec.ts index d463bd95de567..28c17871e97cf 100644 --- a/tests/library/browsercontext-locale.spec.ts +++ b/tests/library/browsercontext-locale.spec.ts @@ -225,9 +225,9 @@ it('should send user Accept-Language header', { it('should send Accept-Language header on WebSocket handshake', { annotation: [{ type: 'issue', description: 'https://github.com/microsoft/playwright/issues/23732' }], -}, async ({ browser, server, browserName, channel }) => { +}, async ({ browser, server, browserName, browserMajorVersion }) => { it.fixme(browserName === 'firefox', 'Firefox does not send Accept-Language on WebSocket handshake'); - it.fixme(!!channel && (channel.startsWith('chrome') || channel.startsWith('msedge')), 'Google Chrome and Microsoft Edge send the browser Accept-Language instead of the emulated locale on WebSocket handshake, https://github.com/microsoft/playwright/issues/23732'); + it.fixme(browserName === 'chromium' && browserMajorVersion === 150, 'Chromium 150 sends the browser Accept-Language instead of the emulated locale on WebSocket handshake, https://github.com/microsoft/playwright/issues/23732'); const context = await browser.newContext({ locale: 'en-GB' }); const page = await context.newPage(); await page.goto(server.EMPTY_PAGE);