Skip to content

feat(expect): add expect.fn() mock functions with awaited assertions#41884

Merged
pavelfeldman merged 1 commit into
microsoft:mainfrom
pavelfeldman:expect-fn
Jul 21, 2026
Merged

feat(expect): add expect.fn() mock functions with awaited assertions#41884
pavelfeldman merged 1 commit into
microsoft:mainfrom
pavelfeldman:expect-fn

Conversation

@pavelfeldman

@pavelfeldman pavelfeldman commented Jul 20, 2026

Copy link
Copy Markdown
Member

Summary

  • expect.fn(implementation?) creates a mock function that records mock.calls / mock.results / mock.settledResults / mock.lastCall, with mockClear/mockReset (vitest semantics), mockImplementation[Once], mockReturnValue[Once], mockResolvedValue[Once], mockRejectedValue[Once], mockName.
  • 15 awaited mock assertions (toHaveBeenCalled*, toHaveReturned*, toHaveResolved*) that retry until they pass or the expect timeout is reached, with fail-fast when the outcome can no longer change.
  • Implementations are async-only: they run in the test process, so a page-side caller always receives a promise. mockReturnValue() values are serialized along with the function (new fn_rv node in the wire format) and consumed synchronously by the page-side callback stub, while every call is still routed back for recording.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds an expect.fn() mock function facility to Playwright Test, including retrying/awaited mock assertions, plus protocol/serialization support so page-side callback stubs can synchronously consume configured return values while still routing calls back for recording.

Changes:

  • Introduce expect.fn() and 15 retrying mock matchers (toHaveBeenCalled*, toHaveReturned*, toHaveResolved*) with fail-fast when outcomes become terminal.
  • Extend the serialized value wire format with fn_rv to carry mock return values alongside callback bindings.
  • Add coverage tests for in-process mocks and page-exposed callback mocks.

Reviewed changes

Copilot reviewed 13 out of 16 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
utils/generate_types/overrides-test.d.ts Adds public type surface for MockFunction, mock matchers, and expect.fn().
utils/generate_types/index.js Ensures MockFunction is included in generated type outputs.
tests/playwright-test/expect-fn.spec.ts New test suite validating mock call/result tracking and retrying assertions.
tests/page/page-evaluate-callback.spec.ts Adds end-to-end tests for exposing expect.fn() into the page and recording calls/results.
packages/protocol/src/validator.ts Accepts new fn_rv shape in protocol validation.
packages/protocol/src/structs.d.ts Extends SerializedValue typing with fn_rv.
packages/protocol/src/serializers.ts Deserializes fn_rv and attaches callback return values to the dummy function.
packages/protocol/spec/serialized.yml Documents the fn_rv field in the wire format spec.
packages/playwright/types/test.d.ts Updates generated public types to include mocks and matchers.
packages/playwright/src/matchers/mockFunctions.ts Implements mock function state tracking and retrying mock matchers.
packages/playwright/src/matchers/matchers.ts Extracts reusable timeout failure message formatting.
packages/playwright/src/matchers/expectLibrary.ts Re-exports equals for reuse in mock matcher comparisons.
packages/playwright/src/matchers/expect.ts Wires expect.fn() and mock matchers into the expect implementation and expect.poll restrictions.
packages/playwright-core/src/client/jsHandle.ts Serializes callback return values (fn_rv) when passing functions into the page.
packages/isomorphic/utilityScriptSerializers.ts Parses/serializes fn_rv in the browser-side serializer and defines callback-return-value helpers/constants.
packages/isomorphic/index.ts Re-exports utility script serializers from the isomorphic index.

Comment thread packages/isomorphic/utilityScriptSerializers.ts
Comment thread packages/isomorphic/utilityScriptSerializers.ts Outdated
Comment thread packages/protocol/src/serializers.ts Outdated
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@pavelfeldman
pavelfeldman force-pushed the expect-fn branch 2 times, most recently from 11c9a60 to 63f1481 Compare July 21, 2026 00:19
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

expect.fn() creates a mock function that records calls, results and
settled results. Mock assertions like toHaveBeenCalledWith() and
toHaveResolvedWith() are asynchronous and retried until they pass or
the expect timeout is reached, so mocks can be handed to concurrent
code, e.g. exposed into the page via page.evaluate(). Assertions whose
outcome can no longer change fail fast instead of waiting.

Implementations are async-only because they run in the test process.
Values set with mockReturnValue() are serialized along with the
function and consumed synchronously by the page-side callback stub.
@github-actions

Copy link
Copy Markdown
Contributor

Test results for "MCP"

3 failed
❌ [chrome] › mcp/sse.spec.ts:88 › sse transport browser lifecycle (isolated) @mcp-macos-latest-chrome
❌ [chromium] › mcp/annotate.spec.ts:57 › should capture multiple screenshots in one annotation @mcp-macos-latest-chromium
❌ [webkit] › mcp/cli-killall.spec.ts:42 › kill-all kills filtered dashboard pid @mcp-macos-latest-webkit

7757 passed, 1249 skipped


Merge workflow run.

@github-actions

Copy link
Copy Markdown
Contributor

Test results for "tests 1"

1 failed
❌ [webkit-page] › page/page-leaks.spec.ts:164 › waitFor should not leak @webkit-ubuntu-22.04-node20

4 flaky ⚠️ [chromium-library] › library/inspector/recorder-api.spec.ts:120 › should type `@chromium-ubuntu-22.04-arm-node20`
⚠️ [chromium-library] › library/beforeunload.spec.ts:130 › should support dismissing the dialog multiple times `@realtime-time-library-chromium-linux`
⚠️ [chromium-library] › library/video.spec.ts:495 › screencast › should capture static page in persistent context @smoke `@chromium-ubuntu-22.04-node24`
⚠️ [firefox-page] › page/page-emulate-media.spec.ts:144 › should keep reduced motion and color emulation after reload `@firefox-ubuntu-22.04-node20`

50191 passed, 1189 skipped


Merge workflow run.

@github-actions

Copy link
Copy Markdown
Contributor

Hi, I'm the Playwright bot and I took a first look at the CI failures here.

🔴 One failure looks caused by this PR

page/page-evaluate-callback.spec.ts › should record calls to a mock function created with expect.fn() fails deterministically on the frozen-clock chromium bot. The MCP failures in the latest report are pre-existing flakes or unproven.

Details

This PR adds expect.fn() and its tests — it touches packages/playwright/src/matchers/*, the serializers, client/jsHandle.ts, and types/test.d.ts. Nothing in it reaches the MCP tool paths.

Caused by this PR

  • [chromium-page] › page/page-evaluate-callback.spec.ts › should record calls to a mock function created with expect.fn() (@frozen-time-library-chromium-linux) — a test this PR introduces, so there's no history where the PR isn't responsible. In the test-results DB it fails only on the frozen-clock bot (3 of 4 runs) and passes on all 7 other botsrealtime-time-library-chromium-linux and every other chromium/firefox/webkit runner (0 failures each). The frozen harness installs a paused clock (context.clock.install({ time: 0 }) + pauseAt(...)), so a page-side timer driving the delayed "later" call never fires and the awaited toHaveBeenCalledWith("later") times out. Real and reproducible, not a flake. Fix: don't drive the delayed call from a live page timer under a frozen clock — advance the installed clock, or skip that portion when the clock is frozen.

Pre-existing flake / infra

The latest merged MCP report lists three failures; the two below flip verdict across many unrelated PRs in the DB, and expect.fn() doesn't exercise the MCP annotate/kill-all flows:

  • [chromium] › mcp/annotate.spec.ts:57 › should capture multiple screenshots in one annotation (@mcp-macos-latest-chromium) — failed 9 of 1380 runs (~1%) across 6 distinct PRs. Cross-PR flake.
  • [webkit] › mcp/cli-killall.spec.ts:42 › kill-all kills filtered dashboard pid (@mcp-macos-latest-webkit) — failed 17 of 1380 runs (~1%) across 9 distinct PRs. Cross-PR flake.

Uncertain

  • [chrome] › mcp/sse.spec.ts:88 › sse transport browser lifecycle (isolated) (@mcp-macos-latest-chrome) — I can't prove this either way. It has passed 1380 of 1380 prior runs in the DB; the single timedOut is on this PR itself, so there's no run where the PR can't be responsible. The diff has no plausible link to MCP sse transport lifecycle on macOS, which leans flake, but with zero history elsewhere I'm leaving it uncertain. A rerun should settle it.

Triaged by the Playwright bot - agent run

@github-actions

Copy link
Copy Markdown
Contributor

Hi, I'm the Playwright bot and I took another look at CI after the latest push.

🟡 No failure I can pin on this PR, but one is unproven

The expect.fn() frozen-clock failure from the earlier runs is gone on the new commit (1d3d5f7). Of what's left, three are confirmed pre-existing flakes and one MCP transport test I can't prove either way.

Details

The PR adds expect.fn() and its matchers — it touches packages/playwright/src/matchers/*, the serializers, client/jsHandle.ts, and types/test.d.ts. Nothing in it reaches the MCP server, sse transport, browser-process lifecycle, or webkit leak detection.

Fixed since the last report

  • [chromium-page] › page/page-evaluate-callback.spec.ts:215 › should record calls to a mock function created with expect.fn() (@frozen-time-library-chromium-linux) — deterministic in the 01:04 report, no longer present in the 02:59 rerun on 1d3d5f7. The force-push at 02:16 appears to have fixed it.

Pre-existing flake / infra

  • [chromium] › mcp/annotate.spec.ts:57 › should capture multiple screenshots in one annotation (@mcp-macos-latest-chromium) — cross-PR flake. In the results DB it fails on chrome/chromium/firefox/webkit across unrelated PRs (41463, 41692, 41704, 41801) and on main pushes.
  • [webkit] › mcp/cli-killall.spec.ts:42 › kill-all kills filtered dashboard pid (@mcp-macos-latest-webkit) — cross-PR flake. Fails on every browser across many unrelated PRs (41687, 41719, 41728, 41806, 41851, 41882, …) and on main. webkit alone: PRs 41687, 41728, 41882.
  • [webkit-page] › page/page-leaks.spec.ts:164 › waitFor should not leak (@webkit-ubuntu-22.04-node20) — pre-existing flake. Now proven: the same test also failed on unrelated PR 41889 (migrate-dogfood-ct), which this PR can't be responsible for. A locator waitFor heap-leak count has no link to the expect.fn() diff.

Uncertain

  • [chrome] › mcp/sse.spec.ts:88 › sse transport browser lifecycle (isolated) (@mcp-macos-latest-chrome) — I can't prove this one. It has never failed anywhere in the results DB (0 of 326 chrome runs, 0 across all browsers), so it isn't a known flake — but the diff also has no plausible path to an MCP sse-transport/browser-lifecycle test, so it isn't a believable regression either. It's an isolated macOS process-lifecycle test, a class that's timing-sensitive on infra, which leans flake. A rerun should settle it.

Triaged by the Playwright bot - agent run

@pavelfeldman
pavelfeldman merged commit ecf2e53 into microsoft:main Jul 21, 2026
44 of 48 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants