Open
Conversation
Contributor
Semver Impact of This PR⚪ None (no version bump detected) 📋 Changelog PreviewThis is how your changes will appear in the changelog. This PR will not appear in the changelog. 🤖 This preview updates automatically when you update the PR. |
Contributor
Android (legacy) Performance metrics 🚀
|
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| 8295078+dirty | 440.48 ms | 470.32 ms | 29.84 ms |
| 9530cff+dirty | 399.24 ms | 427.22 ms | 27.98 ms |
| 16888c0+dirty | 414.88 ms | 470.83 ms | 55.95 ms |
| f5fb57c+dirty | 405.82 ms | 423.92 ms | 18.10 ms |
App size
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| 8295078+dirty | 43.75 MiB | 48.08 MiB | 4.33 MiB |
| 9530cff+dirty | 43.75 MiB | 48.08 MiB | 4.33 MiB |
| 16888c0+dirty | 43.75 MiB | 48.08 MiB | 4.33 MiB |
| f5fb57c+dirty | 43.75 MiB | 48.08 MiB | 4.33 MiB |
Contributor
iOS (legacy) Performance metrics 🚀
|
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| 16888c0+dirty | 1192.35 ms | 1190.87 ms | -1.48 ms |
| 8295078+dirty | 1220.02 ms | 1221.72 ms | 1.70 ms |
| f5fb57c+dirty | 1195.00 ms | 1190.48 ms | -4.52 ms |
| 9530cff+dirty | 1230.51 ms | 1231.96 ms | 1.45 ms |
App size
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| 16888c0+dirty | 3.38 MiB | 4.73 MiB | 1.35 MiB |
| 8295078+dirty | 3.38 MiB | 4.73 MiB | 1.35 MiB |
| f5fb57c+dirty | 3.38 MiB | 4.73 MiB | 1.35 MiB |
| 9530cff+dirty | 3.38 MiB | 4.73 MiB | 1.35 MiB |
Contributor
iOS (new) Performance metrics 🚀
|
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| 16888c0+dirty | 1204.51 ms | 1211.77 ms | 7.26 ms |
| 8295078+dirty | 1212.93 ms | 1210.57 ms | -2.36 ms |
| f5fb57c+dirty | 1246.91 ms | 1241.61 ms | -5.30 ms |
| 9530cff+dirty | 1220.98 ms | 1216.18 ms | -4.80 ms |
App size
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| 16888c0+dirty | 3.38 MiB | 4.73 MiB | 1.35 MiB |
| 8295078+dirty | 3.38 MiB | 4.73 MiB | 1.35 MiB |
| f5fb57c+dirty | 3.38 MiB | 4.73 MiB | 1.35 MiB |
| 9530cff+dirty | 3.38 MiB | 4.73 MiB | 1.35 MiB |
Contributor
Android (new) Performance metrics 🚀
|
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| 8295078+dirty | 348.48 ms | 390.08 ms | 41.60 ms |
| 9530cff+dirty | 395.80 ms | 426.40 ms | 30.60 ms |
| 16888c0+dirty | 378.71 ms | 386.84 ms | 8.13 ms |
| f5fb57c+dirty | 444.45 ms | 473.36 ms | 28.91 ms |
App size
| Revision | Plain | With Sentry | Diff |
|---|---|---|---|
| 8295078+dirty | 43.94 MiB | 48.94 MiB | 5.00 MiB |
| 9530cff+dirty | 43.94 MiB | 48.94 MiB | 5.00 MiB |
| 16888c0+dirty | 43.94 MiB | 48.94 MiB | 5.00 MiB |
| f5fb57c+dirty | 43.94 MiB | 48.94 MiB | 5.00 MiB |
antonis
force-pushed
the
fix/e2e-stable-checks
branch
from
05eb0bd to
605f13a
Compare
antonis
changed the title
antonis
force-pushed
the
fix/e2e-stable-checks
branch
3 times, most recently
from
8f483e2 to
2ff292c
Compare
5 tasks
antonis
force-pushed
the
fix/e2e-stable-checks
branch
4 times, most recently
from
cdfefc9 to
af0c664
Compare
Replace retry-based approach (PR #5830) with deterministic fixes: ### Simulator stability (Cirrus Labs Tart VMs) - `wait_for_boot: true` / `erase_before_boot: false` on simulator-action - `xcrun simctl bootstatus booted -b` to block until boot completes - Settings.app warm-up for SpringBoard/system service initialization - `MAESTRO_DRIVER_STARTUP_TIMEOUT` bumped to 180s ### e2e-v2 test runner (cli.mjs) - Per-flow process isolation via individual `maestro test` calls - Maestro driver warm-up flow before real tests (non-fatal) - crash.yml runs first so the next flow verifies post-crash recovery - `execSync` → `execFileSync` to avoid shell interpolation - SENTRY_AUTH_TOKEN redaction in debug logs ### Sample application test fixes - Search all envelopes for app start transaction (slow VM delivery) - Sort envelopes by timestamp for deterministic ordering - Allow-list for TTID/TTFD ops (`navigation`, `ui.load`) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
antonis
force-pushed
the
fix/e2e-stable-checks
branch
from
af0c664 to
f509434
Compare
![sentry[bot]](https://avatars.githubusercontent.com/in/12637?s=60&v=4){kind=small}
antonis
commented
Mar 30, 2026
Contributor
Author
antonis
left a comment
antonis
left a comment
There was a problem hiding this comment.
@lucas-zimerman I think we can also close #4913 if this solution works
antonis
changed the title
With stdio: 'pipe', failed flow output was swallowed. Now the full Maestro stdout/stderr is printed when a flow fails, making it easier to diagnose flaky tests in CI logs. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Collaborator
|
Other than CI AI comment, the code looks good! |
The iOS simulator readiness and Maestro warm-up steps were incorrectly placed in the test-android job from a merge conflict resolution. They had conditions referencing matrix.platform which doesn't exist in that job, so they never ran. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
📢 Type of change
📜 Description
Fixes E2E test flakiness without retries. Alternative to #5830.
launchAppon Cirrus Labs Tart VMssimctl bootstatus+ Settings.app warm-up + 180s driver timeoutcaptureReplayfix — test app'sbeforeSendwas replacing the replay integration's; now chains correctlyextendedWaitUntilbefore tapping "Report a Bug"execSync→execFileSyncto avoid shell interpolation#skip-changelog💡 Motivation and Context
E2E tests consistently fail on
main— crash cascade, Maestro driver flakiness, broken replaybeforeSendchain, and Android feedback timing.Fixes #5926
💚 How did you test it?
CI
📝 Checklist
sendDefaultPIIis enabled🔮 Next steps
#skip-changelog