Skip to content

fix(ci): harden flaky Cloudflare and queue integration tests#2704

Merged
riderx merged 3 commits into
mainfrom
fix/flaky-ci-tests
Jul 16, 2026
Merged

fix(ci): harden flaky Cloudflare and queue integration tests#2704
riderx merged 3 commits into
mainfrom
fix/flaky-ci-tests

Conversation

@riderx

@riderx riderx commented Jul 16, 2026

Copy link
Copy Markdown
Member

Summary (AI generated)

  • Scope cron plan queue assertions by orgId so parallel cron tests cannot inflate a global queue count (expected 7 to be 6).
  • Lower Cloudflare Workers vitest load (maxWorkers: 3, maxConcurrency: 6) and add one cheap test retry to absorb wrangler-local 503 / Cache API blips without full job restarts.
  • Harden same-channel 60s rate-limit scenario with fresh-device retries; retry concurrent createAppVersions upserts that occasionally return no row.
  • Restore discord_username / github_username on generated users types after schema auto-sync wiped them (unblocks typecheck on main).

Motivation (AI generated)

CI history shows same SHA failing then passing after re-run (often 2–4 attempts), mostly on Cloudflare Workers shards. Full job restarts waste minutes of Supabase/worker boot; root races plus a single in-process retry keep the suite fast and stable.

Business Impact (AI generated)

Faster PR feedback and fewer false-red CI runs that block merges and burn engineer time.

Test Plan (AI generated)

  • Cloudflare Workers shards pass without manual re-run
  • Backend shards pass (queue_cron_stat_org_function, stats createAppVersions path)
  • bun run typecheck passes
  • Rate-limit suite still asserts 429 for same-channel set within 60s

Generated with AI

Made with Cursor

Review in cubic

Same-SHA restarts were masking races: global cron queue counts, wrangler-local
503s under load, Cache API misses on same-channel rate limits, and concurrent
app_versions upserts. Scope queue asserts by org, lower CF workers with one
cheap retry, and retry the known racey helpers/scenarios.

Also restore discord/github username fields wiped from generated types by the
latest schema auto-sync so typecheck matches the merged social-usernames migration.

Co-authored-by: Cursor <cursoragent@cursor.com>
@coderabbitai

coderabbitai Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You’ve reached a temporary PR review limit under our Fair Usage Limits Policy.

Your recent review volume is higher than typical usage, so adaptive limits are currently applied.

Next review available in: 29 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: aefd054e-a82c-4b3a-8bb4-4b23977522f7

📥 Commits

Reviewing files that changed from the base of the PR and between caa8138 and a6f5402.

📒 Files selected for processing (5)
  • src/types/supabase.types.ts
  • supabase/functions/_backend/utils/supabase.types.ts
  • tests/queue_cron_stat_org_function.test.ts
  • tests/test-utils.ts
  • vitest.config.cloudflare.ts

Comment @coderabbitai help to get the list of available commands.

@codspeed-hq

codspeed-hq Bot commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 43 untouched benchmarks
⏩ 2 skipped benchmarks1


Comparing fix/flaky-ci-tests (a6f5402) with main (6e178ee)2

Open in CodSpeed

Footnotes

  1. 2 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

  2. No successful run was found on main (caa8138) during the generation of this report, so 6e178ee was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

@riderx riderx marked this pull request as ready for review July 16, 2026 00:38
@cursor

cursor Bot commented Jul 16, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_06ee7a11-bcb5-47a8-a1cd-0b4a44889f0d)

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale comment

Risk: low. Not approving: Cursor Bugbot completed but could not run (usage limit reached), so automated review signal is incomplete. Human review is needed before merge.

Open in Web View Automation 

Sent by Cursor Approval Agent: Pull Request Approver

@cursor cursor Bot requested review from Dalanir and WcaleNieWolny July 16, 2026 00:38

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale comment

Risk: low. Cursor Bugbot did not complete (check skipped due to usage limit), so I'm not approving. Human reviewers have been assigned.

Open in Web View Automation 

Sent by Cursor Approval Agent: Pull Request Approver External

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 6 files

Confidence score: 5/5

  • Safe to merge after the addressed issues were fixed.

Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic

Comment thread vitest.config.cloudflare.ts Outdated
Keep lower worker concurrency and per-test mitigations, but do not mask
unrelated assertion failures with a global retry.

Co-authored-by: Cursor <cursoragent@cursor.com>
@cursor

cursor Bot commented Jul 16, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_0411a50b-7cdf-4cad-9148-0be35f7939de)

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale comment

Risk: low. Not approving: Cursor Bugbot completed but could not run (usage limit reached), so the automated review signal is incomplete. Human review is still needed before merge; reviewers are already assigned.

Open in Web View Automation 

Sent by Cursor Approval Agent: Pull Request Approver

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale comment

Risk: low. Not approving: Cursor Bugbot completed with status skipped (usage limit reached), so the automated review signal is incomplete. Human reviewers are already assigned.

Open in Web View Automation 

Sent by Cursor Approval Agent: Pull Request Approver External

Drop helper/scenario retry loops; keep org-scoped queue asserts and lower
worker concurrency as the stability fixes.

Co-authored-by: Cursor <cursoragent@cursor.com>
@cursor

cursor Bot commented Jul 16, 2026

Copy link
Copy Markdown

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_a877f2a4-94a3-4b00-8657-564d5485b7b0)

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Risk: low. Not approving: Cursor Bugbot completed with status skipped (usage limit reached), so the automated review signal is incomplete. Human review is still needed before merge; reviewers are already assigned.

Open in Web View Automation 

Sent by Cursor Approval Agent: Pull Request Approver

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Risk: low. Not approving: Cursor Bugbot completed with status skipped (usage limit reached), so the automated review signal is incomplete. Human reviewers are already assigned.

Open in Web View Automation 

Sent by Cursor Approval Agent: Pull Request Approver External

@sonarqubecloud

Copy link
Copy Markdown

@riderx riderx merged commit 34cc5f3 into main Jul 16, 2026
43 checks passed
@riderx riderx deleted the fix/flaky-ci-tests branch July 16, 2026 01:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant