Repository navigation
chore(dev): sync dev with staging - #9332
Open
waleedlatif1 wants to merge 860 commits into
Open
waleedlatif1 wants to merge 860 commits into
waleedlatif1 wants to merge 860 commits into
Conversation
) ## Summary - Add a daily `ci-flakes` workflow (`scripts/report-ci-flakes.ts`): a job that failed on a CI attempt and passed on a re-run of the same commit files one `ci-flake` issue per failing test (Vitest `FAIL` lines and the E2E harnesses' `[Suite] FAIL` lines), or per job when its log names none. Occurrences are keyed by job id, so the overlapping 3-day lookback never double-counts, and a closed issue reopens when its flake recurs. Dependency-free, so the job needs no install - Fix the live flakes found in the last two weeks of re-runs: - GitHub rate-limit reset test: pin the clock; a second boundary between building the header and reading it gave 899000 - OAuth token and token-family integration tests: import the auth graph statically, so its cold load (up to ~43s on a slow runner) no longer counts against the first test's 30s timeout - stop-after E2E: wait for the sibling-pause claim until the resume request settles, not a fixed 2s that a slow `next dev` round trip outlasts - HTTP E2E readiness: 300s → 600s; healthy cold compiles take 60–185s and a degraded runner ran past 300s with every step 3–6x slower - Reconciliation-window integration test: print each scan's plan when a window statement reads past the bound, so its next failure names the cause (diagnostics only; the cause isn't confirmed yet) - Mobile e2e and usage-log flakes were already fixed by later commits, so nothing changes there - Testing rules: a short section on the flake workflow and how to fix one (no retries, longer timeouts, or skips)
…e file names (#9044) * improvement(naming): rename sandbox-tasks to sandboxes and drop folder-name stutter in execution files * improvement(naming): standardize file names to the folder and role conventions Renames every check:file-names baseline entry (stutter and redundant-suffix) so the baseline is empty, plus the same rule's uncaught cases: grandparent-domain prefixes in application/ and orchestration/ folders, loose stores moved to stores/<feature>/store.ts, non-hook files dropped from the use-* prefix, and application context/errors outliers. * improvement(naming): resolve rename ambiguities and stale file references from review
* fix(docs): preserve negotiated response cache headers * fix(docs): retain native header regression checks
…ces (#9039) * improvement(docs): standardize integration copy and generated references * fix(docs): clarify supported integration operations and trigger events * fix(docs): preserve integration search details and event limits * fix(docs): distinguish table query and page budgets * fix(docs): preserve explicit search heading links
* refactor(files): unify mutation accounting and retention * feat(project-files): add core storage and realtime operations * feat(files): unify document read subscriptions and write capabilities * fix(files): resolve capability type and artifact checks * fix(docs): hide unreleased Project file references * fix(files): address review admission and collaboration gaps * test(cli): load canonical documentation in suite setup * fix(files): bound seed reads and preserve browser snapshots * fix(files): keep read provenance private unless requested * fix(files): classify raw Project downloads as opaque
…run, not a 500 (#9047) A synchronous resume whose resumed run failed in-band (a block error) let the engine's error escape executeResumeWorkflow, and the v2 resume route mapped it to INTERNAL_ERROR. The sync branch now settles an error that carries the run's partial result as a failed sync result classified like a sync execute; errors with no run behind them still throw. The legacy resume route keeps its 400 error body for a failed run.
…9045) * fix(knowledge): serialize chunk date tags on the internal chunk read * test(knowledge): parse the presented chunk body directly
…network origin (#9049) * fix(uploads): sign transfers for in-process v2 callers that carry no network origin * test(uploads): give the MCP fixture request a client address
…adable usage ledger (#9046) * fix(resume): retry automatic resumes after read deadlines and an unreadable usage ledger * fix(resume): keep the stack on the usage-status failure log * fix(resume): log usage-status failures with the logger's redacted stack
The folder reorder route checked folder locks inside its transaction but through the global pool, checking out a second connection while the transaction held one. assertFolderMutable now takes an optional executor and the route passes its transaction handle, so the ancestry walk reads on the held connection and under the folder-tree advisory lock.
…rvice outage to non-500 statuses (#9051) * fix(api): map MCP reauth redirect, deleted-account visit, and task service outage to non-500 statuses * fix(api): type the discover route test call and keep POST refresh per-server
* fix(appconfig): share the polled snapshot with Trigger.dev runs Every Trigger.dev run starts a fresh process, and each new AppConfig session is billed for the full configuration it receives. Profiles that opt in (feature-flags, block-visibility, access-control) are now published to Redis by app processes (latest validatedAt wins), and a cold cache inside a Trigger.dev run adopts a snapshot under 120s old (tolerating 5s of clock skew) instead of opening a session. Missing, stale, unreadable, or unparseable snapshots fall back to polling as before. Trigger.dev runs never publish, and strict snapshots (outbound routing) never use the shared copy. * test(appconfig): guard the Redis URL, reuse flushMacrotask, await the older publish
* fix(byok): preserve credential ownership on lookup failures * fix(byok): isolate enterprise entitlement test fixtures
…t workflow-result race (#9056) * fix(logging): log expected outcomes below ERROR and acknowledge a lost workflow-result race - Cancelling a run that already finished is a caller-facing refusal (409 / v2 no-op), so it logs at info. Unclassified failures stay at error. - The PostHog ingest proxy logs a client disconnect at info and answers 499, the same way withRouteHandler does. Upstream failures stay at error with a 502. - A client workflow result that loses the settlement CAS to the server recording the execution's outcome now re-reads the call and gets the stored outcome back. It used to get a 500. The genuine update-failure log no longer lets its completion text overwrite the log message. - An unresolved Stripe price on enterprise or legacy pro/team plans is expected, because those prices are not in the catalog. It logs at info. Catalog plans still log at error. The duplicate warn in onSubscriptionUpdate is removed. * refactor: share legacy-plan check, classify cancel refusals through asOrchestrationError Keep unresolved prices at ERROR when a legacy plan moves to a new price, cover the execution-mismatch 404 on a lost workflow race, and drop the redundant unstub hook. * fix(billing): treat only a real price swap as a legacy plan price change previous_attributes.items is present on seat-only quantity updates too, so compare the previous item price ids with the current ones instead.
* fix(mothership): surface actionable workbench file read failures * chore(lint): drop VfsPathError from unused-exports baseline * fix(mothership): classify workbench staging failures apart from source reads * fix(mothership): report a workbench socket path as not a regular file
…proxy body limit (#9057)
* fix(mothership): drop queued tab adds the server already removed A server-applied remove effect cleared the tab locally but left the client's queued add for the same resource in the persistence queue, so the next flush re-posted it and hydration re-projected the tab. The deleted resource then went out as an open tab on every later message. The remove effect now discards that pending add (the server already removed it, so no client delete is sent). Active-tab resolution now logs a missing resource at info and a revoked one at warn instead of error; other failures still log at error. * fix(mothership): type the stale-tab regression test's stream events * fix(mothership): close in-flight and recovery gaps for server-removed tabs An add already in flight when the server's removal arrives can commit after it, so the queue now schedules a compensating delete ordered after that write settles. Queued or failed adds are still dropped with no delete. Recovered remove effects (replay or deferred batch) now discard the queued add too; previously the recovery path returned before the discard ran. * fix(mothership): gate the stale-tab test stream on the client's add, not the clock * fix(mothership): use the shared deferred helper in stale-tab tests * fix(mothership): a server restore cancels a pending compensating delete An upsert effect means the server stored the resource again, so a client delete still waiting behind an in-flight add must not run. The queue drops that removal the same way a newer add does; nothing is written since the server already holds the resource. * fix(mothership): scope stale-tab fix to queued and failed adds Drops the in-flight compensating delete and the restore cancel path: each added a new client/server ordering hazard. A server remove effect, live or on recovery, now only discards a queued or failed client add for that resource; a write already in flight is left untouched.
…d for non-charging file operations (#9058)
* feat(project-files): add rendered delivery and indexed search * chore(files): shrink obsolete audit baselines * fix(files): fence content delivery and preserve shared render caches * fix(files): await delivery observers and track private CLI downloads * fix(files): bound CSV previews and preserve render provenance * fix(files): fence render names and separate compiler caches * fix(files): align search with rendered cache identities * test(files): guard required document format fixtures
* feat(project-files): add cross-owner copy and bulk transfers * fix(files): satisfy bulk operation lint checks * fix(files): harden bulk admission and staged publication * fix(files): keep download input type module-local * fix(files): recheck archive admission and hydrate copy paths once
… trim Project read transactions (#9063) * fix(project-files): requeue rendered-artifact search after commit and trim Project read transactions - Project artifact reads no longer requeue search inside the read transaction, which held the Project and source row shared and then needed them exclusively; the requeue runs after commit with its owner lock bounded by the short artifact-ready lock timeout and is best effort - Artifact, inline, and CSV preview reads authorize from their own snapshots instead of a separate preflight; artifact inputs are snapshotted only when referenced and download with bounded concurrency after a declared-size budget check; the request signal reaches render and compile - Project inline delivery streams and caches content-addressed key requests like workspace inline - Project search admits, withholds previews, and reports provenance after its snapshot commits, classifies provenance only when a gate or delivery observer needs it, and lists folders directly - A Project page embedding another owner's image is a missing input instead of a 500 - Page limits are shared from page-document.server; knip and the CI isolate build are tightened * test(project-files): start requeue checks from no search revision
…runs out (#9060) * fix(pii): load models only in workers and recycle them before memory runs out The uvicorn supervisor imports the app before spawning workers, so the module-level engine build left it holding an idle ninth copy of every model. Engines now build in the FastAPI lifespan, which runs only in serving processes. With more than one worker, each now exits after a jittered request count and the supervisor respawns it, bounding per-worker memory growth that otherwise reached the task limit about 11 days after a deploy. * improvement(pii): move the start command into pii-entrypoint.sh and build engines in lifespan
…retention (#9061) * chore(ci): tag production promotions with release-<sha> for rollback retention * improvement(ci): share one promote-image script across repos
…ery (#9064) * fix(files): restore workspace read-path behavior after rendered delivery * fix(files): keep the delivery recheck for every principal and narrow the csv parser * fix(files): keep the receipt recheck's provenance read typed for existing callers
* feat(project-files): add public sharing and protected delivery * fix(project-files): align public sharing with current renderer contracts * test(project-files): follow canonical inline cache policy * fix(project-files): enforce public delivery and share policy safety * test(project-files): explicitly retire fixture shares during teardown * test(project-files): restore sharing suite environment
…r interrupted tool (#9059) * fix(mothership): take over runs whose controller died and re-run their interrupted tool A Chat run is driven by the Sim process that admitted it. When that process stops mid-run (a deploy's old task set is terminated, or a crash) and no browser is attached, nothing takes the run over: recovery only ran from a browser reconnect, the in-flight tool's result was never recorded, and the worker eventually ended the run as detached. - Every Sim process now looks every 20s for interactive runs whose chat lock is gone and takes each over through the same recovery a reconnect runs, acting as the run's user under a delegated principal that re-checks that user's current access. The chat lock and controller token still admit exactly one controller, so a racing reconnect or another process loses. - A user who lost access gets the run settled as an error and the worker is told to stop it. - A Sim tool whose owner died before recording a result (its execution lease lapsed while the call was still running) is claimed again under a new owner token and run again, by the server for a workflow tool. Each start is counted; after three, the call fails with a clear message instead. - A takeover waits while one of the run's Sim tools still holds its lease. * fix(mothership): import the takeover by absolute path and regenerate the schema mock * fix(mothership): page past live controllers, stop a scan on shutdown, dispose chat pubsub in tests * test(mothership): assert the claim admits a first execution or an interrupted re-run * test(mothership): add execution_attempts to the browser download fixture table * fix(mothership): bound the takeover scan, stop earlier attempts' commands, decline takeovers on shutdown - The abandoned-run scan resumes from a durable cursor and examines a bounded window per round, so runs whose controller is alive cannot hide later runs. - A re-run's earlier attempts' workbench commands are handed to the workbench recovery to stop before the re-run may use the workbench; each recorded command now names the attempt that started it. - Shutdown aborts a takeover still in flight at its controller claim. - Only an authorization denial settles a run as revoked; other takeover failures leave it recoverable, as a failed reconnect does. - The suite sets REDIS_URL only when a test Redis is configured. * fix(mothership): keep the takeover stop handler module-private * fix(mothership): let the server re-claim a workflow call its dead claimant never started * fix(mothership): harden abandoned-run takeover - Add the `mothership-abandoned-run-takeover-disabled` AppConfig kill switch, checked once per scan; off by default, so takeovers run unless it is on. - Stop a controller whose chat lock lapsed while every renewal threw, so a controller partitioned from Redis cannot keep driving a run taken over. - Never revoke another call awaiting its re-run from a sibling's wait or workbench access; the sibling's workbench access still stops that call's dead attempt's commands. - Treat a workspace access loss found inside recovery like an authorization denial, by error type; other recovery failures stay recoverable. - Cover organization Assistant takeover for a member and a removed member. * fix(mothership): negate the re-runnable predicate without an optional SQL operand * fix(mothership): time the lock lapse from its acquisition, type the re-run predicates as conditions, revoke access mid-recovery in the test * test(mothership): find the attempt cap inside the typed re-run conjunction
…bles in docs markdown (#9066) * improvement(api): publish an RFC 9727 API catalog and compact indented tables in docs markdown * fix(docs): turn off table padding in the markdown serializer and assert exact catalog URLs
…9305) Trace secret projection is CPU-bound and its awaits settle as microtasks, which never let the event loop service I/O. A logs read with trace spans projects a page of traces concurrently, so large traces chained multi-second synchronous blocks back to back and starved every other request on the same server. - Check a process-wide 10ms slice at every unit of projection work (span, tool call, provider segment, content field) and yield a macrotask once it expires. The slice is handed to one waiting projection per event-loop turn, so a turn stays bounded whatever the concurrency. - Run the whole-trace content invariant per span in projection pre-order against one shared budget, so an over-budget trace fails at the span that crosses the budget instead of after projecting the whole trace and discarding it. Output is unchanged.
The logs query agent-CLI engine reads only traceSpans from each per-run detail read (GET /api/v2/logs/{runId}), so it now sends includeWorkflowState=false and the route no longer loads, sanitizes and serializes the saved workflow snapshot. The chunked Promise.all batches are replaced with the existing mapWithConcurrency sliding window (at most 5 reads in flight per query), so one slow read no longer stalls the next chunk. Rows stay in run order and output is unchanged.
…ad no output (#9308) readWorkflowExecutionStatus returns the column-only response with error/finalOutput/blockOutputs null unless the run failed, a completed run was polled with includeOutput, or selectedOutputs were requested - the only three readers of the materialized execution data. Other polls skip the archive fetch, parse, and display projection; response key order is unchanged. Skipped polls no longer emit the archive-unavailable, provenance-fault, or write-time-projected-span warnings, and a projection exception on such a poll no longer turns it into a 500. Adds regression tests for skipped polls (fail fast via a macrotask sentinel) and for selected outputs on a running run, including the truncated-row rejection.
…aching file-type's ASF loop, and patch transitive advisories (#9306) * fix(deps): update Next.js to 16.4.1, stop untrusted Office uploads reaching file-type's ASF loop, and patch transitive advisories * fix(deps): exempt the exact Electron pin from the release-age gate, like Next.js * fix(deps): carry the Next.js Vary-merging patch to 16.4.1 * test(file-parsers): prove the ZIP guard without running the stalling payload in the test worker
Logs-list pages that return execution data (includeTraceSpans/includeFinalOutput, not withheld by permissions) are capped at MATERIALIZE_CONCURRENCY (20) rows before the SQL read and at a 64MB stored-archive budget (MAX_DURABLE_LARGE_VALUE_BYTES), first row always admitted. The cursor is minted from the last returned row, so walks never skip or repeat. Applies to v1 and v2; callers may receive pages shorter than limit and should follow nextCursor. v2 limit description, OpenAPI doc, and sim-cli generated client updated.
…coercion (#9311) - PDF extraction yields a macrotask once 10ms have passed, via one shared time-sliced check used in the page loop, the per-page text-stream read loop, and between and within page-assembly passes (replaces the fixed sleep(0) every 32 pages). pdf.js's fake worker resolves messages as microtasks, so these loops never let timers or I/O run before. - Table CSV append/replace coerces rows in slices of 2,000 with a yield between slices; output and cellsRejected unchanged. - Row limit is checked right after parsing for append and replace, before coercion and before claiming the job slot: an over-limit replace now fails with 'Replace would exceed table row limit…' (HTTP 400), and an over-limit file reports the limit before a concurrent-import conflict.
…9310) - Stream each CSV export row as bounded pieces (toCsvLinePieces, 256K chars) with a cooperative yield between them, so escaping and encoding a multi-megabyte trace no longer holds the event loop; output bytes are identical. - Pieces never split a real surrogate pair; lone surrogates stay lone; non-finite or sub-1 piece sizes are guarded. - Extract the process-wide CPU-slice turnstile from trace-secret-projection.ts into lib/core/utils/event-loop.ts, shared by both callers.
* ci: balance desktop smoke shards by observed file durations * test: verify balancing guards and document duration evidence * style: use the script alias for shard planning * fix: select Playwright file suites rather than test declaration locations * test: prove desktop discovery preserves imported and unmeasured cases
- Spreadsheet buffers (xlsx/xls/xlsm/xlsb/xltx/ods) through parseBuffer parse on a one-worker worker_threads pool running the same XlsxParser, taking synchronous XLSX.read off the event loop (max event-loop gap 1.1-1.9s -> 8-14ms on 16-23 MB workbooks, identical output hashes) - Input is copied into a fresh ArrayBuffer and transferred; queued jobs hold only the caller's reference; idle workers terminate after 30s; Bun ignores resourceLimits, so the bound is concurrency - Abort drops a queued job or terminates the running worker and rejects with signal.reason; FileParserError/ZipBombError/ArchiveIntegrityError rebuild with the same class, code and message - Missing bundle, load failure, sync or async start failure, or a mid-parse crash disables the pool once (one warn) and parses in-process, draining every queued job - Enabled only on the production app server (not dev/tests/Trigger.dev); bundle built by build:xlsx-worker into apps/sim/dist (declared Turbo output), shipped via Dockerfile COPY and an outputFileTracingIncludes entry - xlsx-worker-pool.test.ts builds the real bundle and covers parity, buffer integrity, fallbacks, and cancellation
…e-serializing every subtree (#9316) Each compacted node now carries its JSON byte size, so a parent sums its children (keys, separators, child bytes) instead of re-stringifying its whole subtree, and compactSubflowResults sums its aggregate from per-result sizes. A node serializes for real only when it spills or when a toJSON (its own or a child's, which receives the key) or a reject-mode __proto__ key makes the sum unsound. The walk stays synchronous until something spills or block logs compact; children start in the same order and a sync throw still observes pending siblings. Output identical to the previous serializer on a 60,000-tree differential harness; 2-17x faster on Bun across benched shapes, loop exits about 4-5x. Adds byte-exact spill-boundary, toJSON, and subflow-aggregate regression tests.
…dlers (#9312) - EventLoopTelemetry INFO 'Event loop snapshot' every 60s (lag p50/p99/max/mean, stalls, cpuPct, inFlight, inFlightPeak, windowMs; elu only where real, inFlightDropped when non-zero) and WARN 'Event loop stalled' when a 250ms probe fires >=1s late (max 10 per window) with inFlightDuringStall as concurrency context - withRouteHandler registers each handler in an in-flight registry kept on globalThis so instrumentation and route module instances share it; entries keyed by per-call token, capped at 4,096 with oldest-first eviction - Stall window starts at the probe's due time with a 50ms edge tolerance at both ends - TSDoc documents Bun vs Node lag semantics, the ~1.0-1.25s stall floor, and what is not attributed (streamed bodies, page renders, background loops)
…ting it as missing (#9314) - Secret projection sets `contentWithheld: true` on a span when it omits any of its content (input/output/thinking/errorMessage, tool-call or provider-segment content) and on every span of a structural-only fallback; marker only, no content restored - v2 `LogTraceSpan` contract documents the optional marker (additive); regenerated openapi-v2-logs.json - `logs query` reports a withheld note and a `contentWithheld` count instead of "path not found on span"; under --where, withheld runs are kept (with --field still resolved) instead of silently filtered out
…field was omitted (#9318) - The projection omits a content field whole, so logs query treats a path as withheld only when the span is marked and the content field it reads through (input/output/thinking/errorMessage, tool-call or provider-segment content) is absent; a key missing inside a present field is path not found and fails --where again - A direct path through a withheld field (e.g. thinking) stays unknown instead of falling back to a same-named key under output - Runs whose --where could not be checked get whereUnchecked: true and a separate whereUnchecked summary count; contentWithheld counts only withheld --field rows - Usage text and the v2 contentWithheld description state whole-field semantics (openapi regenerated)
…eep logs off the anonymous one (#9317) * fix(telemetry): send browser events to the configured collector and keep logs off the anonymous one The browser telemetry route read only TELEMETRY_ENDPOINT, so a deployment exporting to its own collector through OTEL_EXPORTER_OTLP_ENDPOINT still posted browser events to the anonymous default, ignored NEXT_TELEMETRY_DISABLED, and built spans without trace or span ids under the retired instrumentationLibrarySpans field. It now resolves the endpoint and headers exactly like the server export, honors the disable flag, and emits valid OTLP. Server logs and metrics are no longer exported to the anonymous collector, which receives traces only. * fix(telemetry): never send collector credentials to the anonymous endpoint, match it by host OTEL_EXPORTER_OTLP_HEADERS belong to the operator's collector, so they are dropped when the endpoint resolves to the anonymous one; that check now compares hosts, so any spelling of the anonymous URL gets traces only. The browser relay falls back to telemetry.config.ts like the server export, and the traces URL is built on the parsed path, keeping query strings and accepting a trailing slash. * fix(telemetry): never attach operator collector credentials to browser events /api/telemetry relays unauthenticated input, so sending it with OTEL_EXPORTER_OTLP_HEADERS let anyone write spans into the operator's authenticated collector. Browser events now go only to TELEMETRY_ENDPOINT or the configured anonymous collector, without credentials; the server's own export keeps using the operator's collector. * fix(telemetry): post browser events to TELEMETRY_ENDPOINT as-is and document routing per side
…evels (#9319) The bottom-up compaction from #9316 walks a tree on one synchronous stack until something spills, so deep trees (and every rejectLargeValues tree past ~1,000 levels) overflow where the old awaiting walk compacted them. Every 256 nesting levels the walk now resumes from a microtask; later siblings wait for the resumed subtree to finish walking, so objects are first seen in the same order and the output and stored values stay identical. Adds regression tests for 10,000-level plain and 8,000-level Date-per-level trees and for a shared object first reached inside a resumed subtree.
…ield inside whole-document passes (#9323) - PDF parsing yields through the shared process-wide turnstile instead of a per-parse slicer - suppressFurniture and the line passes yield between pages instead of only between passes - Text-flow reachability is precomputed linearly, replacing a per-band-line walk that was quadratic on tall pages - The e2e build artifact includes dist/xlsx-worker.mjs so acceptance runs use the spreadsheet worker
…endpoint's traces-only export (#9321)
…ild caches (#9328) * improvement(ci): refresh integration shard balance for GitHub runners * improvement(ci): run database checks alongside application shards * improvement(ci): omit unused source-map processing from E2E builds * fix(tests): await observable document lock contention * improvement(ci): preserve builds across developer documentation edits * fix(tests): use the fixture lock-observation deadline explicitly
…im knowledge import` (#9327) * feat(knowledge): import knowledge base bundles through the API and `sim knowledge import` - v2 knowledge imports resource (create, part URLs, complete, get, cancel) over a new knowledge_import upload purpose - background runner recreates the base, tags, and documents from an exported bundle; reuses vectors when the embedding model matches and queues re-processing otherwise - knowledge_base_import job table with a stale-import janitor sweep - `sim knowledge import <bundle.simkb.zip>` plus `knowledge imports get/cancel`, and an import guide - shared dispatchBackgroundTask helper for Trigger.dev-or-in-process task dispatch * fix(knowledge): validate bundles before writing, close import cancel and cleanup gaps, and fix CI - read every chunk entry before the base is created, reject reversed offsets, and length-check vectors before decoding - confirm the import is still running before creating the base, and remove it if a cancel wins - record the import inside upload completion, so a refusal or crash cannot strand the bundle; re-read on a concurrent completion - serialize progress writes, keep the source character count, cap rebuilt text, and recheck uploads when issuing part URLs - janitor fails in-process documents a dead worker left processing, and deletes bundles when pruning settled imports - shrink the unused-exports baseline, count the new janitor passes in its test, and fix the integration test's Blob typing * fix(knowledge): re-dispatch an import whose first dispatch was lost, and remove a base whose tag setup fails - the base takes the import's id and a worker claims the import before working, so a second dispatch exits untouched - a retried completion dispatches again an import no worker has claimed - a worker that finds its import settled before any claim deletes the unused bundle - tag-definition setup that fails removes the still-empty base * fix(knowledge): keep a failed dispatch from stopping a live import, audit each import once, and close review gaps - a dispatch never settles the import itself; a retry or the stale-import janitor does - projected audit entries take an idempotency key, recorded through recordAuditOnce, so a retried completion audits the import exactly once - a cancel that races completion stops the record the completion wrote - refuse empty rebuilt chunk-only documents; knowledge_base_import.user_id references the user * fix(knowledge): keep import records through account deletion, and recheck uploads on re-dispatch - knowledge_base_import.user_id is attribution only, so the janitor's cleanup record outlives its importer's account - re-dispatching an unclaimed import repeats the knowledge.upload assertion * fix(knowledge): audit an import as started, and drop the mock-only dispatch test - the completion records knowledge_base.import_started; the base exists only once the runner creates it - background-task.test.ts only checked forwarded mock arguments; the delete-async route test and the integration suite exercise the helper's real paths
…#9331) * feat(mothership): run unattended turns durably on the leased chat path A scheduled job's occurrence must outlive the Sim process driving it. The headless lifecycle the wake door uses takes no controller lease, so a pod death leaves its run unrecoverable. An unattended turn now admits like a typed send (user message, run and recovery intent in one commit under the chat lock), runs detached from any HTTP response, and records a new unattended_stream recovery kind. Takeover restores it non-interactive with the secret authority it was admitted with, instead of the interactive default, and the abandoned-run sweep, stream liveness and billing-admission recording treat both durable kinds alike. Durable start now also forwards a caller's billing-admission hook, so a job can key its per-run budget on the run's billing request. The integration test kills the turn's controller three times mid-run; it fails on a headless implementation, on a sweep blind to the new kind, and on a takeover that restores the interactive default. * feat(db): add scheduled job occurrence columns and archive legacy jobs Scheduled jobs return as Sim-owned schedules whose occurrences run as durable chat turns. A job's run record (job_execution_logs) gains the occurrence it runs (unique per schedule), its outcome and summary, and the billing request its charges settle under; a schedule gains a per-run spend cap. Indexes are built concurrently. Every job row written for the retired Go runner is archived, never deleted, so none can start once the tick claims jobs again. The archive is a script migration committing per keyed batch, so it finishes on any row count and resumes after a crash. * feat(mothership): run scheduled jobs from the schedule tick Behind MOTHERSHIP_JOBS_ENABLED (off by default, and to stay off in production until unattended runs hold production effects for approval), the schedule tick gains a job lane: - Claim: each due job's occurrence is recorded (queued) and the schedule advances past it in one transaction, skipping locked rows, so two ticks record it once and a crash never leaves a stale claim. Missed occurrences collapse into one; cadence is never under 15 minutes; a one-shot job (no cron) runs once and completes; max_runs completes. - Dispatch: the occurrence starts as an unattended durable turn in the job's chat under its occurrence id (UUIDv5 of schedule and instant), so every retry finds the turn it admitted. A busy chat skips the occurrence; a payer over its plan limit skips it; an owner who lost access turns the job off. Only secrets a person named for the job mount. - Reconcile: a queued occurrence whose dispatch was lost is dispatched again after two minutes; a running one whose turn ended is finalized; one past six hours is stopped. - Finalize: exactly once under a row lock. A reported outcome is kept; a turn that ended well without one is not_met, summarized by its last reply. Three failed or interrupted runs in a row turn the job off and email its people; a met until_complete job completes. - Budget: each billing callback records the run's cumulative settled cost on its record (keyed by the run's billing request), and a run at its cap pauses at its next step with a job_run_budget card. The cost is cumulative per billing request, which recovered legs reuse, so the cap survives worker hand-offs and Sim takeovers. * improvement(mothership): run the job lane unflagged behind one indexed probe * improvement(mothership): simplify the job lane and stop jobs through the shared Stop delivery * chore(db): format the scheduled job migration metadata * improvement(mothership): keep the durable-run predicates out of the chat protocol graph * fix(mothership): settle job runs before claiming, keep spend-cap links and failure counts durable * fix(mothership): stop orphaned job runs at their limit and skip any number of deleted occurrences * fix(mothership): end a running job record whose run was removed
Replaces dev's content with staging's. dev's own commits are kept on archive/dev-pre-sync-2026-10-23.
|
The latest updates on your projects. Learn more about Vercel for GitHub. |
Contributor
 [High impact] The sync reaches the full dev application and its database schema. Prepare the existing dev database for Projects before merging; otherwise the dev deployment stops before applying schema changes. SummaryThis PR replaces the dev tree with staging while preserving dev history.
|
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
7 issues found across 3000 files
Confidence score: 3/5
signup-form.tsxreportssignup_completedbefore OTP verification, so abandoned or unverified signups can inflate conversion counts. Only report it immediately when verification is disabled.migrate.ymlcan run migrations concurrently against the same environment. Add a concurrency group keyed oninputs.environmentand disable cancellation.landing-seo-geo.mdgives conflicting guidance on whether the H1 should include both “AI workspace” and “AI agents” or only “AI agents.” Align the two recommendations.docs-negotiation.ymlcan skip docs checks when changes topackages/**affect their imported dependencies. Addpackages/**to both path filters.
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name=".claude/rules/landing-seo-geo.md">
<violation number="1" location=".claude/rules/landing-seo-geo.md:27">
P2: The H1 guidance and the keyword-density guidance conflict. The SEO bullet says the H1 should lead with "AI workspace" and "AI agents", but the GEO bullet says the visible headline should name only "AI agents", not "AI workspace". Pick one and make both bullets match.</violation>
</file>
<file name=".cursor/rules/sim-react-performance.mdc">
<violation number="1" location=".cursor/rules/sim-react-performance.mdc:3">
P3: The new `globs` scope stops this rule from loading for `packages/utils`, `apps/realtime`, and scripts. The paragraph below it says the copy-then-sort rule applies everywhere, so either restore `alwaysApply: true` or widen the globs to match.</violation>
</file>
<file name=".github/workflows/docs-negotiation.yml">
<violation number="1" location=".github/workflows/docs-negotiation.yml:6">
P2: The `paths` filters omit `packages/**`, but the docs e2e scripts import `@sim/logger` and `@sim/utils`. A change to those packages can break the docs checks without this workflow running. Add `packages/**` to both the `pull_request` and `push` path lists.</violation>
</file>
<file name=".claude/rules/sim-components.md">
<violation number="1" location=".claude/rules/sim-components.md:43">
P2: The new rationale for `js-tosorted-immutable` contradicts the removed text: the old section said `tsconfig` `lib: ES2023` is set and only affects type-checking. Verify the tsconfig `lib` value. If it is ES2023, `tsc` accepts `toSorted`, so this rationale is wrong. Restore the runtime reason (Safari <16 / iOS 15 throws `TypeError`) so the won't-fix stays justified.</violation>
</file>
<file name=".github/workflows/migrate.yml">
<violation number="1" location=".github/workflows/migrate.yml:36">
P2: This workflow has no `concurrency` group, so two runs targeting the same environment can apply schema changes at the same time. Add a `concurrency` group keyed on `inputs.environment` with `cancel-in-progress: false` so migrations queue instead of overlapping.</violation>
</file>
<file name=".agents/skills/add-managed-cli/SKILL.md">
<violation number="1" location=".agents/skills/add-managed-cli/SKILL.md:114">
P2: This drops `sandboxes.test.ts` (contracts) and `sandbox-editor.test.tsx` from the required validation list, yet section 2 now relies on UI and contract tests for the registry boundary. Restore both paths so the documented validation still covers the contract and editor surfaces.</violation>
</file>
<file name="apps/sim/app/(auth)/signup/signup-form.tsx">
<violation number="1" location="apps/sim/app/(auth)/signup/signup-form.tsx:351">
P2: This reports `signup_completed` before a verification-enabled signup completes OTP verification, so abandoned or unverified accounts are counted as conversions. Report immediately only when verification is disabled; otherwise defer until OTP completion and confirm the refetched session user ID matches `response.data.user.id`.</violation>
</file>
Heads up: you’re close to your flex budget. Increase your flex budget so reviews don’t pause.
Note: This PR contains a large number of files. cubic selects up to 200 of the highest-priority eligible files for this review, so some files may not have been reviewed.
Turn on auto-fix | Re-trigger cubic
| - **Specific numbers**: prefer concrete figures ("1,000+ integrations", "15+ AI providers") over vague claims. | ||
| - **Answer-first pattern**: each section's H2 + first paragraph directly answers a question a user would ask an AI ("What is Sim?", "What integrations does Sim support?", "How much does Sim cost?"). | ||
| - **Atomic answer blocks**: every feature card, template, and pricing tier is independently quotable — self-contained, with "Sim" named explicitly. Never "the platform", "our tool", or a bare pronoun as the subject. | ||
| - **Keyword density**: the hero's `sr-only` summary is the first text in the hero's DOM, so its first 150 characters name "Sim", "AI workspace", and "AI agents" for crawlers. The visible headline and description name "AI agents" (not "AI workspace" or "Sim"); "Sim" is otherwise carried by the title tag and the meta description. |
Contributor
There was a problem hiding this comment.
P2: The H1 guidance and the keyword-density guidance conflict. The SEO bullet says the H1 should lead with "AI workspace" and "AI agents", but the GEO bullet says the visible headline should name only "AI agents", not "AI workspace". Pick one and make both bullets match.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At .claude/rules/landing-seo-geo.md, line 27:
<comment>The H1 guidance and the keyword-density guidance conflict. The SEO bullet says the H1 should lead with "AI workspace" and "AI agents", but the GEO bullet says the visible headline should name only "AI agents", not "AI workspace". Pick one and make both bullets match.</comment>
<file context>
@@ -9,23 +9,24 @@ paths:
-- **Specific numbers**: prefer concrete figures ("1,000+ integrations", "15+ AI providers") over vague claims.
+- **Answer-first pattern**: each section's H2 + first paragraph directly answers a question a user would ask an AI ("What is Sim?", "What integrations does Sim support?", "How much does Sim cost?").
+- **Atomic answer blocks**: every feature card, template, and pricing tier is independently quotable — self-contained, with "Sim" named explicitly. Never "the platform", "our tool", or a bare pronoun as the subject.
+- **Keyword density**: the hero's `sr-only` summary is the first text in the hero's DOM, so its first 150 characters name "Sim", "AI workspace", and "AI agents" for crawlers. The visible headline and description name "AI agents" (not "AI workspace" or "Sim"); "Sim" is otherwise carried by the title tag and the meta description.
+- **sr-only summaries**: the hero (and Templates) each carry a `<p className="sr-only">` (~50 words) stating what Sim is, who it's for, and what it does — a clean citation target for AI summarizers. The hero's summary opens by naming Sim.
+- **Specific numbers**: concrete figures ("1,000+ integrations", "every major LLM", "100,000+ builders") over vague claims — and only numbers that are true and shipped.
</file context>
| on: | ||
| pull_request: | ||
| paths: | ||
| - 'apps/docs/**' |
Contributor
There was a problem hiding this comment.
P2: The paths filters omit packages/**, but the docs e2e scripts import @sim/logger and @sim/utils. A change to those packages can break the docs checks without this workflow running. Add packages/** to both the pull_request and push path lists.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At .github/workflows/docs-negotiation.yml, line 6:
<comment>The `paths` filters omit `packages/**`, but the docs e2e scripts import `@sim/logger` and `@sim/utils`. A change to those packages can break the docs checks without this workflow running. Add `packages/**` to both the `pull_request` and `push` path lists.</comment>
<file context>
@@ -0,0 +1,67 @@
+on:
+ pull_request:
+ paths:
+ - 'apps/docs/**'
+ - 'patches/next@*.patch'
+ - 'package.json'
</file context>
|
|
||
| - `no-barrel-import` — barrel imports are the repo convention (see sim-imports.md, "Barrel Exports"). Keep them. | ||
| - `js-tosorted-immutable` — in `'use client'` code, keep `[...arr].sort(cmp)`; `toSorted` is unpolyfilled and crashes Safari <16 / iOS 15 (see "List-render performance" above). Only apply it in server-only modules. | ||
| - `js-tosorted-immutable` — won't-fix anywhere; `tsc` rejects the ES2023 array methods, because no tsconfig raises `lib` past ES2022. |
Contributor
There was a problem hiding this comment.
P2: The new rationale for js-tosorted-immutable contradicts the removed text: the old section said tsconfig lib: ES2023 is set and only affects type-checking. Verify the tsconfig lib value. If it is ES2023, tsc accepts toSorted, so this rationale is wrong. Restore the runtime reason (Safari <16 / iOS 15 throws TypeError) so the won't-fix stays justified.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At .claude/rules/sim-components.md, line 43:
<comment>The new rationale for `js-tosorted-immutable` contradicts the removed text: the old section said `tsconfig` `lib: ES2023` is set and only affects type-checking. Verify the tsconfig `lib` value. If it is ES2023, `tsc` accepts `toSorted`, so this rationale is wrong. Restore the runtime reason (Safari <16 / iOS 15 throws `TypeError`) so the won't-fix stays justified.</comment>
<file context>
@@ -6,28 +6,41 @@ paths:
- `no-barrel-import` — barrel imports are the repo convention (see sim-imports.md, "Barrel Exports"). Keep them.
-- `js-tosorted-immutable` — in `'use client'` code, keep `[...arr].sort(cmp)`; `toSorted` is unpolyfilled and crashes Safari <16 / iOS 15 (see "List-render performance" above). Only apply it in server-only modules.
+- `js-tosorted-immutable` — won't-fix anywhere; `tsc` rejects the ES2023 array methods, because no tsconfig raises `lib` past ES2022.
- `rerender-state-only-in-handlers` / "state set but never rendered" — a false positive when the `useState` is consumed by a `useEffect`/`useLayoutEffect` dependency (the effect must re-run on change). Only convert to a ref when nothing reads the value reactively.
- `no-render-in-render` — a helper *called inline* (`{renderRow()}`) is reconciled by position and does **not** remount, so extracting it to a component is usually pure churn and can regress behavior (prop-drilling many closures, focus/scroll loss on the inner `<input>`). Apply it only when the helper is genuinely a *component defined during render*, or when the move is mechanical (a stateless, ref-free helper whose closures become a small, explicit prop set).
</file context>
Suggested change
| - `js-tosorted-immutable` — won't-fix anywhere; `tsc` rejects the ES2023 array methods, because no tsconfig raises `lib` past ES2022. | |
| - `js-tosorted-immutable` — won't-fix anywhere: `toSorted` is unpolyfilled and throws `TypeError` on Safari <16 / iOS 15, so keep `[...arr].sort(cmp)` (see CLAUDE.md > Code Conventions). |
| contents: read | ||
|
|
||
| jobs: | ||
| migrate: |
Contributor
There was a problem hiding this comment.
P2: This workflow has no concurrency group, so two runs targeting the same environment can apply schema changes at the same time. Add a concurrency group keyed on inputs.environment with cancel-in-progress: false so migrations queue instead of overlapping.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At .github/workflows/migrate.yml, line 36:
<comment>This workflow has no `concurrency` group, so two runs targeting the same environment can apply schema changes at the same time. Add a `concurrency` group keyed on `inputs.environment` with `cancel-in-progress: false` so migrations queue instead of overlapping.</comment>
<file context>
@@ -0,0 +1,90 @@
+ contents: read
+
+jobs:
+ migrate:
+ name: migrate
+ runs-on: ${{ (vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith') && 'blacksmith-4vcpu-ubuntu-2404' || 'ubuntu-latest' }}
</file context>
Comment on lines
+114
to
+119
| bun run --cwd apps/sim test \ | ||
| lib/execution/remote-sandbox/cli-tools.test.ts \ | ||
| lib/execution/remote-sandbox/cli-tools-boundary.test.ts \ | ||
| lib/execution/remote-sandbox/sandbox-spec.test.ts \ | ||
| lib/execution/remote-sandbox/spec.test.ts \ | ||
| lib/execution/remote-sandbox/resolve.test.ts \ | ||
| lib/api/contracts/sandboxes.test.ts \ | ||
| 'app/workspace/[workspaceId]/settings/components/sandboxes/utils.test.ts' \ | ||
| 'app/workspace/[workspaceId]/settings/components/sandboxes/components/sandbox-editor.test.tsx' | ||
| 'app/workspace/[workspaceId]/settings/components/sandboxes/utils.test.ts' |
Contributor
There was a problem hiding this comment.
P2: This drops sandboxes.test.ts (contracts) and sandbox-editor.test.tsx from the required validation list, yet section 2 now relies on UI and contract tests for the registry boundary. Restore both paths so the documented validation still covers the contract and editor surfaces.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At .agents/skills/add-managed-cli/SKILL.md, line 114:
<comment>This drops `sandboxes.test.ts` (contracts) and `sandbox-editor.test.tsx` from the required validation list, yet section 2 now relies on UI and contract tests for the registry boundary. Restore both paths so the documented validation still covers the contract and editor surfaces.</comment>
<file context>
@@ -110,24 +110,20 @@ Never commit downloaded artifacts or credentials.
-
```bash
-bunx vitest run \
+bun run --cwd apps/sim test \
lib/execution/remote-sandbox/cli-tools.test.ts \
lib/execution/remote-sandbox/cli-tools-boundary.test.ts \
</file context>
Suggested change
| bun run --cwd apps/sim test \ | |
| lib/execution/remote-sandbox/cli-tools.test.ts \ | |
| lib/execution/remote-sandbox/cli-tools-boundary.test.ts \ | |
| lib/execution/remote-sandbox/sandbox-spec.test.ts \ | |
| lib/execution/remote-sandbox/spec.test.ts \ | |
| lib/execution/remote-sandbox/resolve.test.ts \ | |
| lib/api/contracts/sandboxes.test.ts \ | |
| 'app/workspace/[workspaceId]/settings/components/sandboxes/utils.test.ts' \ | |
| 'app/workspace/[workspaceId]/settings/components/sandboxes/components/sandbox-editor.test.tsx' | |
| 'app/workspace/[workspaceId]/settings/components/sandboxes/utils.test.ts' | |
| bun run --cwd apps/sim test \ | |
| lib/execution/remote-sandbox/cli-tools.test.ts \ | |
| lib/execution/remote-sandbox/cli-tools-boundary.test.ts \ | |
| lib/execution/remote-sandbox/spec.test.ts \ | |
| lib/execution/remote-sandbox/resolve.test.ts \ | |
| lib/api/contracts/sandboxes.test.ts \ | |
| 'app/workspace/[workspaceId]/settings/components/sandboxes/utils.test.ts' \ | |
| 'app/workspace/[workspaceId]/settings/components/sandboxes/components/sandbox-editor.test.tsx' |
| @@ -348,6 +348,7 @@ function SignupFormContent({ | |||
| } | |||
|
|
|||
| if (measurement) trackGoogleEvent('sign_up', { method: 'email' }) | |||
| if (marketing) trackFreebuffConversion('signup_completed', response.data.user.id) | |||
Contributor
There was a problem hiding this comment.
P2: This reports signup_completed before a verification-enabled signup completes OTP verification, so abandoned or unverified accounts are counted as conversions. Report immediately only when verification is disabled; otherwise defer until OTP completion and confirm the refetched session user ID matches response.data.user.id.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At apps/sim/app/(auth)/signup/signup-form.tsx, line 351:
<comment>This reports `signup_completed` before a verification-enabled signup completes OTP verification, so abandoned or unverified accounts are counted as conversions. Report immediately only when verification is disabled; otherwise defer until OTP completion and confirm the refetched session user ID matches `response.data.user.id`.</comment>
<file context>
@@ -348,6 +348,7 @@ function SignupFormContent({
}
if (measurement) trackGoogleEvent('sign_up', { method: 'email' })
+ if (marketing) trackFreebuffConversion('signup_completed', response.data.user.id)
try {
</file context>
| --- | ||
| description: "Behavior-preserving React render-performance idioms" | ||
| alwaysApply: true | ||
| globs: ["apps/sim/**/*.ts","apps/sim/**/*.tsx","packages/emcn/**","packages/workflow-renderer/**"] |
Contributor
There was a problem hiding this comment.
P3: The new globs scope stops this rule from loading for packages/utils, apps/realtime, and scripts. The paragraph below it says the copy-then-sort rule applies everywhere, so either restore alwaysApply: true or widen the globs to match.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. When an issue isn't valid or won't be fixed in this PR, reply in its thread with the reason and then resolve the thread. At .cursor/rules/sim-react-performance.mdc, line 3:
<comment>The new `globs` scope stops this rule from loading for `packages/utils`, `apps/realtime`, and scripts. The paragraph below it says the copy-then-sort rule applies everywhere, so either restore `alwaysApply: true` or widen the globs to match.</comment>
<file context>
@@ -1,6 +1,6 @@
---
description: "Behavior-preserving React render-performance idioms"
-alwaysApply: true
+globs: ["apps/sim/**/*.ts","apps/sim/**/*.tsx","packages/emcn/**","packages/workflow-renderer/**"]
---
</file context>
Suggested change
| globs: ["apps/sim/**/*.ts","apps/sim/**/*.tsx","packages/emcn/**","packages/workflow-renderer/**"] | |
| alwaysApply: true |
This branch was previously deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
devmatchstagingexactly (tree-identical); mergesorigin/devwith theoursstrategy so this fast-forwards dev without rewriting historyarchive/dev-pre-sync-2026-10-23db:push --force, non-interactive renames) brings the dev DB to staging's schema and drops tables/columns only the archived dev commits usedType of Change
Testing
git diff origin/staging HEADis empty;origin/devis an ancestor of HEADChecklist