feat(silo,queries)!: Effect engine + status-discriminated handle, shared by queries by scottmessinger · Pull Request #87 · commoncurriculum/supergrain

scottmessinger · 2026-06-03T14:10:50Z

Summary

Rebuilds @supergrain/silo's network/async layer on an internal Effect engine, remodels the reactive handle as a flat status-discriminated union, and brings @supergrain/queries onto the same engine so a query fetch behaves exactly like a document fetch instead of feeling like a separate package. Adapters stay Promise-first; Effect powers the engine internally and is not required at the boundary.

This is net-additive in capability versus main — nothing users have today regresses.

The engine (`@supergrain/silo`)

One adapter runner — runAdapter. A single exported entrypoint turns any adapter call into a typed, resilient, abortable Effect: the Promise→AdapterError boundary, a fresh per-attempt AbortController, retry (a Schedule), and timeout (a Duration). The store's finder and @supergrain/queries both go through it, so resilience/abort behave identically on every surface. The finder's private toAdapterEffect/adapterEffect duplicates are gone, and the document/query drains share one settleChunk pipeline.
Shared default retry. A built-in defaultRetry (fibonacci 1s–60s, retrying until success) plus a store-wide DocumentStoreConfig.retry / timeout. Every document and query fetch that doesn't set its own retry inherits the store's defaults.retry, so both surfaces retry identically out of the box. Precedence: per-model / per-query → store-wide → built-in default. Disable with Schedule.recurs(0), or bound it with Schedule.recurs(3).
Typed error channel — AdapterError / NotFoundError / ProcessorError (union SiloError), exported from the root.
Adapters may return an Effect (not just a Promise) to own the failure channel / compose retries / manage resources.
AbortSignal plumbing — find(ids, { signal }) aborts when the adapter Effect is interrupted (e.g. a timeout fires or a retry abandons the prior attempt).
Optional onError telemetry sink on DocumentStoreConfig — called with the typed error + failing type/keys whenever a fetch settles into a failure. A throwing sink is isolated so observability can't break the engine.
Effect-clock batch window — deterministic timing in tests.

Status-discriminated handle (breaking — API shape, not capability)

DocumentHandle / QueryHandle are now a status-discriminated union over flat fields (value, error, isFetching, fetchedAt, promise):

handle.data → handle.value
handle.isPending → handle.value === undefined && handle.isFetching
handle.hasData → handle.value !== undefined
error is now a typed SiloError
status literals are lowercase; the old "IDLE" folded into "pending". TypeScript flags any leftover "IDLE" comparison, so it can't break silently.

Migration guidance is in .changeset/silo-effect-migration.md and the silo README.

`@supergrain/queries` runs on the same engine (breaking)

createQuery runs every fetch through runAdapter on an interruptible fiber.
Resilience config matches ModelConfig: retry?: Schedule + timeout?: Duration replace the bespoke backoff loop. With no per-query retry, the query inherits the store's default (fibonacci defaultRetry, or a store-wide override) — so it retries like a document find. fibonacciBackoff is removed (its behavior is now defaultRetry).
QueryAdapter.fetch(id, { offset, limit, signal }) gains an AbortSignal; Query.error is now typed SiloError | undefined.
Single-flight: a new refetch() / fetchNextPage() / destroy() interrupts the in-flight fetch (its adapter signal aborts), so overlapping requests can't race to write the store.

  // retry is inherited from the store default; override per-query if needed:
- createQuery({ store, adapter, type, id, backoff: (n) => n * 1000 });
+ createQuery({ store, adapter, type, id, retry: Schedule.recurs(3) });

On cancellation

Auto-cancellation of in-flight document fetches on component unmount (silo) was prototyped on this branch and deliberately reverted (it required vendoring a fork of alien-signals' operator layer for a marginal benefit in a normalized shared cache). The silo hooks remain pure reactive reads; an in-flight fetch completes and populates the shared cache. @supergrain/queries, by contrast, now cancels imperatively: a superseding fetch or destroy() aborts the in-flight run.

Misc

Hardened stableStringify (query slot keys): encodes Date / bigint and is total — distinct Date params no longer collapse onto one cache slot.

Checks

All five CI gates pass locally: pnpm test (616), pnpm run test:validate (5), pnpm run typecheck, pnpm lint, pnpm format.

Generated by Claude Code

Design sketch comparing a tagged-union public DocumentHandle/QueryHandle (Data.TaggedEnum + $match) against keeping the flat handle shape with an internal statechart, as the first step toward migrating silo's network/async orchestration to Effect. No code wired up yet. https://claude.ai/code/session_016FNH6xY9sFPmBaThUcHPAc

…roxy Achieves type safety (illegal states unrepresentable, exhaustive matching) AND per-field reactivity by separating runtime representation from type: keep the existing stable reactive proxy with in-place field mutation, but type the public handle as a status-discriminated union. Effect's Data.TaggedEnum statechart style is used internally for the transition reducer only. https://claude.ai/code/session_016FNH6xY9sFPmBaThUcHPAc

A single async enum can't represent stale-data-plus-background-refetch or stale-data-plus-refetch-error without throwing information away. Model the lifecycle as two orthogonal statechart regions instead: data availability (Absent | Present) and fetch activity (Idle | Fetching | Failed). All six combinations are meaningful; value/error stay type-narrowed per region; the two reactive cells preserve per-field reactivity. matchHandle collapses the product into the four common consumer cases. https://claude.ai/code/session_016FNH6xY9sFPmBaThUcHPAc

No collapsing into a Ready super-state. Consumers narrow each region directly; all six data x fetch combinations are first-class. Add the rationale for two regions over a flat six-variant enum: a single discriminant would force value-readers to subscribe to background fetch activity, re-rendering on every refetch — the factoring into two cells is what preserves per-field reactivity. https://claude.ai/code/session_016FNH6xY9sFPmBaThUcHPAc

…ndle BREAKING CHANGE: adapters now return Effect instead of Promise; the DocumentHandle/QueryHandle flat fields are replaced by two orthogonal regions (data: Absent|Present, fetch: Idle|Fetching|Failed). - src/errors.ts: AdapterError/NotFoundError/ProcessorError (Data.TaggedError) - src/transitions.ts: HandleEvent tagged enum + pure region reducers + applyEvent (promise lifecycle) - src/store.ts, src/queries.ts: two-region handles, Effect adapters, per-model retry/timeout - src/finder.ts: Effect.forEach fan-out, typed catchAll, statechart settlement - effect as a peer dependency; READMEs, docstrings, changeset updated Tests are migrated in a follow-up commit. https://claude.ai/code/session_016FNH6xY9sFPmBaThUcHPAc

…vity robustness - Translate all silo tests + queries test to Effect adapters and the two-region handle (data: Absent|Present, fetch: Idle|Fetching|Failed). - applyEvent: read handle state UNTRACKED (via unwrap) while writing through the reactive proxy, so insertDocument inside a tracked render no longer self-subscribes and loops; genuine readers are still notified. - Build events from lowercase factory + plain tagged literals (keeps the TaggedEnum type), resolve oxlint findings across errors/finder/transitions. - queries: Effect.succeed stub adapters + effect devDependency. All green: silo 144, queries 32, doc-tests 5; typecheck/lint/format clean. https://claude.ai/code/session_016FNH6xY9sFPmBaThUcHPAc

…d handle silo: - Handle is now a status-discriminated union over flat fields (value/error/isFetching/fetchedAt/status/promise). value/error coexist in the success arm (stale-while-revalidate); isFetching is orthogonal to status so status doesn't flip on a background refetch; per-field reactivity intact. - insertDocument again clears a prior error (fresh value supersedes it). - Statechart unit tests + retry/timeout + error tests restore src to 100% coverage (167 tests). queries: - QueryAdapter.fetch returns Effect<_, AdapterError>; create-query runs it via Effect.runPromise(Effect.either(...)); pagination/backoff unchanged. husk: - reactivePromise/reactiveTask accept Effect programs, run via Effect.runPromise(Effect.either(effect), { signal }) bridging the existing abort model to Effect interruption; typed error channel. effect added as a peer dependency of silo, queries, and husk. Docs + changeset updated to the flat union shape. https://claude.ai/code/session_016FNH6xY9sFPmBaThUcHPAc

…ngine Keep Effect as the internal engine but take it off the public boundary, so adopting the libraries no longer requires writing Effect. silo: - DocumentAdapter.find returns Promise<unknown> | Effect<unknown, AdapterError>. The common case is a plain Promise; the finder normalizes it onto the Effect engine (Effect.isEffect ? use : Effect.tryPromise) and a rejection becomes an AdapterError (passed through untouched if the adapter already threw one). Effect adapters still work as-is for typed errors / custom retries / resources. - Engine (batching, retry/timeout, statechart) and typed errors unchanged. queries: - QueryAdapter.fetch returns Promise | Effect; create-query normalizes the same way before funneling through Either into its backoff loop. husk: - Reverted to the Promise-based reactivePromise/reactiveTask (no Effect engine to keep; its boundary IS the primitive). Drops the effect dependency. Promise/Effect boundary tests added (finder + create-query); silo executable src stays at 100% coverage. Docs + changeset updated to Promise-first. https://claude.ai/code/session_016FNH6xY9sFPmBaThUcHPAc

The engine now runs entirely on Effect, with subscriber-gated request cancellation — the capability the fiber foundation was for. - Batch window runs on Effect.sleep (was setTimeout), so the whole pipeline is on Effect's clock and deterministic under TestClock. - Each chunk runs on its own interruptible fiber, tracked as in-flight. - The store ref-counts subscribers per key: subscribeDocument/subscribeQuery return an unsubscribe fn; useDocument/useQuery call them on mount/unmount. When the last subscriber for every key in an in-flight chunk leaves, the chunk's fiber is interrupted and its handles reset to idle (new Abort event) so renewed interest refetches. - Adapters receive an AbortSignal: find(ids, { signal }) — thread it into fetch for a real network abort, or ignore it (interruption still discards the result, no stale write). Effect adapters interrupt natively. - gcTimeMs config (default 0 = next tick) defers the interrupt so a synchronous re-subscribe (StrictMode remount, fast nav-back) cancels it. New tests: cancellation.test.ts (subscriber-gating, partial-batch, re-subscribe, refetch, signal-ignored discard, query cancellation, ref-count edges, evicted handle/bucket) and test-clock.test.ts (TestClock window). silo src 100% covered; 184 silo tests pass. https://claude.ai/code/session_016FNH6xY9sFPmBaThUcHPAc

Two changeset claims lacked tests: - React hooks cancel on unmount: add browser tests proving useDocument's subscribe/unsubscribe wiring aborts the in-flight fetch when the last consumer unmounts, and keeps it alive while another component still wants the same doc (shared-cache ref-counting, end-to-end through StrictMode). - 'Effect adapters interrupt natively': add a node test where an Effect adapter hangs on Effect.never with an onInterrupt finalizer; cancellation runs the adapter's own finalizer and resets the handle. https://claude.ai/code/session_016FNH6xY9sFPmBaThUcHPAc

…er seam Thermo-nuclear review follow-ups on the cancellation machinery: - Collapse the subscriber ref-count store from a nested Record<Surface, Map<type, Map<key, count>>> to a single flat Map<string, number> keyed by the same 'surface type key' string already used for gc timers. Unifies the keying convention, shortens subscribe/unsubscribe/ subCount, and deletes the 'unknown type' special-case branch (the flat map handles a missing key uniformly). - Remove FinderScheduler: a production injection seam whose only consumer was test-clock.test.ts, and which diverged from silo's canonical fake-timer test pattern. The batch window stays on Effect.sleep; the fake-timer cancellation and finder tests already prove its behavior deterministically. Reverts the three call sites to Effect.runFork/runPromise and drops the TestClock test. No behavior change. silo src 100% covered; 186 tests pass. https://claude.ai/code/session_016FNH6xY9sFPmBaThUcHPAc

…second serializer useQuery computed its own JSON.stringify(params) for the effect dependency while the store keys queries with stableStringify. Two serializers for one concept: order-sensitive vs order-independent, so the subscription key and the rendered handle could diverge (latent correctness hazard), and reordered-param re-renders churned the subscription needlessly. findQuery already returns the SAME reactive handle for deep-equal params (it owns the one canonical key). Depend on that stable identity instead — the signals-native key. No second serialization, nothing to drift, no churn: deep-equal params keep the same handle (effect doesn't re-run); a different params yields a different handle (re-subscribes). Pin the order-independence contract the hook now relies on with a findQuery test that reorders param keys and asserts a stable handle. https://claude.ai/code/session_016FNH6xY9sFPmBaThUcHPAc

…lation opt-in The hooks no longer use useEffect to drive an imperative subscription. They are pure reactive reads now: `return store.find(...)` / `return store.findQuery(...)`. The whole point of useQuery is that consumers don't reach for useEffect — and neither should the hook, when the value it adds is just a reactive read. Fetch cancellation stays as an explicit, tested capability on the store (subscribeDocument / subscribeQuery + fiber interruption + AbortSignal), for callers that want to wire unmount-driven cancellation themselves. It is no longer auto-wired through React: a signals-native 'cancel when a handle has no reactive observers' wants an observation-lifecycle primitive in the kernel core (alien-signals 2.0.7 has none; effect-cleanup landed in 3.x) — a separate change to the reactive core, not silo. - Drop useEffect + the hook-level subscription from useDocument/useQuery. - Remove the hook-driven cancellation tests (covered behavior is gone); the store-level cancellation tests remain and still exercise the finder machinery. - Update store/README/changeset docs to frame cancellation as opt-in. silo src 100% covered; 185 tests pass. https://claude.ai/code/session_016FNH6xY9sFPmBaThUcHPAc

Upgrade the reactive core. 3.x renamed getCurrentSub/setCurrentSub → getActiveSub/setActiveSub (renamed throughout; kernel/internal re-exports the new names) and moved ReactiveNode to the alien-signals/system subpath. The notable behavioral change: effect(fn) now treats fn's return value as a cleanup function (runs before each re-run and on dispose) — the React-useEffect model. Callbacks that returned a value relying on 2.x's lenient ignore now throw 'cleanup is not a function' on re-run; fixed the affected test/bench callbacks to read-for-subscription via void. useSignalEffect now accepts and wires through a cleanup return. Documented the semantics on the public effect re-export. All reactive semantics (fine-grained tracking, batching, Map/Set notification coalescing) verified unchanged: 621 tests across kernel/mill/husk/silo, doc validation, typecheck, lint, format all green. https://claude.ai/code/session_016FNH6xY9sFPmBaThUcHPAc

…n in silo Kernel now owns its reactive operator layer on top of `alien-signals/system`'s `createReactiveSystem` (faithful port of alien-signals 3.x) so it can fire per-node observation callbacks when a reactive node loses its last subscriber. New `onObservationChange` / `getObservationNode` (public) and `trackNode` / `isObserved` (internal); the dispatch is counter-gated so the hot path is unchanged when observation is unused. All kernel + husk imports repointed to the owned system module. Silo replaces its opt-in `subscribe*` ref-counting with observation-driven cancellation: each handle carries a dedicated liveness node that the rendering component subscribes to via `find`/`findQuery`; when the last observer unmounts, the in-flight fetch is interrupted after the `gcTimeMs` grace window (partial-batch rule honored). `useDocument` / `useQuery` stay pure reactive reads. Removed `subscribeDocument` / `subscribeQuery`. https://claude.ai/code/session_018FyYY6YYb1CW5LnBfPMgod

…tion path Addresses self-review feedback on the observation primitive + silo cancellation: - Kernel: defer `onUnobserved` to a coalesced microtask flush that re-checks each node, so a `tracked()` re-render's transient unlink/re-link never fires (kills the per-render timer thrash and its scheduler race). Drop the `onObserved` hook and its `link` wrapper entirely — the hot path now calls `baseLink` directly, so observation adds zero per-`link` overhead. - Kernel: `getObservationNode` dedupes frozen targets via a WeakMap fallback (previously returned a fresh, un-deduped node — observation would silently break). Cover the `flush` error-recovery path with a real test and drop its c8-ignore (only the genuinely-unreachable dirty-recheck fallback remains). - Silo: collapse the per-key gc timers into one coalesced sweep over a pending set; drop the now-unused `onObserved`/`unregister` plumbing; store the liveness node directly. Replace the NUL-delimited `keyOf` with a JSON-encoded tuple — still collision-safe, but plain text so finder.ts stays diffable. - Changesets: stop editing #82's silo changeset; add a separate additive #83 changeset instead. Update the kernel changeset to match the onObserved-free API and the coalescing semantics. All five gates green (657 tests). silo finder/store 100%; kernel system.ts 100% lines/statements/functions. https://claude.ai/code/session_018FyYY6YYb1CW5LnBfPMgod

The kernel had vendored a 558-line port of alien-signals' operator layer (`system.ts`) solely to expose an `unwatched` hook, which silo used to cancel in-flight fetches when a handle lost its last reactive observer. alien-signals exposes no public seam for "node lost its last subscriber" (the `unwatched` callback is a private closure arg to `createReactiveSystem`), so the only way to get it was to own the operator layer — a permanent maintenance tax that must be re-ported on every upgrade. Remove the fork entirely and rely on alien-signals' published primitives: - kernel: delete `system.ts`; import `signal`/`computed`/`effect`, `getActiveSub`/`setActiveSub`, `startBatch`/`endBatch` directly from `alien-signals` (the 2.0.7 -> 3.2.1 upgrade stays). Drop the observation primitives (`onObservationChange`/`getObservationNode`/`trackNode`/ `isObserved`/`createObservationNode`/`$OBSERVE`) and their tests. - silo: drop auto-cancellation. `find`/`findQuery` are pure reactive reads; remove the liveness-node observation, gc sweep, and fiber-interrupt path from the finder, the `gcTimeMs` config, and the `Abort` handle event. Adapters still receive `{ signal }`, which now aborts only on adapter-Effect interruption (e.g. a per-model `timeout`). The Effect migration, typed errors, statechart handle, and per-model retry/timeout are unchanged. Docs/changesets updated to match. https://claude.ai/code/session_012E1i6gcCAVc5Qtkt9oxtrG

It documented the observation-primitive approach that was abandoned with the alien-signals fork; keeping it risks tempting a rebuild of the fork. https://claude.ai/code/session_012E1i6gcCAVc5Qtkt9oxtrG

The Promise-adapter branch of create-query's Effect boundary rejected with a pre-built AdapterError was uncovered (codecov patch miss). Add a test asserting such an error is passed through untouched rather than re-wrapped. https://claude.ai/code/session_012E1i6gcCAVc5Qtkt9oxtrG

Address two structural findings from the code-quality review: - Extract the Promise→AdapterError boundary (`coerceAdapter`) into `@supergrain/silo`'s errors module and reuse it from both the silo finder (`toAdapterEffect`, which now only layers the AbortController wiring on top) and `@supergrain/queries`' create-query. The "Promise wrapped, Effect as-is, existing AdapterError passed through" rule now lives in exactly one place. - Collapse the repeated get-or-create-handle dance in `store.ts` into a `getOrCreateHandle` helper (used by find/findQuery/insertDocument/ insertQueryResult) and fold the duplicated idle-check + fetch-trigger of `find`/`findQuery` into a shared `findOrFetch`. No behavior change; all changed files remain at 100% coverage. https://claude.ai/code/session_012E1i6gcCAVc5Qtkt9oxtrG

Address Copilot review on #85 — leftover docs from before auto-cancellation was dropped and the handle was reshaped: - DocumentAdapter / QueryAdapter JSDoc: `ctx.signal` aborts on adapter-Effect interruption (e.g. a timeout), not "when every subscriber has gone away". - react useDocument comment: drop the reference to a non-existent `subscribeDocument` API; note that an in-flight fetch isn't cancelled on unmount. - README Suspense section: `data`/`fetch` → `value`/`isFetching`/`error`/`status`; fix the missing non-null assertion in `use(user.promise!)`. Docs only; no behavior change. https://claude.ai/code/session_012E1i6gcCAVc5Qtkt9oxtrG

- queries.ts QueryHandle JSDoc: `data`/`fetch` regions → the flat `value`/`error`/`isFetching`/`fetchedAt` fields (matches the reshaped handle). - Add a changeset for @supergrain/queries (published): the Promise-first `QueryAdapter.fetch` boundary and `AdapterError`-typed `Query.error`. https://claude.ai/code/session_018CvsdvgjWHvFe8jTxMKMam

The handle status dropped the "IDLE" state — it folded into "pending", with "has a fetch started" now on the orthogonal isFetching axis. Spell this out so consumers who branched on status === "IDLE" know to rewrite it as status === "pending" && !isFetching (TS flags it, but the changeset note "string literals are now lowercase" was misleading for IDLE, which has no lowercase counterpart). - changeset: add explicit IDLE migration + note no capability is lost - README: clarify the "idle handle" reads status "pending"/isFetching false in both the hooks list and the handle reference section https://claude.ai/code/session_016WEDqKFVejFUL6uBZYkyqZ

codecov · 2026-06-03T14:12:37Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (a8a0c30) to head (ffb6f0e).

Additional details and impacted files

@@            Coverage Diff            @@
##              main       #87   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           31        34    +3     
  Lines         1312      1361   +49     
  Branches       256       291   +35     
=========================================
+ Hits          1312      1361   +49

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copilot

Pull request overview

Rebuilds @supergrain/silo’s async/batching layer on an internal Effect runtime and remodels document/query handles into a flat status-discriminated union (value/error/isFetching/fetchedAt/promise). The PR also upgrades the reactive core (alien-signals 3.x) and aligns @supergrain/queries with silo’s “Promise-first (Effect optional)” adapter boundary.

Changes:

Introduces Effect-powered batching/timeout/retry in the silo Finder plus typed error classes (AdapterError / NotFoundError / ProcessorError).
Replaces handle shape (data, isPending, hasData, "IDLE"|"PENDING"|…) with flat orthogonal fields and lowercase statuses.
Upgrades alien-signals to 3.2.1 and updates kernel/react integration to new cleanup semantics + renamed subscriber APIs.

Reviewed changes

Copilot reviewed 50 out of 51 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
README.md	Updates top-level usage docs/examples to the new handle surface and Effect-based adapter examples.
pnpm-lock.yaml	Locks `alien-signals@3.2.1` and adds `effect@3.21.2` (plus transitive updates).
packages/silo/tests/types.test-d.ts	Pins updated public types (handle fields, `SiloError`, `HandleStatus`, Effect adapter return type).
packages/silo/tests/transitions.test.ts	Adds unit tests for the new handle statechart reducer (`applyEvent`).
packages/silo/tests/store.test.ts	Updates store behavior tests to the new handle shape and typed errors.
packages/silo/tests/resilience.test.ts	Adds tests for per-model/query `retry` (Schedule) and `timeout` (Duration) behavior.
packages/silo/tests/react/json-api.test.tsx	Updates React JSON-API relationship hook tests to `value`/`error`/`isFetching`; fixes tracked seed loops.
packages/silo/tests/react/index.test.tsx	Updates core React hook tests to new handle semantics; avoids tracked write-only seeds.
packages/silo/tests/queries.test.ts	Updates query store tests and adds param-key-order identity assertion.
packages/silo/tests/property.test.ts	Updates property-based tests to use Effect-wrapped adapters.
packages/silo/tests/finder.test.ts	Updates Finder contract tests and adds Promise-vs-Effect adapter boundary coverage.
packages/silo/tests/example-app.ts	Updates example adapters to Effect.tryPromise + typed `AdapterError`.
packages/silo/tests/errors.test.ts	Adds tests pinning the new public error classes and message shape.
packages/silo/tests/adapters.test.ts	Updates adapter tests to read `handle.value` instead of `handle.data`.
packages/silo/src/transitions.ts	Introduces the handle statechart (`HandleEvent`, `applyEvent`, promise lifecycle plumbing).
packages/silo/src/store.ts	Updates public types (new handle union), adapter signature (Promise \| Effect + AbortSignal ctx), and store operations to use transitions.
packages/silo/src/react/json-api.ts	Updates JSON-API hook docs/examples to `value`-based reads.
packages/silo/src/react/index.ts	Clarifies hooks are pure reactive reads (no unmount cancellation) in comments.
packages/silo/src/queries.ts	Updates QueryAdapter/QueryHandle types to match the new handle union and adapter boundary.
packages/silo/src/index.ts	Exports `HandleStatus` and new error types from the package root.
packages/silo/src/finder.ts	Replaces timer batching with Effect sleep + fibers; adds timeout/retry wrapping and AbortSignal interruption wiring.
packages/silo/src/errors.ts	Adds `AdapterError`/`NotFoundError`/`ProcessorError` and the shared `coerceAdapter` boundary.
packages/silo/README.md	Updates package-level documentation and migration guidance to new handle/adapters/errors/resilience features.
packages/silo/package.json	Adds Effect dev + peer dependency metadata.
packages/queries/tests/create-query.test.ts	Updates queries tests to allow Effect adapters and typed AdapterError behavior.
packages/queries/src/types.ts	Updates QueryAdapter typing to Promise \| Effect and introduces a `QueryEnvelope` type.
packages/queries/src/create-query.ts	Routes adapters through `coerceAdapter` and runs them on Effect runtime via `Either`.
packages/queries/package.json	Adds Effect dev + peer dependency metadata.
packages/mill/tests/operators.test.ts	Updates effect subscriptions to avoid alien-signals 3.x cleanup return-value pitfalls (`void` reads).
packages/mill/package.json	Bumps `alien-signals` dependency to `^3.2.1`.
packages/kernel/vite.config.ts	Adds `alien-signals/system` to build output externals.
packages/kernel/tests/read/tracking-isolation.test.ts	Switches from `getCurrentSub/setCurrentSub` to `getActiveSub/setActiveSub`.
packages/kernel/tests/core/contracts.test.ts	Updates contract assertions for internal subscriber API rename.
packages/kernel/tests/core/collections.test.ts	Updates effect callbacks to avoid returning non-cleanup values (`void` reads).
packages/kernel/src/read.ts	Migrates tracking checks to `getActiveSub`.
packages/kernel/src/react/use-signal-effect.ts	Extends hook signature to support cleanup return values (alien-signals 3.x semantics).
packages/kernel/src/react/tracked.ts	Switches `ReactiveNode` import path and active subscriber getters/setters.
packages/kernel/src/react/for.ts	Updates subscriber untracking logic to `getActiveSub/setActiveSub`.
packages/kernel/src/internal.ts	Re-exports alien-signals 3.x primitives (`getActiveSub/setActiveSub`).
packages/kernel/src/index.ts	Updates docs and adds a note about alien-signals 3.x cleanup semantics.
packages/kernel/src/collections.ts	Migrates collection tracking to `getActiveSub`.
packages/kernel/package.json	Bumps `alien-signals` dependency to `^3.2.1`.
packages/kernel/benchmarks/additional.bench.ts	Updates benchmark effect callbacks to avoid cleanup return-value issues (`void` reads).
packages/kernel/ARCHITECTURE.md	Updates internal API names and tracking docs for alien-signals 3.x.
packages/js-krauset/vite.config.ts	Fixes alias ordering and adds `@supergrain/kernel/internal` alias.
packages/js-krauset/analyze-gap.ts	Updates text references to the new subscriber API names.
packages/husk/src/resource.ts	Migrates subscriber untracking to `getActiveSub/setActiveSub` and uses kernel’s `effect` export.
package.json	Bumps workspace `alien-signals` dependency to `^3.2.1`.
.changeset/silo-effect-migration.md	Adds migration guide for new Effect engine + handle surface changes (major bump).
.changeset/queries-adapter-boundary.md	Documents queries adapter boundary alignment (minor bump).
.changeset/alien-signals-3.md	Documents alien-signals 3.x upgrade and cleanup semantics (kernel major bump).

Files not reviewed (1)

pnpm-lock.yaml: Language not supported

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

- Make adapter context `signal` optional (`{ signal?: AbortSignal }`) on DocumentAdapter/QueryAdapter so `find(ids, { signal } = {})` typechecks. - silo README: correct stale `data`/`fetch` region reference to `handle.value`/`handle.isFetching`. - queries changeset: soften the typed-error claim to match the `Error | undefined` surface (AdapterError on the normal failure path). - top-level README: note adapters are Promise-first (not Effect-only) and carry failing query params into `AdapterError.keys`.

/simplify cleanups on the new transitions.ts statechart: - Extract `clearResolvers(handle)` for the resolve/reject = undefined pair repeated across the Insert/Settled/Failed/Reset promise transitions. - Move the unhandled-rejection suppression (`promise.catch(() => {})`) into `withResolvers` so it can't be forgotten at a call site (matches husk's `deferred` pattern), and drop the now-redundant caller-side suppression. Behavior-preserving; all five gates green.

Make @supergrain/queries use the same infrastructure as the rest of the store so a query fetch doesn't feel like a different package. silo: - Extract `runAdapter` (+ `AdapterRunOptions`) — the single engine entrypoint that turns one adapter call into a typed, resilient, abortable Effect (Promise→AdapterError boundary, per-attempt AbortController, retry, timeout). Export from the root. The finder now uses it, dropping its private `toAdapterEffect` + `adapterEffect` duplicates. - Harden `stableStringify`: encode `Date` (was colliding to `{}`) and `bigint`, and make it total (no longer returns the value `undefined`). queries: - `createQuery` runs every fetch through `runAdapter` on an interruptible fiber. Resilience config now matches `ModelConfig`: `retry?: Schedule` + `timeout?: Duration` replace the bespoke `backoff` loop. No built-in auto-retry (a failure settles `error` immediately unless you opt into `retry`), matching a silo document fetch. - `QueryAdapter.fetch(id, { offset, limit, signal })` gains an AbortSignal; `Query.error` is now typed `SiloError | undefined`. - Single-flight: a new refetch/fetchNextPage/destroy interrupts the in-flight fetch (its adapter signal aborts). Remove `fibonacciBackoff`.

- Extract `settleChunk` so the document and query drains share the single run→commit-on-success / fail-chunk-on-AdapterError pipeline; the only difference is which bucket/keys to fail and how to commit. - Add an optional `DocumentStoreConfig.onError` sink — called with the typed SiloError + failing type/keys whenever a fetch settles into a failure (AdapterError / NotFoundError / ProcessorError). A throwing sink is isolated so observability can't break the engine. - Tests: per-attempt fresh/non-aborted AbortSignal under retry; onError fires for adapter failure and NotFound and survives a throwing sink; query-key stability (distinct Date params don't collide, key-order independent).

…evel A document `find` and a `createQuery` fetch now retry identically by default. - Add `defaultRetry` (fibonacci 1s–60s, retrying until success), exported from silo, applied to every fetch that doesn't set its own retry. - Add store-wide `DocumentStoreConfig.retry` / `timeout` defaults; precedence is per-model/per-query → store-wide → built-in default. - Expose the resolved `store.defaults` ({ retry, timeout }); `createQuery` inherits them so a query fetch retries like a document fetch unless the call overrides them. - Tests: default-retry wiring + identity, fibonacci recovery of a transient failure, createQuery inheritance + per-query override. Test fixtures pin `retry: Schedule.recurs(0)` so failure assertions stay deterministic.

Harden the shared retry engine and remove the store.defaults leak. - Failures visible while retrying: handles carry failureCount/lastError and onError fires per failed attempt (not only on give-up), so an infinite default retry no longer hides an outage behind a silent spinner. - Jittered default backoff (0.8-1.2x) to avoid synchronized retry storms. - AdapterError.retryable gates the retry schedule; deterministic failures (e.g. 4xx) fail fast instead of looping. - New deadline knob caps all attempts together (vs per-attempt timeout). - store.defaults -> store.resolveAdapterOptions(perCall), the single merge point used by the finder and @supergrain/queries.

…e failure sink - Add a config-level retryable?: (error) => boolean classifier (model / query / store / createQuery) so Promise-first adapters, which reject rather than construct an AdapterError, can veto retries on deterministic failures (e.g. a 4xx) by inspecting error.cause. The error's own retryable: false stays a hard veto over the predicate. - Isolate a throwing onError/onFailure sink in runAdapter (per-attempt tapError and the deadline onTimeout) in try/catch, matching the finder's existing contract that observability can't break the engine.

…richer telemetry - isolateFailures (model/query/store): bisect a terminally-failed batch to isolate the offending id so its healthy batch-mates still load, instead of failing the whole chunk together. Sub-fetches run once (recurs(0)). - maxConcurrency (store): cap how many adapter.find chunks fan out per drain; defaults to unbounded (prior behavior). - AdapterError.reason ("adapter" | "timeout" | "deadline"): branch on a stable tag instead of regex-matching cause.message. - Enrich onError ctx with { attempt, retryable } (1-based attempt number and whether the failure passed the retryable check); additive to { type, keys }. - Harden stableStringify: encode NaN/+/-Infinity distinctly (JSON.stringify collapses them to null, colliding on one cache slot) and throw a clear error on cyclic params instead of overflowing the stack.

…e coverage Codecov flagged uncovered lines from the resilience additions: - query-surface isolateFailures bisect (finder) - stableStringify bigint / nested-undefined / symbol / function params - runAdapter without an onFailure sink (and without retry) - Query.failureCount / Query.lastError getters

… path Supersession (a newer fetch or destroy()) aborts the run's controller, which interrupts the fiber — so the success/error/onFailure channel never runs after ownership is lost. The owned() guards there were therefore always true (the coverage data confirmed it: only the `ensuring` finalizer's owned() branch, which runs on interruption, was ever exercised). Remove the three redundant guards; keep owned() solely on the finalizer where it actually fires. Behavior identical, and restores 100% coverage.

claude added 23 commits June 1, 2026 20:42

Remove stale effect-handle design proposal

101dede

It documented the observation-primitive approach that was abandoned with the alien-signals fork; keeping it risks tempting a rebuild of the fork. https://claude.ai/code/session_012E1i6gcCAVc5Qtkt9oxtrG

scottmessinger marked this pull request as ready for review June 3, 2026 14:16

Copilot AI review requested due to automatic review settings June 3, 2026 14:16

Copilot started reviewing on behalf of scottmessinger June 3, 2026 14:16 View session

Copilot AI reviewed Jun 3, 2026

View reviewed changes

Comment thread README.md Outdated

Comment thread packages/silo/README.md Outdated

Comment thread packages/silo/src/store.ts

Comment thread packages/silo/src/queries.ts

Comment thread .changeset/queries-adapter-boundary.md Outdated

Comment thread README.md Outdated

claude added 2 commits June 4, 2026 02:52

claude added 2 commits June 4, 2026 13:18

scottmessinger changed the title ~~feat(silo)!: Effect-powered network layer + status-discriminated handle~~ feat(silo,queries)!: Effect engine + status-discriminated handle, shared by queries Jun 4, 2026

claude added 4 commits June 4, 2026 13:58

scottmessinger mentioned this pull request Jun 4, 2026

Proposal: opt-in read-after-write retry, staleness, and memory bounding for @supergrain/silo #88

Open

claude added 2 commits June 4, 2026 18:43

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(silo,queries)!: Effect engine + status-discriminated handle, shared by queries#87

feat(silo,queries)!: Effect engine + status-discriminated handle, shared by queries#87
scottmessinger wants to merge 33 commits into
mainfrom
claude/quirky-bardeen-gLpRQ

scottmessinger commented Jun 3, 2026 •

edited

Loading

Uh oh!

codecov Bot commented Jun 3, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

scottmessinger commented Jun 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

The engine (@supergrain/silo)

Status-discriminated handle (breaking — API shape, not capability)

@supergrain/queries runs on the same engine (breaking)

On cancellation

Misc

Checks

Uh oh!

codecov Bot commented Jun 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

scottmessinger commented Jun 3, 2026 •

edited

Loading

The engine (`@supergrain/silo`)

`@supergrain/queries` runs on the same engine (breaking)

codecov Bot commented Jun 3, 2026 •

edited

Loading