Repository navigation
#506 Stores: one link per Store, connections bind to providers only #509
Description
Sub-issue of #506. Owner direction 2026-10-30: each local SQLite Store links to at most one remote file, and a remote file is linked by at most one Store of the home. A connection (MinIO, Cloudflare R2; many of each) binds to the provider only, never to a Store.
Spec: core/docs/issues/506/links.md on branch issue506-store-backends-remote (PR #507). The Connect design that uses it lives in Connect's own repository (docs/ui/storage-links.md).
Why
The runtime's stores.json keeps a list of copies per Store (one per remote), while the native remote-state/<store> keeps one last transfer. They disagree:
team-exampleswas pushed tolocal-minio, then tor2. A new push tolocal-miniois refused as "the remote copy moved" although MinIO holds exactly what was pushed: the check compares against R2's record.- A clone is checked at a path named after the clone, where nothing was pushed, so every clone reads "missing".
- The object path is the Store's name, so a Store cannot track a file of another name.
Model
- Connection: provider, endpoint or account, bucket, prefix, credential reference. No Store field.
- SQLite Store:
link: { remote, file, linked, last } | nullreplacesremoteandcopies.filenames the object<prefix>/<file>/store.sqlite. - One to one per home, checked under the registry write lock.
- PostgreSQL Stores: unchanged, never linked.
Calls
| Call | Does |
|---|---|
store.start_publish(store, { remote, file }) |
an unlinked Store to a new file, linked; Exists if the file is there or linked, Linked if the Store already is |
store.start_push(store, { force }) |
to its link; NotLinked; Conflict against the link's own record |
store.start_pull(store, { confirm }) |
from its link; NotLinked |
store.start_send(store, { remote, file }) |
a copy to a new, unlinked file; the Store's link does not change |
store.start_load(remote, { file, store }) |
a new local Store linked to the file; Linked if already linked; replaces start_clone |
store.unlink(store, { confirm }) |
drops the link; the file stays |
store.check(store) |
HEAD of the linked file |
remote.list_stores(remote) |
each file gains linkedBy |
remote.remove(name) |
InUse while a Store links to it |
Removed: start_clone, the remote argument of push and pull, copies, the Store-level remote field. The native record gains file and ms times.
Phases
- 1. Engine and runtime: the calls above, native
fileinremote-state, tests (one-to-one, refusals, theteam-examplesconflict case), README §6 updated. Bundle withlayerfs-node/postgresql,layerfs-daemon/postgresql,layerfs-node/tls. - 2. Connect (private): host view and routes on one link; Storage page with one Linked-to cell per Store, a Connections section with each bucket's files (Load / Open), Publish, Send a copy, Load, Unlink and push-conflict dialogs; Clone and copy lists removed.
- 3. Settle today's data (after 1 and 2 work): choose each Store's link from evidence (native record plus remote sha256), write the new records, give every link a baseline push (conflict-checked, never forced), fix the running Connect host's empty
heads, and post the before/after table here.
Done when
- Every SQLite Store is linked and in sync, or local only.
- No link reads unknown or missing, and no file is linked twice.
- Every file in each connection reads linked or not loaded.
- Connect's Storage page shows exactly that.
- Each commit records production LOC, and every test command is bounded at 120 s.
Activity
docs(506): one link per Store, implementation spec (#509)
feat(runtime,binding): one link per SQLite Store (#509)
yifanxuaaa commentedon Oct 29, 2026
Phase 3: today's data settled (
~/.layerfs-connect/home, 2026-10-30)Engine and runtime: f65efb4 on PR #507. Connect's host and Storage page are rebuilt for one link per Store (Connect's own repository, not committed there). Connect now runs the worker bundle
build-linux-a1b3c1d6…(source f65efb4, release,layerfs-node/postgresql,layerfs-daemon/postgresql,layerfs-node/tls).Evidence (step 1): the HEAD sha256 of every file, read through
remote.list_stores:r2 / r2-examplesb440dfca…r2-examples's record (push);r2-examples-clonepulled itr2 / servicesc5497495…services's record (push)r2 / team-examplesdc2903cb…team-examples's record (push);team-examples-r2pulled itlocal-minio / services2be72dc7…services' older MinIO push (no record)local-minio / team-examplesdc2903cb…team-examples' older MinIO push;team-examples-copy's record (pull)Before and after (after = Connect's
/api/storagefollowing a check):team-exampleslocal-minioandr2; recordr2r2 / team-examples, in syncserviceslocal-minioandr2; recordr2r2 / services, in syncr2-examplesr2r2 / r2-examples, in syncteam-examples-copylocal-minio / team-examples, read missinglocal-minio / team-examples(no other Store claims it), in syncteam-examples-r2r2 / team-examples, read missingteam-examplesclaims the file)r2-examples-cloner2 / r2-examples, read missingr2-examplesclaims the file)dsa,heavy-check,playgroundconnect-test-13321,-73419,-84622,-9697team-examples-pg,services-pgr2 / r2-examples,r2 / services,r2 / team-exampleslocal-minio / team-examplesteam-examples-copylocal-minio / servicesNo link reads unknown or missing, and no file is linked twice.
How (steps 3–4): the four links were written with the runtime's own
registry.linkunder the registry lock (stores.json is now version 2; the version 1 file stays asstores.v1.json). Each native record was rewritten withfileand ms times. The six records from before #509 are kept inhome/remote-state.v1/; the two clones' records were moved there, not deleted. Each linked Store was then pushed through Connect with the conflict check (never forced): all four were accepted, with no refusals. Each pushed sha256 equals the one before, so the local Stores had not changed; the push recordedheads, and the next check read in sync from equal heads on both sides.Step 5, the host's empty
heads: not reproduced, cause not found. After the worker swap, the running Connect host and a fresh process both read heads for all 13 SQLite Stores, including with every Store held open by the worker. The old host process was gone before this work started, so it could not be inspected. What is known:runtime.store.statwas already in the old bundle (00665aa721).endpoint.jsonon every call, so a stale endpoint is not the cause.api/runtime/src/store/calls.jslocals()turns any stat failure intoheads: nullfor every Store, with no trace. That is why the cause left no evidence.Making that failure visible is the next step if the gap returns.
Not done here: no screenshot of the Storage page (checked through the same host API the page renders);
stores_real.mjsandstorage_real.mjsE2E not run. The first R2 check after the worker swap, and one R2 listing from a fresh process, failed with "connection closed before the reply" (Unknown, not resent); the next calls succeeded.