fix(db): Clone read-only databases in memory

[reneleonhardt]

fix(db): Clone read-only databases in memory

🧢 Changes

• Capture a stable database-and-WAL snapshot without opening the source through
  SQLite.
• Let SQLite rebuild derived SHM state in private writable storage, then back
  the result into :memory:.
• Retry bounded source changes and fail closed rather than returning a mixed
  snapshot.

☕️ Reasoning

A WAL database can be readable while its directory cannot create SQLite's
-shm coordination file. A normal read-only open then fails; immutable=1
avoids the sidecar but ignores WAL-only state and can return stale data.

This PR copies matching database and WAL bytes to private storage, lets SQLite
reconstruct SHM there, and backs the result into a disposable in-memory
connection. Source checks before and after backup prevent a concurrent writer
from producing a mixed snapshot. SHM is excluded because it is derived from the
database and WAL.

The change is isolated to the existing but-db observation path. It does not
make mutation commands work with read-only project storage. Relocating project
metadata remains the correct option when writes are intended.

For agentic coding in managed sandboxes, observation can include current WAL
state without granting project-directory writes merely for SQLite coordination.

🧪 Reproduction

cargo test -p but-db read_only_clone_works_when_directory_mode_overstates_writability

Before this fix, the observational open failed because SQLite could not create
source-directory coordination state. An immutable open was not a safe fallback
because it omitted committed WAL state.

✅ Verification

• Ten focused contracts cover physical read-only state, active WAL data,
  unreadable WAL files, discarded clone writes, and bounded concurrent changes.
• The full but-db suite passed: 154 tests, with one pre-existing migration
  stress test ignored.
• Successful and failed paths preserve source database, WAL, and SHM bytes and
  never create a missing source sidecar.
• Clippy, formatting, GitHub Rust/Tauri/Windows/E2E checks, and independent
  read-only review passed.

Developed with carefully directed, manually reviewed AI assistance.

Co-Authored-By: GPT-5.6 Sol codex@openai.com

[Byron]

but-db happens to be a place I know pretty well, even though time has passed and familiarity with details is fading.

With that said, I my intuition is to close this issue as I can only guess that it tries to create administrative files (and technically even the database itself) even when the DB is opened read-only, an abstraction that comes with Rust and the way we use it.

There are situations in containers where everything is read-only, and trying to open a database fails, possibly fatally so this PR tries to make it not do that.

But here we are and I think a lot more will fail later as even read-only operations will sometimes try to makes change to the database (to reconcile data, for instance).

With that said, I'd love to understand the problem this PR is solving.

PS: https://docs.gitbutler.com/development/debugging#per-project-repository-data might help to point the metadata to a place where agents are allowed to write. Maybe this helps.

[reneleonhardt]

This PR does not make mutation commands work with read-only project storage. It
fixes the database-open path used by observational workspace projections.

The failing case is a WAL database whose directory cannot create SQLite's
-shm sidecar. A normal read-only open fails, while immutable=1 ignores
WAL-only state and can return stale data. The PR copies a stable database-and-WAL
snapshot to private writable storage, lets SQLite reconstruct SHM there, and
backs the result into a disposable :memory: connection. Source checks before
and after backup reject mixed snapshots from concurrent writers.

Relocating project metadata is the correct option when writes are intended.
Mutation-capable commands still require writable project storage.

The focused regression is:

cargo test -p but-db --test db \
  handle::read_only_clone_observes_wal_in_physically_read_only_database -- --exact

[Byron]

Thanks. The additional explanation seems to vouch for the solution in this PR while I think it's way to complex and hacky.
Instead, I think relocating the gitbutler storage is the way to go.

Please try that - if it doesn't work, please create an issue that clearly explains the situation and how to reproduce it. Thank you.