Prepare Anvil 0.3.0 release

[zcourts]

Summary

This PR prepares the Anvil 0.3.0 release. It packages the CoreStore-centred storage rewrite, the hardened public/admin API split, the Rust client release surface, the Docker-first release path, and the documentation needed for operators and application developers to adopt this version with realistic expectations.

The main point of the release is architectural stability. Metadata now has one durable centre in CoreMeta, CoreMeta uses RocksDB column families as the local metadata engine, bounded tiny payloads can be stored through the inline payload column family, and larger durable bytes go through the CoreStore byte pipeline as erasure-coded shard data. Feature-specific formats still exist where they make reads faster or more natural, but they are writer outputs over CoreStore rather than independent durable stores.

I am treating this as a minor release because the public story is materially broader than the previous release line: the crate version moves to 0.3.0, the documentation adds a first-class Architecture book, the release blog post documents the storage model and practical performance progression, and the README now describes the current server/client/docs surfaces rather than the older object-store-only framing.

Current release-gate delta

This update also fixes a bootstrap/admin-plane mismatch found while integrating Anvil from dependent projects: generated system-admin credentials are now created under Anvil's reserved system storage tenant 0, so the JWT minted from that credential is accepted by the admin listener and still checked through the system-realm Zanzibar relation model. The regression test proves the credential JSON, persisted app row, minted token, and admin relation check all use the reserved system tenant.

The release workflow now builds both linux/amd64 and linux/arm64 images using the same scripts/build-image.sh path. CI uploads one artifact per architecture, Docker integration tests run against the amd64 artifact on GitHub's native runner, the arm64 artifact is build/smoke checked, and the release job publishes architecture-specific tags before creating the public multi-architecture GHCR manifest. The old build-test-image-fast.sh name has been removed in favour of the production-shaped build-image.sh.

The shared gate script is split into explicit groups (static, rust, server-core, docker-auth, docker-storage, docker-index, docker-mesh) so CI and release can run independent gates in parallel while local maintainers can still run the full sequence with ./scripts/release-gates.sh. Each step emits timing lines and uses ANVIL_GATE_STEP_TIMEOUT_SECONDS where GNU timeout is available, so a stuck gate reports the exact step instead of silently consuming a full release run.

Release outputs

This PR prepares these release artefacts:

• Docker image: ghcr.io/worka-ai/anvil:v0.3.0
• Rust client crate: anvil-storage = "0.3.0"
• Server and CLI binaries in the image: anvil, anvil-admin, and the server binary
• Fission static documentation site from documentation/
• GitHub release notes rendered from the release blog post
• README and operator release checklist aligned to v0.3.0

The server crate remains a server/image artefact rather than a crates.io API surface. The Rust client crate is the published application integration surface for this release.

Why this release matters

Anvil has grown beyond plain object storage. A stored object can now be the source for listings, metadata filters, typed JSON queries, full-text search, vector search, hybrid ranking, watches, public access, relationship authorisation, PersonalDB witness state, source artefacts, and gateway protocols. That only works safely if those systems share one durable model and one security model.

This release makes that model explicit:

• CoreMeta is the metadata plane.
• RocksDB is the local CoreMeta engine.
• The inline payload path is bounded and documented.
• The byte pipeline is the durable large-payload path.
• Index segment bodies follow the same writer-output storage rules as object bodies and stream payloads.
• S3 is a gateway, not the system model.
• The admin plane is a private network control plane, not a local storage writer.
• The public CLI is anvil; the admin CLI is anvil-admin.
• Documentation now explains the architecture rather than only listing commands.

The practical result is that dependent projects can integrate against Anvil without expecting a future patch to move data out of feature-specific JSON files, SQLite files, or ad hoc index directories. There are still staged improvements, but they build on this storage shape rather than replacing it.

Architecture status

CoreMeta and RocksDB

CoreMeta stores the durable metadata rows. The current RocksDB column families cover metadata versioning, root cache records, transactions, object heads, object versions, inline payloads, stream heads, stream records, index definitions, index rows, boundaries, authorisation state, PersonalDB state, registry/gateway records, mesh records, leases/fences, materialisation state, refcounts, and observability rows.

The important rule is that RocksDB is not the object store. It is the local metadata engine for CoreMeta and a bounded tiny-payload fast path. RocksDB values have a 64 KiB encoded CoreMeta value ceiling, while ordinary raw object payload inline eligibility defaults to 32 KiB. Eligibility is based on the raw accepted bytes and encoded record limit, not on whether RocksDB compression happens to make a large object small on disk.

CoreStore byte pipeline

Payload-like durable bytes go through the byte pipeline. That includes large object bodies, multipart outputs, large append-stream payloads, large index segments, PersonalDB snapshots/pages, source packs, gateway blobs, registry blobs, and future writer outputs that do not fit the inline policy.

The byte pipeline stages bytes, hashes them, applies compression/encryption policy where configured, constructs blocks, erasure-codes the blocks, places shards, waits for shard receipts, and records the resulting manifest/locator through CoreMeta. Root-visible metadata should not point at bytes that have not met the storage-class publish threshold.

Read-after-write and root visibility

Writes are visible after their evidence is complete. For large payloads, that means the shard receipts are sufficient for the storage class, the CoreMeta batch has reached metadata quorum, commit evidence has been persisted, and the relevant root generation advances. For inline writes, the byte plane can be skipped, but CoreMeta quorum and root publication still apply.

Normal object writes are intended to be read-after-write consistent after the finalised write returns. Explicit transactions stage writes until commit. Index reads observe the latest materialised index generation available to the query path, and callers that need freshness evidence can require index catch-up to a watch cursor.

Index and query architecture

Indexes are derived views over source records. An index definition stores selectors, extractors, kind, authorisation mode, and build policy. Index rows store segment metadata and locators. Segment bytes are writer output and follow the same inline-or-byte-pipeline rule as other durable bytes.

The live index families covered by the release are:

• path indexes for prefix/listing-shaped acceleration;
• metadata-filter indexes over object user metadata;
• typed JSON indexes over object bodies, object metadata, and append records;
• full-text indexes using tokenised postings and BM25 scoring;
• vector indexes using HNSW graph data;
• hybrid search combining full-text and vector candidates;
• PersonalDB row/projection metadata indexes;
• git-source/source-pack index records.

The shared planner path narrows candidates before payload/stored-field fetch where the index shape supports it, intersects boundary and authorisation candidates where available, binds page tokens to query scope and generation evidence, and still keeps final visibility checks as the last safety gate.

Full text, vectors, and hybrid search

Full-text search currently exposes tokenised search with BM25 scoring and phrase mode. The storage architecture supports richer boolean grammar later without changing the segment storage model.

Vector search uses HNSW segments with configured dimensions, metric, modality, and embedding provenance. The documentation distinguishes caller-supplied vectors, configured embedding providers, and deterministic development providers.

Hybrid search currently uses a fixed fusion recipe over full-text and vector candidates. That keeps the API useful now while leaving room for configurable fusion semantics in a later release.

Streams, watches, and mesh transport

Append streams are ordered application-visible record streams. They are not Anvil's internal metadata WAL. Stream heads and stream record indexes are CoreMeta rows; payloads follow inline-or-byte-pipeline storage rules.

Prefix watches are bucket/prefix/cursor streams for committed object changes. They are intentionally not a general query language. Rich metadata, typed, full-text, or vector predicates belong in index queries.

CoreMeta metadata quorum traffic uses persistent bidirectional gRPC streams. The stream tracks request ids, pending responses, timeouts, closure, eviction, and reconnect. TCP_NODELAY is enabled on server listeners. Shard reads stream response chunks; shard write transport can continue to improve without changing object manifests or RocksDB layout.

Public and admin planes

The public plane is tenant-facing. It supports buckets, objects, object links, public policy, app credentials, relationship authorisation, indexes, watches, append streams, task leases, PersonalDB, tenant diagnostics, tenant repair, and gateways.

The admin plane is private. It supports tenant creation, first credential handover, system policy grants, topology lifecycle, routing repair, secret-envelope rotation, global diagnostics, global repair, and administrative audit. The admin CLI talks to the admin API; it does not write directly into the server storage directory.

Practical status report

This release is ready to be consumed as the new Anvil baseline for dependent projects that need Docker deployment, the Rust client, object operations, S3-compatible object workflows, relationship-aware operations, indexes, watches, PersonalDB witness support, and the admin/public CLI split.

The storage layout is the main compatibility point. CoreMeta/RocksDB and the byte pipeline are the release boundary. Future work can improve query expressiveness, gateway coverage, transport performance, and watch ergonomics without requiring a disruptive move away from the storage layout introduced here.

The staged extensions are documented as release context rather than framed as release blockers:

• richer full-text boolean query grammar over the existing postings architecture;
• more configurable hybrid fusion over the existing full-text/vector segment architecture;
• application-level heartbeat signals for long-idle internal streams;
• broader streaming optimisation for blob shard writes;
• richer filtered watch APIs where index query is not the better interface.

Performance progression

The following table records the observed progression from the release optimisation work. These are wall-clock measurements from the investigation runs, not universal benchmarks for every deployment. They are included to set realistic expectations about where the release stands in practice.

Write path

Run	Main change	Tenant	App	7 grants	Token	Bucket	PUT 27B	Authz write
Baseline	before optimisation	66.42s	16.02s	19.48s	-	48.70s	32.01s	19.04s
v1	first three optimisations	7.871s	1.781s	5.043s	54.1ms	9.291s	6.976s	5.679s
v2	inline path	7.904s	1.826s	5.107s	1.86ms	9.067s	6.941s	2.426s
v3	authz delta	7.944s	1.841s	2.119s	1.95ms	5.989s	3.691s	2.440s
v4	CoreMeta batching	5.366s	1.333s	1.544s	5.16ms	4.350s	2.814s	1.998s
v5	stream batching	3.914s	816ms	1.296s	2.21ms	3.226s	2.439s	1.656s
v6	RPC instrumentation	3.878s	833ms	1.248s	2.10ms	3.289s	2.691s	1.758s
v7	CoreMeta streaming	1.261s	272ms	373ms	3.88ms	1.462s	1.500s	531ms

Read and query path

Run	GET 27B	Permission check	List authz objects	List objects cold	List objects warm
Baseline	440ms	280-480ms	-	26.26s	1.33s
v1	8.86ms	5.11ms	4.71ms	403ms	401ms
v2	7.07ms	3.70ms	4.28ms	365ms	363ms
v3	6.06ms	6.39ms	7.59ms	24.5ms	24.0ms
v4	4.45ms	5.60ms	5.08ms	25.2ms	21.0ms
v5	4.99ms	4.70ms	4.94ms	21.2ms	18.5ms
v6	10.6ms	5.86ms	5.24ms	27.3ms	20.9ms
v7	9.86ms	5.51ms	7.49ms	21.4ms	19.6ms

The key release signal is that metadata-heavy writes moved from tens of seconds into low-single-digit seconds or better in the observed path, while reads/checks/listing moved into millisecond or tens-of-milliseconds territory. That gives us a deployable baseline while keeping the storage design coherent.

Documentation changes

This PR adds a new top-level Architecture section to the Fission documentation site:

• Architecture Overview
• CoreMeta and Blob Storage Layout
• Indexing and Query Architecture
• Streams, Watches, and Mesh Transport
• Release Architecture Status
• Contributor Architecture Guide

It also updates the home page, README, release blog post, release readiness checklist, and overview/reference pages so readers can move between the Learn, Architecture, Tutorials, Operators, and Reference books.

The purpose of the Architecture book is to make the system understandable to contributors and future coding agents. A contributor should be able to read it and understand where metadata belongs, when bytes go through the byte pipeline, how indexes are stored, why watches are cursor-driven, how mesh replication fits in, and what staged extensions are expected after this release.

Validation in this update

Local validation run for this documentation/version update:

fission site check --project-dir documentation --release
cargo metadata --no-deps --format-version 1

The Fission site check rendered 73 static routes successfully, including the new Architecture section. Cargo metadata reports the Anvil workspace crates at 0.3.0 while keeping the documentation helper crate at its own 0.1.0 package version.

Release CI remains the authority for the Docker image, Rust client publishing gate, and GitHub release creation.

Review focus

Reviewers should focus on these points:

1. The release-facing version and artefact names should consistently say 0.3.0 / v0.3.0.
2. The Architecture book should accurately describe the current storage layout and not imply feature-specific side stores are the source of truth.
3. The release status page should set realistic expectations without undermining the release.
4. The README should point users to Docker-first deployment, the Rust client, the public/admin CLI split, and the new Architecture docs.
5. The release blog post should be detailed enough to become the GitHub release notes source.

Saga API reservation

This PR reserves the saga protocol surface without enabling saga execution. The protobuf package now includes the planned SagaService, saga mutation contexts, and response extension fields, and the Rust client exposes the corresponding raw and high-level types. The server rejects every saga service call and any mutation carrying saga operation context with UNIMPLEMENTED; high-level Rust saga helper methods panic with an explicit reserved API message.