[Provider] Viability & upgrade resilience: version guard, on-chain protocol logic, adaptability to runtime changes

[bkontur]

Motivation

A storage provider is meant to be a long-running, high-uptime service. Today the provider binary is tightly coupled to the exact shape of the on-chain runtime — storage layouts, runtime-API signatures, event/extrinsic encodings. That means a runtime upgrade which changes any of those shapes can force a provider rebuild + coordinated redeploy, i.e. downtime, and puts the whole provider fleet on the runtime's release cadence.

Goal: the fewest possible provider-binary upgrades and the least downtime. The provider should adapt to most on-chain changes (new runtime-API versions, evolved storage shapes, tweaked protocol parameters) without a redeploy, and when a redeploy is genuinely required, it should be safe, detectable in advance, and ideally automatic.

This is a topic-scoping issue — we're capturing the problem space and the directions worth investigating, not committing to a design yet.

Problem statement

• The provider decodes on-chain storage and calls runtime APIs directly; changing either on-chain side can break the running binary.
• The typed-bindings direction (Migration: subxt::dynamic → typed storage_subxt bindings #243, storage-subxt) improves safety but tightens the shape coupling: a shape change means regenerating bindings and shipping a new binary. Dynamic decoding is more tolerant but loses type safety. We need a story that gets both.
• There is no explicit version handshake between provider and chain: nothing tells a provider "you are too old / too new for this runtime" before it starts making bad calls.
• A runtime upgrade today has no defined provider-side reaction beyond "operators notice and upgrade."

Directions to investigate (later)

1. Version guard / compatibility negotiation. Define a provider-facing protocol version (distinct from the runtime spec_version). Expose it on-chain; have the provider declare the range it supports and check compatibility on startup and when it observes a runtime upgrade. On mismatch: refuse to start / enter a degraded read-only mode / warn loudly — never silently mis-decode.

2. Stable, versioned provider interface. Insulate the provider from internal runtime refactors behind a curated, versioned RuntimeApi surface (builds on Expose interfaces via a separate RuntimeApi #237) so cosmetic runtime changes don't ripple into the binary.

3. Decoupling from storage shapes. Versioned / encapsulated on-chain structures (Encapsulate the Commitment structure #250 Commitment) and forward/backward-compatible encodings so a provider can read both the old and new shape across an upgrade window. Weigh against the typed-bindings work (Migration: subxt::dynamic → typed storage_subxt bindings #243) and storage-iteration hardening (Clarify entry-count logging and harden manual key-byte parsing in subxt storage iteration #174).

4. On-chain protocol logic ("mini-runtime"). Put a slice of protocol logic and/or parameters on-chain — from simple version-tagged config up to a small downloadable blob/wasm the provider fetches and hot-applies when its on-chain version changes. Behavior can then evolve without a binary upgrade; the provider just reads → downloads-on-change → applies. Needs careful thought on trust, determinism, and the sandbox/execution model.

5. Automatic / low-downtime binary upgrades. For changes that do require a new binary: a self-update path gated by the version guard, plus graceful/rolling restart and config hot-reload so a provider swaps versions without dropping agreements or challenge responsiveness.

6. Chain-connection resilience across upgrades. How the provider stays connected and reacts to a runtime upgrade at runtime — ties into the light-client-vs-RPC investigation ([Investigation] Provider chain connection: smoldot light client vs RPC node + event-driven coordinators #222) and event-driven coordinators (Event-driven coordinators #227).

7. Observability of version drift. Surface provider↔chain version compatibility as a first-class metric/alert so incompatibility is caught before it causes missed checkpoints or challenges (ties into telemetry [Deployment] Design telemetry and metrics for on-chain and off-chain components #214).

Out of scope (for now)

Concrete design, the execution/sandbox model for any on-chain logic, and the exact versioning scheme — all deferred to follow-up investigation.

Relevant / related issues

• Expose interfaces via a separate RuntimeApi #237 — Expose interfaces via a separate RuntimeApi (stable versioned surface)
• Migration: subxt::dynamic → typed storage_subxt bindings #243 — Migration: subxt::dynamic → typed storage_subxt bindings (shape-coupling tradeoff)
• Encapsulate the Commitment structure #250 — Encapsulate the Commitment structure (versionable storage shapes)
• Clarify entry-count logging and harden manual key-byte parsing in subxt storage iteration #174 — Harden subxt storage iteration / key parsing (robust decoding)
• [Investigation] Provider chain connection: smoldot light client vs RPC node + event-driven coordinators #222 — Provider chain connection: smoldot light client vs RPC + event-driven coordinators
• Event-driven coordinators #227 — Event-driven coordinators (reacting to on-chain changes)
• [Deployment] Design telemetry and metrics for on-chain and off-chain components #214 — Design telemetry and metrics for on-chain and off-chain components
• Repo structure: split provider-node, break the storage-client dependency, tidy crate layout #178 — Repo structure / storage-subxt home (where metadata bindings live)
• Production readiness: Web3 principles compliance and Polkadot SDK standards #44 — Production readiness: Web3 principles + Polkadot SDK standards
• Deploy MVP to Paseo #48 — Deploy MVP to Paseo (real upgrade cadence to design against)