diff --git a/docs/sequence_diagrams/README.md b/docs/sequence_diagrams/README.md new file mode 100644 index 00000000..7dd05fa8 --- /dev/null +++ b/docs/sequence_diagrams/README.md @@ -0,0 +1,42 @@ +# Sequence Diagrams + +Visual sequence diagrams for the core protocol flows in Scalable Web3 Storage. + +## Provider Lifecycle + +| Diagram | Description | +|---|---| +| [Provider Registration](./provider-registration.md) | On-chain registration (stake locking, settings), provider node startup sequence, and adding stake. | +| [Provider Deregistration](./provider-deregistration.md) | Two-step exit: announce (cooldown >= ChallengeTimeout), then complete (stake returned). Includes cancellation flow. | + +## S3 Bucket Creation + +| Diagram | Description | +|---|---| +| [S3 Bucket Creation](./s3-bucket-creation.md) | End-to-end flow: provider discovery, off-chain negotiation with term signing, single atomic on-chain extrinsic (Layer 0 bucket + agreement + S3 registry), and UI completion. | + +## S3 Object Read & Write + +| Diagram | Description | +|---|---| +| [S3 Write Flow (putObject)](./s3-write-flow.md) | Client upload to primary provider: auth, chunking (256 KiB), Merkle tree, MMR commit, S3 index update. Includes replica sync (background pull from primaries). | +| [S3 Read Flow (getObject)](./s3-read-flow.md) | Client download: provider URL resolution, auth, DFS tree traversal, chunk reassembly. Includes Rust SDK verified download and multi-provider topology. | + +## Checkpoint Flows + +How on-chain `BucketSnapshot` records are created, making providers liable for stored data. + +| Diagram | Description | +|---|---| +| [Client-Initiated Checkpoint](./checkpoint-client-initiated.md) | Client collects provider signatures via HTTP, verifies consensus, submits on-chain. | +| [Provider-Initiated Checkpoint](./checkpoint-provider-initiated.md) | Autonomous leader election, peer co-signing, grace period fallback, and missed-window penalties. | + +## Challenge Flows + +The dispute mechanism: clients challenge providers to prove they still hold data. + +| Diagram | Description | +|---|---| +| [Challenge Creation](./challenge-creation.md) | How clients create challenges (both `challenge_checkpoint` and `challenge_offchain`). Includes how the client obtains `chunk_index` and `provider_sig`. | +| [Challenge Response](./challenge-response.md) | Provider detects challenge, builds two-layer MMR + Merkle proof, submits on-chain. Includes time-based cost split. | +| [Challenge Timeout & Slashing](./challenge-timeout-slashing.md) | Automatic slashing via `on_finalize` when a provider fails to respond. | diff --git a/docs/sequence_diagrams/challenge-creation.md b/docs/sequence_diagrams/challenge-creation.md new file mode 100644 index 00000000..904b4307 --- /dev/null +++ b/docs/sequence_diagrams/challenge-creation.md @@ -0,0 +1,83 @@ +# Challenge Creation + +The client (challenger) initiates a challenge when a provider fails to serve data. Two entry points converge into the same on-chain challenge. + +```mermaid +sequenceDiagram + autonumber + participant Client as Client (Challenger) + participant Provider as Provider Node (HTTP) + participant Chain as Parachain (Pallet) + + Note over Client,Provider: Earlier: Upload & Commit (client saves metadata) + Client->>Provider: POST /upload (chunked data) + Provider-->>Client: data_root + + Client->>Provider: POST /commit {bucket_id, data_roots} + Provider->>Provider: Append leaves to MMR, sign CommitmentPayload (leaf_count=0) + Provider-->>Client: {mmr_root, start_seq, leaf_indices, provider_signature} + Client->>Client: Save leaf_indices, provider_sig, chunk count per leaf + + Note over Client,Chain: Later: Client spot-checks by downloading a chunk + Client->>Provider: GET /download?bucket_id=X&leaf=3&chunk=7 + Provider-->>Client: (no response / wrong data / timeout) + Client->>Client: Verification failed — initiate challenge + + Note over Client,Chain: Option A: challenge_checkpoint (on-chain snapshot) + Client->>Chain: challenge_checkpoint(bucket_id, provider, leaf_index, chunk_index) + Chain->>Chain: Verify provider bit set in snapshot.primary_signers + Chain->>Chain: Reserve deposit (100 units) from client + Chain->>Chain: Store Challenge at deadline = now + 48h + Chain-->>Client: Event: ChallengeCreated{challenge_id, respond_by} + + Note over Client,Chain: Option B: challenge_offchain (saved provider_sig) + Client->>Chain: challenge_offchain(bucket_id, provider, mmr_root, start_seq, leaf_index, chunk_index, provider_sig) + Chain->>Chain: Verify provider_sig over CommitmentPayload{..., leaf_count=0} + Chain->>Chain: Reserve deposit, store Challenge at deadline + Chain-->>Client: Event: ChallengeCreated{challenge_id, respond_by} +``` + +## How the Client Knows `chunk_index` and `provider_sig` + +| Parameter | Source | +|---|---| +| `chunk_index` | The client chunked the data itself (256 KiB chunks). Any index from `0` to `ceil(data_size / 256KiB) - 1` is valid. | +| `leaf_index` | Returned by `POST /commit` in the `leaf_indices` array. | +| `provider_sig` | Returned by `POST /commit` or `GET /commitment` (signed with `leaf_count=0`). | +| `mmr_root` | Returned by `POST /commit` or `GET /commitment`. | +| `start_seq` | Returned by `POST /commit` or `GET /commitment`. | + +## Two Challenge Modes + +### `challenge_checkpoint` (Recommended) + +Uses the on-chain `BucketSnapshot` created by a prior checkpoint. The client only needs `leaf_index` and `chunk_index` — all commitment data (MMR root, provider signatures) is already on-chain. + +**Advantages:** +- No local state required beyond knowing which leaf/chunk to challenge. +- Any client (or third party) can challenge — the on-chain snapshot is public. +- Works identically regardless of which device or user agent uploaded the data. +- Immune to client-side data loss — the chain is the source of truth. + +**Requirement:** A checkpoint must exist for the bucket. This is why enabling and funding provider-initiated checkpoints (see [Provider-Initiated Checkpoint](./checkpoint-provider-initiated.md)) is important — it ensures the chain always has a recent snapshot to challenge against. + +### `challenge_offchain` (Limited Applicability) + +Uses a provider's off-chain commitment signature obtained during the write flow. The client must supply the `mmr_root`, `start_seq`, `leaf_index`, `chunk_index`, and `provider_signature` — all of which were returned in the `POST /commit` response at upload time. + +**This mode has significant practical limitations:** + +1. **Client must retain the commit response.** The `provider_signature`, `mmr_root`, `start_seq`, and `leaf_indices` are returned in a single HTTP response during upload. If the client does not persist this data, it is lost — the provider has no obligation to re-issue the same signature, and the MMR root changes with every subsequent write. + +2. **Client-side data loss defeats the purpose.** If the device that performed the upload is lost, wiped, or the application data is cleared, the client no longer has the signature needed to create a challenge. This is precisely the scenario where a challenge might be most needed (the client cannot verify their data because they have lost their local record of it). + +3. **Multi-device / multi-user-agent fragmentation.** When multiple devices (mobile, tablet, browser) write to the same bucket, each write produces a different `POST /commit` response with a different MMR root and provider signature. Device A has no knowledge of the signatures that device B received. This means: + - Each device can only challenge based on its own writes. + - No single device has a complete picture of the bucket's commitment history. + - There is no built-in mechanism to synchronize these ephemeral signatures across devices. + +4. **Signatures are point-in-time snapshots.** The provider signature covers the MMR root at the moment of that specific commit. As the bucket grows (more writes from any client or device), the MMR root changes. The old signature remains valid for challenging the specific leaves it covers, but the client must track which signature corresponds to which leaf indices. + +**When it might still be useful:** For single-client, single-device scenarios where the application persists commit responses and needs to challenge data that has not yet been checkpointed on-chain. Even then, `challenge_checkpoint` is preferred once a checkpoint exists. + +**Bottom line:** `challenge_checkpoint` is the robust, production-ready path. `challenge_offchain` is a lower-level primitive that requires careful client-side state management and does not scale well to multi-device use cases. Clients should ensure checkpoints are regularly created (via provider-initiated or client-initiated flows) so that `challenge_checkpoint` is always available. diff --git a/docs/sequence_diagrams/challenge-response.md b/docs/sequence_diagrams/challenge-response.md new file mode 100644 index 00000000..f458472c --- /dev/null +++ b/docs/sequence_diagrams/challenge-response.md @@ -0,0 +1,59 @@ +# Challenge Response (Successful Defense) + +The provider detects a pending challenge, builds a two-layer proof from its storage backend, and submits it on-chain. + +```mermaid +sequenceDiagram + autonumber + participant Chain as Parachain (Pallet) + participant Provider as Provider Node + participant Storage as Storage Backend + + Note over Provider: ChallengeResponder polls every ~6s + Provider->>Chain: poll_challenges() + Chain-->>Provider: DetectedChallenge{bucket_id, leaf_index, chunk_index, deadline} + + Provider->>Storage: get_mmr_proof(bucket_id, leaf_index) + Storage-->>Provider: MmrProof{peaks, leaf{data_root, data_size}, leaf_proof} + + Provider->>Storage: get_chunk_at_index(data_root, chunk_index) + Storage-->>Provider: chunk_data, chunk_proof + + Provider->>Chain: respond_to_challenge(challenge_id, Proof{chunk_data, mmr_proof, chunk_proof}) + + Chain->>Chain: Verify caller == challenged provider + Chain->>Chain: Verify current_block <= deadline + Chain->>Chain: chunk_hash = blake2_256(chunk_data) + Chain->>Chain: verify_merkle_proof(chunk_hash, chunk_index, chunk_proof, data_root) + Chain->>Chain: verify_mmr_proof(mmr_proof, challenged mmr_root) + Chain->>Chain: Remove challenge from storage + + Note over Chain: Cost split by response time + Chain->>Chain: 1 blk: challenger 90% / provider 10% + Chain->>Chain: 2-5: 80/20, 6-24: 70/30, 25-95: 60/40, 96+: 50/50 + Chain->>Chain: Partial deposit back to challenger, partial slash on provider + Chain-->>Provider: Event: ChallengeDefended +``` + +## Two-Layer Proof Verification + +1. **Chunk in leaf**: `verify_merkle_proof(blake2_256(chunk_data), chunk_index, chunk_proof, data_root)` proves the chunk belongs to the data blob. +2. **Leaf in MMR**: `verify_mmr_proof(mmr_proof, mmr_root)` proves the leaf (containing `data_root`) is in the MMR the provider committed to. + +## Time-Based Cost Split + +The faster the provider responds, the more the challenger pays (discouraging frivolous challenges): + +| Response Time (blocks) | Challenger Pays | Provider Pays | +|---|---|---| +| 1 | 90% | 10% | +| 2-5 | 80% | 20% | +| 6-24 | 70% | 30% | +| 25-95 | 60% | 40% | +| 96+ | 50% | 50% | + +## Alternative Defenses + +Besides `Proof`, providers can also respond with: +- **`Deleted`**: Admin signed a newer `CommitmentPayload` with `start_seq` beyond the challenged leaf (data was legitimately deleted). +- **`Superseded`**: A newer on-chain snapshot already covers the challenged leaf index (challenge is moot). diff --git a/docs/sequence_diagrams/challenge-timeout-slashing.md b/docs/sequence_diagrams/challenge-timeout-slashing.md new file mode 100644 index 00000000..0704e99b --- /dev/null +++ b/docs/sequence_diagrams/challenge-timeout-slashing.md @@ -0,0 +1,32 @@ +# Challenge Timeout & Slashing + +When a provider fails to respond before the deadline, slashing happens automatically in `on_finalize` — no extrinsic needed. + +```mermaid +sequenceDiagram + autonumber + participant Client as Client (Challenger) + participant Chain as Parachain (on_finalize) + + Note over Chain: Block N reaches challenge deadline + Chain->>Chain: on_finalize(N): Challenges::take(N) + Chain->>Chain: Provider failed to respond in time + + Chain->>Chain: Slash provider ENTIRE stake + Chain->>Chain: Refund client full deposit (100 units) + Chain->>Chain: Mint 10% of slashed stake to client as reward + Chain->>Chain: Remaining 90% burned / treasury + Chain->>Chain: provider.stake = 0 + Chain->>Chain: Increment provider.challenges_failed + + Chain-->>Client: Event: ChallengeSlashed{slashed_amount, challenger_reward} +``` + +## Key Details + +- **Automatic**: Slashing is triggered by `on_finalize` at the deadline block. No one needs to call an extrinsic. +- **Full stake loss**: The provider's **entire stake** is slashed (not just a portion). +- **Challenger incentive**: Full deposit refund + 10% of the slashed stake as reward. +- **Remaining 90%**: Burned or sent to treasury via the runtime's slash handler. +- **Provider state**: `provider.stake` is set to 0 and `challenges_failed` is incremented. +- **Deadline**: `ChallengeTimeout` is configured at 48 hours (in blocks) in the runtime. diff --git a/docs/sequence_diagrams/checkpoint-client-initiated.md b/docs/sequence_diagrams/checkpoint-client-initiated.md new file mode 100644 index 00000000..39006688 --- /dev/null +++ b/docs/sequence_diagrams/checkpoint-client-initiated.md @@ -0,0 +1,44 @@ +# Client-Initiated Checkpoint + +The client orchestrates: collects provider signatures via HTTP, verifies consensus, then submits a single on-chain extrinsic. + +```mermaid +sequenceDiagram + autonumber + participant Client as Client SDK + participant P1 as Provider 1 + participant P2 as Provider 2 + participant Chain as Parachain + + Note over Client: submit_checkpoint(bucket_id) + + par Collect signatures + Client->>P1: GET /checkpoint-signature?bucket_id=X + P1->>P1: Sign CommitmentPayload (sr25519) + P1-->>Client: mmr_root, start_seq, leaf_count, sig + and + Client->>P2: GET /checkpoint-signature?bucket_id=X + P2->>P2: Sign CommitmentPayload (sr25519) + P2-->>Client: mmr_root, start_seq, leaf_count, sig + end + + Client->>Client: Group by mmr_root, check consensus >= 51% + + alt Consensus reached + Client->>Chain: checkpoint(bucket_id, mmr_root, start_seq, leaf_count, sigs[]) + Chain->>Chain: Verify caller is writer/admin + Chain->>Chain: Verify each signature against registered pubkeys + Chain->>Chain: Check signing_count >= min_providers + Chain->>Chain: Create BucketSnapshot + Chain-->>Client: Event BucketCheckpointed + else No consensus + Client-->>Client: CheckpointResult::NoConsensus + end +``` + +## Key Details + +- **CommitmentPayload** is signed with the **real** `leaf_count` (via `/checkpoint-signature`). +- The client verifies majority consensus (same `mmr_root` from >= 51% of providers) before submitting. +- On-chain, each signature is verified against the provider's registered public key, and the provider's bit is set in the `primary_signers` bitfield. +- The resulting `BucketSnapshot` makes providers liable for the data. diff --git a/docs/sequence_diagrams/checkpoint-provider-initiated.md b/docs/sequence_diagrams/checkpoint-provider-initiated.md new file mode 100644 index 00000000..9605a6bf --- /dev/null +++ b/docs/sequence_diagrams/checkpoint-provider-initiated.md @@ -0,0 +1,118 @@ +# Provider-Initiated Checkpoint + +Autonomous: a deterministic leader coordinates signature collection from peers and submits on-chain. Includes grace period fallback and missed-window slashing. + +## Prerequisite: Client Must Enable and Fund + +Provider-initiated checkpoints are **enabled by default** on every bucket (with runtime defaults: interval=100 blocks, grace=20 blocks). However, the system only works sustainably when the **client (bucket admin)** actively configures and funds it: + +1. **Configure** — The bucket admin calls `configure_checkpoint_window(bucket_id, interval, grace_period, enabled)` to set the checkpoint schedule. Setting `enabled=false` disables provider-initiated checkpoints entirely (client-initiated checkpoints via the `checkpoint` extrinsic still work regardless). + +2. **Fund the reward pool** — The bucket admin (or anyone) calls `fund_checkpoint_pool(bucket_id, amount)` to deposit tokens into the bucket's reward pool. Each successful provider checkpoint deducts `CheckpointReward` (1 token) from this pool. **If the pool is empty, checkpoints still proceed but providers receive zero reward** — reducing their economic incentive to submit on time. + +**This is a cost borne by the client.** The client is paying providers to periodically commit the bucket's MMR state on-chain, which: +- Makes providers slashable for data loss (the chain has a canonical snapshot to challenge against) +- Creates on-chain proof of data integrity at regular intervals +- Enables the `challenge_checkpoint` flow (which requires an on-chain snapshot) + +Without funding, providers have no economic incentive to submit checkpoints (though they can still be slashed via `report_missed_checkpoint` if checkpoints are enabled). The client should budget for `CheckpointReward * expected_windows` tokens in the pool. + +```mermaid +sequenceDiagram + autonumber + participant Admin as Bucket Admin (Client) + participant Chain as Parachain + + Admin->>Chain: configure_checkpoint_window(
bucket_id, interval=100, grace=20, enabled=true) + Chain->>Chain: Ensure caller is bucket admin + Chain->>Chain: Store CheckpointWindowConfig + Chain-->>Admin: Event: CheckpointConfigUpdated + + Admin->>Chain: fund_checkpoint_pool(bucket_id, amount=100 tokens) + Chain->>Chain: Currency::reserve(admin, amount) + Chain->>Chain: CheckpointPool[bucket_id] += amount + Chain-->>Admin: Event: CheckpointPoolFunded{funder, amount} + + Note over Admin: Provider-initiated checkpoints now
active and funded. Each checkpoint
costs 1 token from the pool. +``` + +## Checkpoint Submission Flow + +```mermaid +sequenceDiagram + autonumber + participant Leader as Leader Provider + participant Peer as Peer Provider + participant Chain as Parachain + participant Reporter as Reporter + + Note over Chain: Leader = blake2_256(bucket_id || window) % N + + Leader->>Chain: get_active_checkpoint_duties() + Chain-->>Leader: Duty: bucket_id, window, is_leader=true + + Leader->>Leader: Build CheckpointProposal with window + Leader->>Leader: Self-sign proposal (sr25519) + + Leader->>Peer: POST /checkpoint/sign + Peer->>Peer: Compare local state to proposal + + alt State matches + Peer-->>Leader: agreed=true, signature + else State disagrees + Peer-->>Leader: agreed=false + end + + Leader->>Leader: Collected sigs >= min_providers + + Leader->>Chain: provider_checkpoint(bucket_id, mmr_root, window, sigs[]) + Chain->>Chain: Verify window == current_window + Chain->>Chain: Verify window > LastCheckpointWindow + + alt Within grace period + Chain->>Chain: Only designated leader allowed + else After grace period + Chain->>Chain: Any primary provider allowed + end + + Chain->>Chain: Verify signatures, update BucketSnapshot + Chain->>Chain: Pay reward from CheckpointPool + Chain-->>Leader: Event ProviderCheckpointSubmitted + + Note over Reporter, Chain: Missed Checkpoint Penalty + + Reporter->>Chain: report_missed_checkpoint(bucket_id, window) + Chain->>Chain: Verify window passed without checkpoint + Chain->>Chain: Slash leader 0.5 token, reward reporter 10% + Chain-->>Reporter: Event CheckpointMissPenalized +``` + +## Key Details + +- **Window**: `current_block / interval`. Default interval is 100 blocks. +- **Leader election**: Deterministic via `blake2_256(bucket_id || window) % num_providers`. +- **Grace period** (default 20 blocks): Only the leader can submit during grace. After grace, any primary provider can submit as fallback. +- **CheckpointProposal** includes the `window` field (unlike `CommitmentPayload`) to prevent cross-window replay. +- **Rewards**: Submitter receives `CheckpointReward` (1 token) from the `CheckpointPool` funded by clients. If the pool is empty, the checkpoint is still accepted but with zero reward. +- **Penalties**: If no checkpoint is submitted for a past window, anyone can call `report_missed_checkpoint`. The leader is slashed `CheckpointMissPenalty` (0.5 token), reporter gets 10%. + +## Cost Awareness for Clients + +| Item | Who Pays | Amount | When | +|------|----------|--------|------| +| `configure_checkpoint_window` | Bucket admin | Transaction fee only | One-time setup | +| `fund_checkpoint_pool` | Bucket admin (or anyone) | Deposited amount is reserved | As needed to keep pool funded | +| Checkpoint reward | Deducted from pool | `CheckpointReward` (1 token) per checkpoint | Each successful provider checkpoint | +| Missed checkpoint penalty | Provider (slashed) | `CheckpointMissPenalty` (0.5 token) | Leader fails to submit in time | + +**Budget estimation**: For a bucket with `interval=100` blocks and ~6 second block time, one checkpoint occurs every ~10 minutes. That's ~144 checkpoints/day, costing 144 tokens/day from the pool. The client should fund the pool proportionally to the desired coverage period. + +## Disabling Provider Checkpoints + +The bucket admin can disable provider-initiated checkpoints at any time: + +``` +configure_checkpoint_window(bucket_id, interval, grace, enabled=false) +``` + +This prevents the `provider_checkpoint` extrinsic from succeeding (`ProviderCheckpointsDisabled` error). Client-initiated checkpoints via the `checkpoint` extrinsic are **unaffected** and always work regardless of this setting. diff --git a/docs/sequence_diagrams/provider-deregistration.md b/docs/sequence_diagrams/provider-deregistration.md new file mode 100644 index 00000000..429620d0 --- /dev/null +++ b/docs/sequence_diagrams/provider-deregistration.md @@ -0,0 +1,93 @@ +# Provider Deregistration (Two-Step Exit) + +Providers cannot exit instantly — a cooldown period (>= `ChallengeTimeout`, default 48h) ensures any pending challenges can still be resolved. The provider remains on-chain and slashable during the wait. + +## Deregistration Flow + +```mermaid +sequenceDiagram + autonumber + participant Provider as Provider (Wallet) + participant Chain as Parachain (StorageProvider Pallet) + + Note over Provider: Step 1: Announce intent to deregister + + Provider->>Chain: deregister_provider() + + Chain->>Chain: Ensure provider registered + Chain->>Chain: Ensure committed_bytes == 0
(no active agreements) + Chain->>Chain: Set deregister_at = now + DeregisterAnnouncementPeriod + Chain->>Chain: Force accepting_primary = false + Chain->>Chain: Force accepting_extensions = false + + Chain-->>Provider: Event: DeregisterAnnounced{
provider, complete_after} + + Note over Provider,Chain: Cooldown period (>= ChallengeTimeout)
Provider remains on-chain and slashable.
Cannot accept new agreements.
Existing challenges can still be resolved. + + rect rgb(255, 245, 235) + Note over Provider,Chain: Wait for DeregisterAnnouncementPeriod to elapse + end + + Note over Provider: Step 2: Complete deregistration + + Provider->>Chain: complete_deregister() + + Chain->>Chain: Ensure current_block >= deregister_at + Chain->>Chain: Ensure committed_bytes == 0 (still no agreements) + Chain->>Chain: Drain pending CheckpointRewards into free balance + Chain->>Chain: Currency::unreserve(provider, stake)
Tokens moved: reserved -> free balance + Chain->>Chain: Remove Providers storage entry + Chain->>Chain: Remove ProviderReplayStates entry + + Chain-->>Provider: Event: ProviderDeregistered{
provider, stake_returned} + + Note over Provider: Provider fully removed from chain.
Stake returned to free balance. +``` + +## Cancellation Flow + +A provider can cancel the deregistration at any time before completing it. + +```mermaid +sequenceDiagram + autonumber + participant Provider as Provider (Wallet) + participant Chain as Parachain + + Provider->>Chain: cancel_deregister() + + Chain->>Chain: Ensure deregister_at is set + Chain->>Chain: Clear deregister_at = None + Chain->>Chain: Restore accepting_primary = true + Chain->>Chain: Restore accepting_extensions = true + + Chain-->>Provider: Event: DeregisterCancelled{provider} + + Note over Provider: Provider back to normal operation +``` + +## Why Two Steps? + +The cooldown period exists for security: + +1. **Challenge resolution**: Any client who has challenged the provider needs time for the challenge to be resolved. If providers could exit instantly, they could dodge slashing by deregistering before the challenge deadline. +2. **DeregisterAnnouncementPeriod >= ChallengeTimeout**: This guarantee ensures that any challenge created before the announcement will have its deadline expire before the provider can withdraw stake. +3. **Provider remains slashable**: During the cooldown, the provider's stake is still reserved and can be slashed by `on_finalize` if a challenge expires without response. + +## Preconditions + +| Check | Error | +|-------|-------| +| Provider must be registered | `ProviderNotFound` | +| `committed_bytes == 0` (no active agreements) | `ProviderHasActiveAgreements` | +| `deregister_at` not already set (for announce) | `DeregisterAnnounced` | +| `deregister_at` is set (for complete/cancel) | `DeregisterNotAnnounced` | +| `current_block >= deregister_at` (for complete) | `DeregisterPeriodNotElapsed` | + +## Events + +| Event | When | +|-------|------| +| `DeregisterAnnounced { provider, complete_after }` | Cooldown begins | +| `DeregisterCancelled { provider }` | Provider cancels exit | +| `ProviderDeregistered { provider, stake_returned }` | Stake returned, provider removed | diff --git a/docs/sequence_diagrams/provider-registration.md b/docs/sequence_diagrams/provider-registration.md new file mode 100644 index 00000000..b8b3714a --- /dev/null +++ b/docs/sequence_diagrams/provider-registration.md @@ -0,0 +1,151 @@ +# Provider Registration + +End-to-end flow from the provider UI wizard through on-chain registration, settings configuration, and provider node startup. + +## 1. On-Chain Registration (Two Transactions) + +```mermaid +sequenceDiagram + autonumber + participant Provider as Provider (Wallet) + participant Chain as Parachain (StorageProvider Pallet) + + Provider->>Chain: register_provider(
multiaddr: "/ip4/127.0.0.1/tcp/3333",
public_key: sr25519 (32 bytes),
stake: 1000+ tokens) + + Chain->>Chain: Ensure not already registered + Chain->>Chain: Ensure stake >= MinProviderStake (1000 tokens) + Chain->>Chain: Validate public_key length (32, 33, or 64 bytes) + Chain->>Chain: validate_settings(defaults, committed=0, stake) + + Chain->>Chain: Currency::reserve(provider, stake)
Tokens moved: free balance -> reserved balance + Chain->>Chain: Create ProviderInfo{multiaddr, public_key,
stake, committed_bytes=0,
settings=default, stats={registered_at=now}} + Chain->>Chain: Insert into Providers storage map + Chain->>Chain: Insert default ProviderReplayStates (nonce window) + + Chain-->>Provider: Event: ProviderRegistered{provider, stake} + + Note over Provider,Chain: Immediately update settings + + Provider->>Chain: update_provider_settings({
min_duration, max_duration,
price_per_byte, accepting_primary: true,
replica_sync_price, accepting_extensions,
max_capacity}) + + Chain->>Chain: Validate min_duration <= max_duration + Chain->>Chain: Ensure provider not in deregister state + Chain->>Chain: validate_settings: if max_capacity > 0,
stake >= max_capacity * MinStakePerByte + + Chain->>Chain: Update ProviderInfo.settings + Chain-->>Provider: Event: ProviderSettingsUpdated{provider, settings} + + Note over Provider: Provider now discoverable via
find_matching_providers runtime API +``` + +## 2. Provider Node Startup + +```mermaid +sequenceDiagram + autonumber + participant Node as Provider Node (CLI) + participant Chain as Parachain + + Note over Node: storage-provider-node --keyfile key.txt
--chain-rpc ws://... --bind-addr 0.0.0.0:3333 + + Node->>Node: Load seed from keyfile (chmod 600) + Node->>Node: Derive Sr25519 keypair -> SS58 account ID + Node->>Node: Create storage backend (in-memory or disk) + + Node->>Chain: Read Providers storage for this account + alt Not registered + Node->>Node: Log warning: "register it before starting" + else Registered + Node->>Node: Load settings (price, capacity, etc.) + end + + Node->>Chain: Read ProviderReplayStates.hsn + Node->>Node: Bootstrap nonce counter (hsn + 1) + + opt --enable-checkpoint-coordinator + Node->>Node: Start checkpoint coordinator (polls every 6s) + end + + opt Replica sync enabled + Node->>Node: Start replica sync coordinator + end + + Node->>Chain: Read on-chain multiaddr + alt On-chain multiaddr != bind/public addr + Node->>Chain: update_provider_multiaddr(new_multiaddr) + Chain-->>Node: Event: ProviderMultiaddrUpdated + end + + Node->>Node: Start HTTP server on bind address + + Note over Node: Ready to accept uploads,
serve downloads, respond to challenges +``` + +## 3. Adding Stake + +```mermaid +sequenceDiagram + autonumber + participant Provider as Provider (Wallet) + participant Chain as Parachain + + Provider->>Chain: add_provider_stake(amount) + Chain->>Chain: Ensure provider registered + Chain->>Chain: Ensure not in deregister state + Chain->>Chain: Currency::reserve(provider, amount) + Chain->>Chain: provider.stake += amount + Chain-->>Provider: Event: ProviderStakeAdded{provider, amount, total_stake} +``` + +## Key Types + +### ProviderInfo (on-chain) + +| Field | Type | Description | +|-------|------|-------------| +| `multiaddr` | `BoundedVec` | Network address, e.g. `/ip4/127.0.0.1/tcp/3333` | +| `public_key` | `BoundedVec` | Sr25519 (32b), Ed25519 (32b), or Ecdsa (33/64b) | +| `stake` | `Balance` | Locked collateral | +| `committed_bytes` | `u64` | Sum of `max_bytes` across active agreements | +| `settings` | `ProviderSettings` | Pricing and availability config | +| `stats` | `ProviderStats` | On-chain reputation metrics | +| `deregister_at` | `Option` | Set during two-step exit | + +### ProviderSettings + +| Field | Default | Description | +|-------|---------|-------------| +| `min_duration` | 0 | Minimum agreement duration (blocks) | +| `max_duration` | MAX | Maximum agreement duration (blocks) | +| `price_per_byte` | 0 | Price per byte per block | +| `accepting_primary` | true | Accepting new primary agreements | +| `replica_sync_price` | None | None = not accepting replicas | +| `accepting_extensions` | true | Accepting agreement extensions | +| `max_capacity` | 0 | 0 = unlimited | + +### Stake Requirements + +``` +MinProviderStake = 1000 tokens (absolute minimum) +MinStakePerByte = 1000 units per byte of declared capacity + +Required stake = max(MinProviderStake, max_capacity * MinStakePerByte) +``` + +If `max_capacity = 0` (unlimited), only `MinProviderStake` applies. + +## Storage Items Created + +| Storage Item | Key | Value | +|-------------|-----|-------| +| `Providers` | `AccountId` | `ProviderInfo { multiaddr, public_key, stake, settings, stats, ... }` | +| `ProviderReplayStates` | `AccountId` | `ReplayWindow` (nonce sliding window for agreement signing) | + +## Events + +| Event | When | +|-------|------| +| `ProviderRegistered { provider, stake }` | Registration succeeds | +| `ProviderSettingsUpdated { provider, settings }` | Settings changed | +| `ProviderMultiaddrUpdated { provider }` | Network address changed | +| `ProviderStakeAdded { provider, amount, total_stake }` | Additional stake locked | diff --git a/docs/sequence_diagrams/s3-bucket-creation.md b/docs/sequence_diagrams/s3-bucket-creation.md new file mode 100644 index 00000000..f1d2a635 --- /dev/null +++ b/docs/sequence_diagrams/s3-bucket-creation.md @@ -0,0 +1,143 @@ +# S3 Bucket Creation + +End-to-end flow from the UI through off-chain negotiation to a single atomic on-chain extrinsic that creates a Layer 0 bucket, a storage agreement, and an S3 registry entry. + +## 1. Provider Discovery + +```mermaid +sequenceDiagram + autonumber + participant UI as S3 UI (NewBucketDialog) + participant Chain as Parachain + + UI->>Chain: StorageProviderApi.find_matching_providers(
{bytesNeeded, minDuration, maxPricePerByte, primaryOnly}, limit) + Chain->>Chain: Score providers 0-100 based on:
accepting status, capacity, price, duration range + Chain-->>UI: MatchingProviders[] sorted by matchScore + + UI->>UI: Display provider list with capacity,
price, reputation stats + UI->>UI: User selects a provider +``` + +## 2. Off-Chain Negotiation + +```mermaid +sequenceDiagram + autonumber + participant UI as S3 UI + participant Provider as Provider Node (HTTP) + + UI->>UI: parseMultiaddrToHttp(provider.multiaddr) + + UI->>Provider: POST /negotiate {owner, max_bytes,
duration, price_per_byte, bucket_id: null} + + Provider->>Provider: Validate against on-chain settings:
- accepting_primary == true
- price >= provider's listed price
- duration in [min_duration, max_duration]
- capacity not exceeded + + Provider->>Provider: Build AgreementTerms {
owner, max_bytes, duration,
price_per_byte (provider's own price),
valid_until, nonce (monotonic),
bucket_id: None, replica_params: None
} + + Provider->>Provider: Sign: sr25519_sign(
blake2_256("primary-term-v1:" | SCALE(terms))) + + Provider-->>UI: SignedTerms { terms, signature } +``` + +## 3. On-Chain Submission (Single Atomic Extrinsic) + +```mermaid +sequenceDiagram + autonumber + participant UI as S3 UI + participant Chain as Parachain + participant SP as StorageProvider Pallet + participant S3 as S3Registry Pallet + + UI->>Chain: S3Registry.create_s3_bucket(name, provider, terms, sig) + + Note over S3: S3Registry pallet validates name + S3->>S3: validate_bucket_name (3-63 chars, lowercase + hyphens) + S3->>S3: Check name uniqueness (BucketNameToId) + S3->>S3: Check user bucket limit (MaxBucketsPerUser) + + Note over S3,SP: Delegates to Layer 0 pallet + S3->>SP: establish_storage_agreement_internal(owner, provider, terms, sig) + + SP->>SP: Verify owner == terms.owner + SP->>SP: Verify bucket_id is None (new primary bucket) + SP->>SP: Verify terms not expired (valid_until >= current_block) + SP->>SP: Verify provider signature:
blake2_256("primary-term-v1:" | SCALE(terms)) + SP->>SP: Replay protection: nonce sliding window check + SP->>SP: Provider active + accepting_primary + SP->>SP: Duration in [min_duration, max_duration] + SP->>SP: Capacity: committed_bytes + max_bytes <= max_capacity + SP->>SP: Stake: provider.stake >= (new_committed * MinStakePerByte) + + SP->>SP: Reserve payment from owner:
price_per_byte * max_bytes * duration + + SP->>SP: Create Layer 0 bucket (owner as Admin, provider as primary) + SP-->>Chain: Event: BucketCreated{bucket_id, admin} + + SP->>SP: Insert StorageAgreement{max_bytes, payment, expires_at, ...} + SP->>SP: Update provider: committed_bytes, stats + SP-->>Chain: Event: StorageAgreementEstablished{
bucket_id, provider, owner, terms, expires_at} + + SP-->>S3: return layer0_bucket_id + + Note over S3: S3Registry creates the S3 layer entry + S3->>S3: Allocate s3_bucket_id (auto-increment) + S3->>S3: Store S3BucketInfo{s3_bucket_id, name,
layer0_bucket_id, owner, created_at} + S3->>S3: Insert BucketNameToId, update UserBuckets + S3-->>Chain: Event: S3BucketCreated{
s3_bucket_id, name, layer0_bucket_id, owner} + + Chain-->>UI: Transaction finalized + 3 events +``` + +## 4. UI Completion + +```mermaid +sequenceDiagram + autonumber + participant UI as S3 UI + + UI->>UI: Extract S3BucketCreated event
(s3_bucket_id, layer0_bucket_id) + UI->>UI: Cache provider URL keyed by layer0_bucket_id + UI->>UI: Update creation status to "ready" + UI->>UI: refreshBuckets() — reload full bucket list + UI->>UI: selectBucket(new bucket) — auto-select + UI->>UI: refreshBalance() — update on-chain balance
(payment was reserved) + + Note over UI: Bucket is ready for uploads +``` + +## Events Emitted (in order) + +A successful `create_s3_bucket` extrinsic emits three events: + +| # | Pallet | Event | Key Fields | +|---|--------|-------|------------| +| 1 | `StorageProvider` | `BucketCreated` | `bucket_id`, `admin` | +| 2 | `StorageProvider` | `StorageAgreementEstablished` | `bucket_id`, `provider`, `owner`, `terms`, `expires_at` | +| 3 | `S3Registry` | `S3BucketCreated` | `s3_bucket_id`, `name`, `layer0_bucket_id`, `owner` | + +## On-Chain Storage Created + +| Pallet | Storage Item | Key | Value | +|--------|-------------|-----|-------| +| `StorageProvider` | `Buckets` | `bucket_id` | `Bucket { members, primary_providers, snapshot, ... }` | +| `StorageProvider` | `MemberBuckets` | `account_id` | `BoundedVec` | +| `StorageProvider` | `StorageAgreements` | `(bucket_id, provider)` | `StorageAgreement { max_bytes, payment_locked, expires_at, ... }` | +| `StorageProvider` | `Providers` | `provider` | Updated `committed_bytes` and `stats` | +| `StorageProvider` | `ProviderReplayStates` | `provider` | Nonce window advanced | +| `S3Registry` | `S3Buckets` | `s3_bucket_id` | `S3BucketInfo { name, layer0_bucket_id, owner, ... }` | +| `S3Registry` | `BucketNameToId` | `name` | `s3_bucket_id` | +| `S3Registry` | `UserBuckets` | `account_id` | `BoundedVec` | +| `S3Registry` | `NextS3BucketId` | (value) | Incremented by 1 | + +## Payment Calculation + +``` +payment = price_per_byte * max_bytes * duration +``` + +The payment is reserved (locked) from the owner's account at bucket creation time. The provider's listed `price_per_byte` is used (set during negotiation, not the client's proposed price). + +## Replay Protection + +The provider allocates a monotonic nonce for each negotiation. On-chain, a sliding window (`ProviderReplayState`) tracks used nonces. This prevents replay attacks where a stale `SignedTerms` could be submitted again. diff --git a/docs/sequence_diagrams/s3-read-flow.md b/docs/sequence_diagrams/s3-read-flow.md new file mode 100644 index 00000000..bbd5f60a --- /dev/null +++ b/docs/sequence_diagrams/s3-read-flow.md @@ -0,0 +1,122 @@ +# S3 Read Flow (getObject) + +End-to-end flow from the client requesting an object through provider URL resolution, authorization, Merkle tree traversal, and data reassembly. + +## Read Flow + +```mermaid +sequenceDiagram + autonumber + participant Client as Client (S3 UI) + participant Chain as Parachain + participant Primary as Primary Provider (HTTP) + participant Storage as Storage Backend (RocksDB) + + Client->>Client: Sign request: sr25519_sign(
"web3storage:GET:{bucket_id}:{timestamp}") + + Client->>Chain: Resolve provider URL (cached):
Buckets.primary_providers[0]
-> Providers.multiaddr -> HTTP URL + Chain-->>Client: Provider URL + + Client->>Primary: GET /s3/{bucket_id}/object?key=photos/cat.jpg
Authorization: Web3Storage {pubkey}:{sig}:{ts} + + Primary->>Primary: Auth: verify sr25519 signature
+ check Reader role via membership cache + + Primary->>Primary: S3 index lookup: key -> ObjectMeta{
data_root, size, content_type, etag, leaf_index} + + alt Key not found + Primary-->>Client: 404 ObjectNotFound + end + + Note over Primary,Storage: DFS traversal from data_root to collect chunks + + Primary->>Storage: collect_chunks(data_root) + + rect rgb(245, 245, 255) + Note over Storage: Iterative DFS (stack-based) + Storage->>Storage: Push data_root onto stack + + loop While stack not empty + Storage->>Storage: Pop hash from stack + Storage->>Storage: Fetch node from RocksDB (CF_NODES) + + alt Internal node (has children) + Storage->>Storage: Push children onto stack (reversed for order) + else Leaf node (no children) + Storage->>Storage: Append chunk data to result + end + end + end + + Storage-->>Primary: chunks[] (ordered left-to-right) + + Primary->>Primary: Concatenate chunks + Primary->>Primary: Truncate to original size (remove Merkle padding) + + Primary-->>Client: 200 OK
Content-Type: {content_type}
ETag: {etag}
Body: raw bytes +``` + +## Rust Client (Layer 0) — Verified Download + +The Rust SDK downloads with chunk-level verification, unlike the S3 API which trusts the provider. + +```mermaid +sequenceDiagram + autonumber + participant RustClient as Rust Client SDK + participant Primary as Primary Provider (HTTP) + + RustClient->>Primary: GET /read?data_root={hash}&offset=0&length=N + + Primary-->>RustClient: chunks[] with Merkle proofs + + loop For each chunk + RustClient->>RustClient: Verify blake2_256(chunk_data) == expected_hash + RustClient->>RustClient: Verify Merkle proof up to data_root + end + + RustClient->>RustClient: Reassemble, trim to requested range + RustClient->>RustClient: Decrypt if encryption key set +``` + +## How Multiple Providers Serve the Same Bucket + +```mermaid +sequenceDiagram + autonumber + participant Client as Client + participant Chain as Parachain + participant P1 as Primary 1 + participant P2 as Primary 2 + participant R1 as Replica 1 + + Note over Client,Chain: Provider resolution + + Client->>Chain: Buckets(id).primary_providers + Chain-->>Client: [P1_account, P2_account] + Client->>Chain: Providers(P1_account).multiaddr + Chain-->>Client: /ip4/.../tcp/3333 + Client->>Client: Use P1 (first primary) + + Note over Client,P1: Normal read/write path + Client->>P1: PUT /s3/{id}/object (write) + Client->>P1: GET /s3/{id}/object (read) + + Note over P1,P2: Checkpoint coordination (both sign) + P1->>P2: POST /checkpoint/sign (agree on MMR state) + + Note over P1,R1: Replica sync (background) + R1->>P1: GET /mmr_peaks + GET /node (pull data) + R1->>Chain: confirm_replica_sync (get paid) + + Note over Client: Currently no client-side fallback
to P2 or R1 if P1 is down.
Replicas serve as a data availability
backstop, not a read path. +``` + +## Key Differences: S3 API vs Layer 0 API + +| Aspect | S3 API (TypeScript) | Layer 0 API (Rust) | +|--------|--------------------|--------------------| +| Upload | Single PUT request (auto-chunks, auto-commits) | Multi-step: upload chunks, build tree, commit separately | +| Download | Returns raw bytes | Returns chunks with Merkle proofs | +| Verification | Trusts provider | Client verifies each chunk hash | +| Provider resolution | Dynamic from chain (cached) | Static URL list in config | +| Encryption | Done in client state layer before upload | Built into SDK with `ChunkingStrategy` | diff --git a/docs/sequence_diagrams/s3-write-flow.md b/docs/sequence_diagrams/s3-write-flow.md new file mode 100644 index 00000000..68e71c9f --- /dev/null +++ b/docs/sequence_diagrams/s3-write-flow.md @@ -0,0 +1,157 @@ +# S3 Write Flow (putObject) + +End-to-end flow from the client uploading an object through chunking, Merkle tree construction, and MMR commit on the primary provider, followed by replica sync. + +## 1. Client Upload to Primary Provider + +```mermaid +sequenceDiagram + autonumber + participant Client as Client (S3 UI) + participant Chain as Parachain + participant Primary as Primary Provider (HTTP) + participant Storage as Storage Backend (RocksDB) + + Client->>Client: Sign request: sr25519_sign(
"web3storage:PUT:{bucket_id}:{timestamp}") + + Client->>Chain: Resolve provider URL:
Buckets(bucket_id).primary_providers[0]
-> Providers(account).multiaddr
-> parse to HTTP URL + Chain-->>Client: Provider URL (cached per bucket) + + Client->>Primary: PUT /s3/{bucket_id}/object?key=photos/cat.jpg
Authorization: Web3Storage {pubkey}:{sig}:{ts}
Body: raw bytes + + Primary->>Primary: Auth: verify sr25519 signature
+ check Writer role via membership cache
(chain lookup with TTL cache) + + Note over Primary,Storage: Step 1: Chunk data (256 KiB each) + + Primary->>Primary: Split body into N chunks
(N = ceil(size / 256KiB)) + + Note over Primary,Storage: Step 2: Store chunks (content-addressed) + + loop For each chunk + Primary->>Primary: chunk_hash = blake2_256(chunk_data) + Primary->>Storage: store_node(bucket_id, chunk_hash, data, children=None) + Storage->>Storage: Verify blake2_256(data) == chunk_hash + Storage->>Storage: Check quota: used_bytes + len <= max_bytes + Storage->>Storage: Store in RocksDB CF_NODES keyed by hash + end + + Note over Primary,Storage: Step 3: Build balanced Merkle tree + + Primary->>Primary: Pad chunk hashes to next power of 2 (zeros) + loop Bottom-up tree construction + Primary->>Primary: parent_hash = blake2_256(left || right) + Primary->>Storage: store_node(bucket_id, parent_hash, data, children=[left, right]) + end + Primary->>Primary: data_root = tree root hash + + Note over Primary,Storage: Step 4: Commit to MMR + + Primary->>Storage: commit(bucket_id, [data_root]) + Storage->>Storage: Create MmrLeaf{data_root, data_size, total_size} + Storage->>Storage: leaf_hash = blake2_256(SCALE(MmrLeaf)) + Storage->>Storage: Append to MMR, recalculate root (bag peaks) + Storage-->>Primary: (mmr_root, start_seq, leaf_indices) + + Note over Primary: Step 5: Update S3 index + + Primary->>Primary: s3_index.put(key -> ObjectMeta{
data_root, size, etag, leaf_index, ...}) + + Primary-->>Client: 200 OK {etag, data_root, size, leaf_index} +``` + +## 2. Replica Sync (Background) + +After the primary stores data, replicas pull it autonomously. + +```mermaid +sequenceDiagram + autonumber + participant Replica as Replica Provider + participant Chain as Parachain + participant Primary as Primary Provider (HTTP) + + Note over Replica: ReplicaSyncCoordinator polls every ~12s + + Replica->>Chain: Fetch replica agreements for this provider + Chain-->>Replica: SyncDuty{bucket_id, target_mmr_root,
primary_endpoints, sync_balance, sync_price} + + Replica->>Replica: Check eligibility:
- sync_balance >= sync_price
- sync_interval elapsed
- local_root != target_root + + Replica->>Primary: GET /mmr_peaks?bucket_id=X + Primary-->>Replica: peaks[] (root hashes of MMR subtrees) + + loop For each peak, recursively fetch subtree + Replica->>Primary: GET /node?hash={peak_hash} + Primary-->>Replica: {data, children} + Replica->>Replica: Verify blake2_256(data) == hash + Replica->>Replica: Store node locally + + opt Node has children (internal node) + Note over Replica,Primary: Recurse: fetch each child
(skip if already in local storage) + end + end + + Replica->>Replica: Verify local mmr_root == target_root + + Replica->>Chain: confirm_replica_sync(bucket_id, roots[7]) + Chain->>Chain: Match submitted root against
snapshot + 6 historical_roots + Chain->>Chain: Deduct sync_price from sync_balance + Chain->>Chain: Transfer sync_price to replica provider + Chain->>Chain: Update last_sync = (root, current_block) + Chain-->>Replica: Event: ReplicaSynced{bucket_id, provider, root} +``` + +## Data Structure: From Object to MMR + +``` +Object (raw bytes) + | + v +Chunks (256 KiB each, content-addressed by blake2_256) + [chunk_0] [chunk_1] [chunk_2] [chunk_3] ... + \ / \ / + \ / \ / + Balanced Merkle Tree (padded to power of 2) + [internal] [internal] + \ / + \ / + data_root (H256) + | + v + MmrLeaf { data_root, data_size, total_size } + | + v + leaf_hash = blake2_256(SCALE(MmrLeaf)) + | + v + MMR (append-only, peaks bagged into mmr_root) +``` + +## Provider URL Resolution + +The client resolves which provider to talk to by reading on-chain state: + +``` +bucket.primary_providers[0] (AccountId from Buckets storage) + -> provider.multiaddr (from Providers storage) + -> parseMultiaddrToUrl() (/ip4/127.0.0.1/tcp/3333 -> http://127.0.0.1:3333) +``` + +Result is cached per bucket. Currently, the client always talks to the **first primary provider** — there is no fallback to replicas on the read/write path. + +## Authorization + +Requests are signed with sr25519: + +``` +Authorization: Web3Storage :: +Signed message: "web3storage:::" +``` + +The provider verifies the signature, resolves the account's role from on-chain bucket membership (cached with TTL), and checks: + +| Operation | Required Role | +|-----------|--------------| +| PUT (write) | Writer or Admin | +| GET (read) | Reader, Writer, or Admin | +| DELETE | Writer or Admin |