Add sequence diagrams for core protocol flows

docs/sequence_diagrams/README.md

@@ -0,0 +1,42 @@
+# Sequence Diagrams
+
+Visual sequence diagrams for the core protocol flows in Scalable Web3 Storage.
+
+## Provider Lifecycle
+
+| Diagram | Description |
+|---|---|
+| [Provider Registration](./provider-registration.md) | On-chain registration (stake locking, settings), provider node startup sequence, and adding stake. |
+| [Provider Deregistration](./provider-deregistration.md) | Two-step exit: announce (cooldown >= ChallengeTimeout), then complete (stake returned). Includes cancellation flow. |
+
+## S3 Bucket Creation
+
+| Diagram | Description |
+|---|---|
+| [S3 Bucket Creation](./s3-bucket-creation.md) | End-to-end flow: provider discovery, off-chain negotiation with term signing, single atomic on-chain extrinsic (Layer 0 bucket + agreement + S3 registry), and UI completion. |
+
+## S3 Object Read & Write
+
+| Diagram | Description |
+|---|---|
+| [S3 Write Flow (putObject)](./s3-write-flow.md) | Client upload to primary provider: auth, chunking (256 KiB), Merkle tree, MMR commit, S3 index update. Includes replica sync (background pull from primaries). |
+| [S3 Read Flow (getObject)](./s3-read-flow.md) | Client download: provider URL resolution, auth, DFS tree traversal, chunk reassembly. Includes Rust SDK verified download and multi-provider topology. |
+
+## Checkpoint Flows
+
+How on-chain `BucketSnapshot` records are created, making providers liable for stored data.
+
+| Diagram | Description |
+|---|---|
+| [Client-Initiated Checkpoint](./checkpoint-client-initiated.md) | Client collects provider signatures via HTTP, verifies consensus, submits on-chain. |
+| [Provider-Initiated Checkpoint](./checkpoint-provider-initiated.md) | Autonomous leader election, peer co-signing, grace period fallback, and missed-window penalties. |
+
+## Challenge Flows
+
+The dispute mechanism: clients challenge providers to prove they still hold data.
+
+| Diagram | Description |
+|---|---|
+| [Challenge Creation](./challenge-creation.md) | How clients create challenges (both `challenge_checkpoint` and `challenge_offchain`). Includes how the client obtains `chunk_index` and `provider_sig`. |
+| [Challenge Response](./challenge-response.md) | Provider detects challenge, builds two-layer MMR + Merkle proof, submits on-chain. Includes time-based cost split. |
+| [Challenge Timeout & Slashing](./challenge-timeout-slashing.md) | Automatic slashing via `on_finalize` when a provider fails to respond. |

docs/sequence_diagrams/challenge-creation.md

@@ -0,0 +1,83 @@
+# Challenge Creation
+
+The client (challenger) initiates a challenge when a provider fails to serve data. Two entry points converge into the same on-chain challenge.
+
+```mermaid
+sequenceDiagram
+    autonumber
+    participant Client as Client (Challenger)
+    participant Provider as Provider Node (HTTP)
+    participant Chain as Parachain (Pallet)
+
+    Note over Client,Provider: Earlier: Upload & Commit (client saves metadata)
+    Client->>Provider: POST /upload (chunked data)
+    Provider-->>Client: data_root
+
+    Client->>Provider: POST /commit {bucket_id, data_roots}
+    Provider->>Provider: Append leaves to MMR, sign CommitmentPayload (leaf_count=0)
+    Provider-->>Client: {mmr_root, start_seq, leaf_indices, provider_signature}
+    Client->>Client: Save leaf_indices, provider_sig, chunk count per leaf
+
+    Note over Client,Chain: Later: Client spot-checks by downloading a chunk
+    Client->>Provider: GET /download?bucket_id=X&leaf=3&chunk=7
+    Provider-->>Client: (no response / wrong data / timeout)
+    Client->>Client: Verification failed — initiate challenge
+
+    Note over Client,Chain: Option A: challenge_checkpoint (on-chain snapshot)
+    Client->>Chain: challenge_checkpoint(bucket_id, provider, leaf_index, chunk_index)
+    Chain->>Chain: Verify provider bit set in snapshot.primary_signers
+    Chain->>Chain: Reserve deposit (100 units) from client
+    Chain->>Chain: Store Challenge at deadline = now + 48h
+    Chain-->>Client: Event: ChallengeCreated{challenge_id, respond_by}
+
+    Note over Client,Chain: Option B: challenge_offchain (saved provider_sig)
+    Client->>Chain: challenge_offchain(bucket_id, provider, mmr_root, start_seq, leaf_index, chunk_index, provider_sig)
+    Chain->>Chain: Verify provider_sig over CommitmentPayload{..., leaf_count=0}
+    Chain->>Chain: Reserve deposit, store Challenge at deadline
+    Chain-->>Client: Event: ChallengeCreated{challenge_id, respond_by}
+```
+
+## How the Client Knows `chunk_index` and `provider_sig`
+
+| Parameter | Source |
+|---|---|
+| `chunk_index` | The client chunked the data itself (256 KiB chunks). Any index from `0` to `ceil(data_size / 256KiB) - 1` is valid. |
+| `leaf_index` | Returned by `POST /commit` in the `leaf_indices` array. |
+| `provider_sig` | Returned by `POST /commit` or `GET /commitment` (signed with `leaf_count=0`). |
+| `mmr_root` | Returned by `POST /commit` or `GET /commitment`. |
+| `start_seq` | Returned by `POST /commit` or `GET /commitment`. |
+
+## Two Challenge Modes
+
+### `challenge_checkpoint` (Recommended)
+
+Uses the on-chain `BucketSnapshot` created by a prior checkpoint. The client only needs `leaf_index` and `chunk_index` — all commitment data (MMR root, provider signatures) is already on-chain.
+
+**Advantages:**
+- No local state required beyond knowing which leaf/chunk to challenge.
+- Any client (or third party) can challenge — the on-chain snapshot is public.
+- Works identically regardless of which device or user agent uploaded the data.
+- Immune to client-side data loss — the chain is the source of truth.
+
+**Requirement:** A checkpoint must exist for the bucket. This is why enabling and funding provider-initiated checkpoints (see [Provider-Initiated Checkpoint](./checkpoint-provider-initiated.md)) is important — it ensures the chain always has a recent snapshot to challenge against.
+
+### `challenge_offchain` (Limited Applicability)
+
+Uses a provider's off-chain commitment signature obtained during the write flow. The client must supply the `mmr_root`, `start_seq`, `leaf_index`, `chunk_index`, and `provider_signature` — all of which were returned in the `POST /commit` response at upload time.
+
+**This mode has significant practical limitations:**
+
+1. **Client must retain the commit response.** The `provider_signature`, `mmr_root`, `start_seq`, and `leaf_indices` are returned in a single HTTP response during upload. If the client does not persist this data, it is lost — the provider has no obligation to re-issue the same signature, and the MMR root changes with every subsequent write.
+
+2. **Client-side data loss defeats the purpose.** If the device that performed the upload is lost, wiped, or the application data is cleared, the client no longer has the signature needed to create a challenge. This is precisely the scenario where a challenge might be most needed (the client cannot verify their data because they have lost their local record of it).
+
+3. **Multi-device / multi-user-agent fragmentation.** When multiple devices (mobile, tablet, browser) write to the same bucket, each write produces a different `POST /commit` response with a different MMR root and provider signature. Device A has no knowledge of the signatures that device B received. This means:
+   - Each device can only challenge based on its own writes.
+   - No single device has a complete picture of the bucket's commitment history.
+   - There is no built-in mechanism to synchronize these ephemeral signatures across devices.
+
+4. **Signatures are point-in-time snapshots.** The provider signature covers the MMR root at the moment of that specific commit. As the bucket grows (more writes from any client or device), the MMR root changes. The old signature remains valid for challenging the specific leaves it covers, but the client must track which signature corresponds to which leaf indices.
+
+**When it might still be useful:** For single-client, single-device scenarios where the application persists commit responses and needs to challenge data that has not yet been checkpointed on-chain. Even then, `challenge_checkpoint` is preferred once a checkpoint exists.
+
+**Bottom line:** `challenge_checkpoint` is the robust, production-ready path. `challenge_offchain` is a lower-level primitive that requires careful client-side state management and does not scale well to multi-device use cases. Clients should ensure checkpoints are regularly created (via provider-initiated or client-initiated flows) so that `challenge_checkpoint` is always available.

docs/sequence_diagrams/challenge-response.md

@@ -0,0 +1,59 @@
+# Challenge Response (Successful Defense)
+
+The provider detects a pending challenge, builds a two-layer proof from its storage backend, and submits it on-chain.
+
+```mermaid
+sequenceDiagram
+    autonumber
+    participant Chain as Parachain (Pallet)
+    participant Provider as Provider Node
+    participant Storage as Storage Backend
+
+    Note over Provider: ChallengeResponder polls every ~6s
+    Provider->>Chain: poll_challenges()
+    Chain-->>Provider: DetectedChallenge{bucket_id, leaf_index, chunk_index, deadline}
+
+    Provider->>Storage: get_mmr_proof(bucket_id, leaf_index)
+    Storage-->>Provider: MmrProof{peaks, leaf{data_root, data_size}, leaf_proof}
+
+    Provider->>Storage: get_chunk_at_index(data_root, chunk_index)
+    Storage-->>Provider: chunk_data, chunk_proof
+
+    Provider->>Chain: respond_to_challenge(challenge_id, Proof{chunk_data, mmr_proof, chunk_proof})
+
+    Chain->>Chain: Verify caller == challenged provider
+    Chain->>Chain: Verify current_block <= deadline
+    Chain->>Chain: chunk_hash = blake2_256(chunk_data)
+    Chain->>Chain: verify_merkle_proof(chunk_hash, chunk_index, chunk_proof, data_root)
+    Chain->>Chain: verify_mmr_proof(mmr_proof, challenged mmr_root)
+    Chain->>Chain: Remove challenge from storage
+
+    Note over Chain: Cost split by response time
+    Chain->>Chain: 1 blk: challenger 90% / provider 10%
+    Chain->>Chain: 2-5: 80/20, 6-24: 70/30, 25-95: 60/40, 96+: 50/50
+    Chain->>Chain: Partial deposit back to challenger, partial slash on provider
+    Chain-->>Provider: Event: ChallengeDefended
+```
+
+## Two-Layer Proof Verification
+
+1. **Chunk in leaf**: `verify_merkle_proof(blake2_256(chunk_data), chunk_index, chunk_proof, data_root)` proves the chunk belongs to the data blob.
+2. **Leaf in MMR**: `verify_mmr_proof(mmr_proof, mmr_root)` proves the leaf (containing `data_root`) is in the MMR the provider committed to.
+
+## Time-Based Cost Split
+
+The faster the provider responds, the more the challenger pays (discouraging frivolous challenges):
+
+| Response Time (blocks) | Challenger Pays | Provider Pays |
+|---|---|---|
+| 1 | 90% | 10% |
+| 2-5 | 80% | 20% |
+| 6-24 | 70% | 30% |
+| 25-95 | 60% | 40% |
+| 96+ | 50% | 50% |
+
+## Alternative Defenses
+
+Besides `Proof`, providers can also respond with:
+- **`Deleted`**: Admin signed a newer `CommitmentPayload` with `start_seq` beyond the challenged leaf (data was legitimately deleted).
+- **`Superseded`**: A newer on-chain snapshot already covers the challenged leaf index (challenge is moot).

docs/sequence_diagrams/challenge-timeout-slashing.md

@@ -0,0 +1,32 @@
+# Challenge Timeout & Slashing
+
+When a provider fails to respond before the deadline, slashing happens automatically in `on_finalize` — no extrinsic needed.
+
+```mermaid
+sequenceDiagram
+    autonumber
+    participant Client as Client (Challenger)
+    participant Chain as Parachain (on_finalize)
+
+    Note over Chain: Block N reaches challenge deadline
+    Chain->>Chain: on_finalize(N): Challenges::take(N)
+    Chain->>Chain: Provider failed to respond in time
+
+    Chain->>Chain: Slash provider ENTIRE stake
+    Chain->>Chain: Refund client full deposit (100 units)
+    Chain->>Chain: Mint 10% of slashed stake to client as reward
+    Chain->>Chain: Remaining 90% burned / treasury
+    Chain->>Chain: provider.stake = 0
+    Chain->>Chain: Increment provider.challenges_failed
+
+    Chain-->>Client: Event: ChallengeSlashed{slashed_amount, challenger_reward}
+```
+
+## Key Details
+
+- **Automatic**: Slashing is triggered by `on_finalize` at the deadline block. No one needs to call an extrinsic.
+- **Full stake loss**: The provider's **entire stake** is slashed (not just a portion).
+- **Challenger incentive**: Full deposit refund + 10% of the slashed stake as reward.
+- **Remaining 90%**: Burned or sent to treasury via the runtime's slash handler.
+- **Provider state**: `provider.stake` is set to 0 and `challenges_failed` is incremented.
+- **Deadline**: `ChallengeTimeout` is configured at 48 hours (in blocks) in the runtime.

docs/sequence_diagrams/checkpoint-client-initiated.md

@@ -0,0 +1,44 @@
+# Client-Initiated Checkpoint
+
+The client orchestrates: collects provider signatures via HTTP, verifies consensus, then submits a single on-chain extrinsic.
+
+```mermaid
+sequenceDiagram
+    autonumber
+    participant Client as Client SDK
+    participant P1 as Provider 1
+    participant P2 as Provider 2
+    participant Chain as Parachain
+
+    Note over Client: submit_checkpoint(bucket_id)
+
+    par Collect signatures
+        Client->>P1: GET /checkpoint-signature?bucket_id=X
+        P1->>P1: Sign CommitmentPayload (sr25519)
+        P1-->>Client: mmr_root, start_seq, leaf_count, sig
+    and
+        Client->>P2: GET /checkpoint-signature?bucket_id=X
+        P2->>P2: Sign CommitmentPayload (sr25519)
+        P2-->>Client: mmr_root, start_seq, leaf_count, sig
+    end
+
+    Client->>Client: Group by mmr_root, check consensus >= 51%
+
+    alt Consensus reached
+        Client->>Chain: checkpoint(bucket_id, mmr_root, start_seq, leaf_count, sigs[])
+        Chain->>Chain: Verify caller is writer/admin
+        Chain->>Chain: Verify each signature against registered pubkeys
+        Chain->>Chain: Check signing_count >= min_providers
+        Chain->>Chain: Create BucketSnapshot
+        Chain-->>Client: Event BucketCheckpointed
+    else No consensus
+        Client-->>Client: CheckpointResult::NoConsensus
+    end
+```
+
+## Key Details
+
+- **CommitmentPayload** is signed with the **real** `leaf_count` (via `/checkpoint-signature`).
+- The client verifies majority consensus (same `mmr_root` from >= 51% of providers) before submitting.
+- On-chain, each signature is verified against the provider's registered public key, and the provider's bit is set in the `primary_signers` bitfield.
+- The resulting `BucketSnapshot` makes providers liable for the data.