-
Notifications
You must be signed in to change notification settings - Fork 176
feat(solana-indexer): PR 2 — add types module scaffolding
#4506
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
tilacog
wants to merge
24
commits into
main
Choose a base branch
from
solana-indexer/PR2-bootstrap
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
24 commits
Select commit
Hold shift + click to select a range
5b9b6a5
feat(solana-indexer): add types module scaffolding
tilacog a938616
toml fmt
tilacog c7b2dbc
fix(ci): reformat TOML files with tombi 1.1.0
tilacog bfb4dcf
fix(solana-indexer): include slot fields in ReplayWindowExceeded erro…
tilacog 6e1f41e
refactor(solana-indexer): rename as_str to as_label on Commitment and…
tilacog 0fef0ed
refactor(solana-indexer): drop unused PartialEq+Eq from Commitment
tilacog f57158f
docs(solana-indexer): remove org/spec references from StreamUpdate co…
tilacog 72146c4
Concise commitment-level doc comments in solana-indexer
tilacog 399c2ec
docs: tighten commitment module doc-comment, name each type's role
tilacog cd72ab5
refactor(solana-indexer): use bytes::Bytes for opaque byte blobs
tilacog 57b5c76
Add `Slot` newtype for Solana ledger slots
tilacog c7936f4
refactor(solana-indexer): introduce OrderUid newtype for order identi…
tilacog 3d65b11
fix(toml): alphabetize bytes/derive_more in solana-indexer/Cargo.toml
tilacog f6f7cb8
Use u64 for auction_id in solana SettlementFinalized domain type
tilacog 64166cb
docs(solana-indexer): clarify that TradeDelta.order_fulfilled is infe…
tilacog fe154a4
nit: fmt comments
tilacog 271539d
docs(solana-indexer): clarify SolFlowEvent::OrderEnabled doc comment
tilacog 6b2e8a1
chore(solana-indexer): Remove metrics module
tilacog 362e86e
restrict types visibility to pub(crate) and suppress dead_code warnings
tilacog dfc1272
refactor(solana-indexer): extract Slot and OrderUid into dedicated mo…
tilacog 47f8edf
fix: ix_index u8 → u16
tilacog 135042e
Merge branch 'main' into solana-indexer/PR2-bootstrap
squadgazzz 5092df3
Update cargo.lock
squadgazzz 9f05387
Merge remote-tracking branch 'origin/main' into solana-indexer/PR2-bo…
squadgazzz File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1 +1,3 @@ | ||
| //! `solana-indexer` — Solana settlement indexer. | ||
|
|
||
| pub mod types; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,63 @@ | ||
| #![expect(dead_code)] | ||
| //! Message types passed over the internal channels. | ||
| //! | ||
| //! The ingester pushes [`StreamUpdate`] into the channel to the decoder; the | ||
| //! decoder pushes [`PartialEvent`] / [`PartialHalf`] to the partial-event | ||
| //! watchdog. | ||
|
|
||
| use crate::types::{ | ||
| Signature, | ||
| slot::Slot, | ||
| wire::{SubscribeUpdateAccountInfo, SubscribeUpdateTransactionInfo}, | ||
| }; | ||
|
|
||
| /// From `Ingester` → `Decoder`. | ||
| /// | ||
| /// One multiplexed wire message, tagged with the slot the message was observed | ||
| /// at. | ||
| #[derive(Debug, Clone)] | ||
| pub(crate) enum StreamUpdate { | ||
| /// A transaction-update slot message. | ||
| Tx { | ||
| /// Slot the message was observed at. | ||
| slot: Slot, | ||
| /// Transaction signature. | ||
| signature: Signature, | ||
| /// Wire message body. | ||
| inner: Box<SubscribeUpdateTransactionInfo>, | ||
| }, | ||
| /// An account-update slot message. | ||
| Account { | ||
| /// Slot the message was observed at. | ||
| slot: Slot, | ||
| /// Optional signature linking the write back to its originating | ||
| /// transaction. | ||
| txn_signature: Option<Signature>, | ||
| /// Wire message body. | ||
| inner: Box<SubscribeUpdateAccountInfo>, | ||
| }, | ||
| } | ||
|
|
||
| /// From `Decoder` → `PartialEventWatchdog`. | ||
| /// | ||
| /// The watchdog holds incomplete `(slot, signature)` pairs until both halves | ||
| /// arrive; each delivery carries the half that just landed. | ||
| #[derive(Debug, Clone, Copy)] | ||
| pub(crate) struct PartialEvent { | ||
| /// Slot the partial was observed at. | ||
| pub slot: Slot, | ||
| /// Transaction signature the partial corresponds to. | ||
| pub signature: Signature, | ||
| } | ||
|
|
||
| /// One of the two halves a [`StreamUpdate`] can produce. | ||
| /// | ||
| /// The decoder pushes one `PartialEvent` per `StreamUpdate` it processes; the | ||
| /// watchdog uses the `(slot, signature)` key to match pairs. | ||
| #[derive(Debug, Clone)] | ||
| pub(crate) enum PartialHalf { | ||
| /// Transaction-update half. | ||
| Tx(Box<SubscribeUpdateTransactionInfo>), | ||
| /// Account-update half. | ||
| Account(Box<SubscribeUpdateAccountInfo>), | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,81 @@ | ||
| #![expect(dead_code)] | ||
| //! Commitment-tracking types. | ||
| //! | ||
| //! This module holds the types we use to track how far a transaction has | ||
| //! progressed through Solana's commitment pipeline, plus the row shapes the | ||
| //! finalization worker reads and writes. | ||
| //! | ||
| //! The indexer captures transactions at `confirmed` commitment. A later | ||
| //! finalization pass polls `getSignatureStatuses` (whose result is modeled by | ||
| //! [`SignatureStatus`]) and either promotes the row to `finalized` or marks it | ||
| //! `rolled_back`. [`UnfinalizedRow`] is the shape the finalization worker | ||
| //! queries for when sweeping aged confirmed rows, and [`AccountInfo`] holds | ||
| //! account snapshots used for recovery when accounts aren't obtained normally | ||
| //! through the ingestion stream. | ||
|
|
||
| use { | ||
| crate::types::{Signature, slot::Slot}, | ||
| bytes::Bytes, | ||
| solana_sdk::pubkey::Pubkey, | ||
| }; | ||
|
|
||
| /// Commitment level persisted by the indexer. | ||
| /// | ||
| /// Solana consensus defines `processed`, `confirmed`, and `finalized` | ||
| /// commitment levels, but we only store the two durable states plus a terminal | ||
| /// failure state for abandoned slots. `processed` is omitted because it | ||
| /// reflects the node's latest view and is still rollback-prone. | ||
| #[derive(Debug, Clone, Copy)] | ||
| pub(crate) enum Commitment { | ||
| /// Voted on by a supermajority but can still be rolled back. Watched by the | ||
| /// finalization worker. | ||
| Confirmed, | ||
| /// Rooted by the cluster and considered permanently settled. | ||
| Finalized, | ||
| /// Never landed, or its slot was abandoned by the cluster. | ||
| RolledBack, | ||
| } | ||
|
|
||
| impl Commitment { | ||
| /// String label used in `solana.*` `commitment` columns. | ||
| pub fn as_label(self) -> &'static str { | ||
| match self { | ||
| Self::Confirmed => "confirmed", | ||
| Self::Finalized => "finalized", | ||
| Self::RolledBack => "rolled_back", | ||
| } | ||
| } | ||
|
jmg-duarte marked this conversation as resolved.
|
||
| } | ||
|
|
||
| /// Result of an RPC `getSignatureStatuses` poll. | ||
| #[derive(Debug, Clone, Copy)] | ||
| pub(crate) struct SignatureStatus { | ||
| /// Slot the transaction landed at, if known. | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. That this data can be missing seems to indicate that this might better be an enum. |
||
| pub slot: Slot, | ||
| /// Confirmation status reported by the RPC. | ||
| pub confirmation_status: Commitment, | ||
| } | ||
|
|
||
| /// Snapshot of an account at a given slot (from `getAccountInfo`). | ||
| #[derive(Debug, Clone)] | ||
| pub(crate) struct AccountInfo { | ||
| /// Slot the snapshot was read at. | ||
| pub slot: Slot, | ||
| /// Account data (serialized). | ||
| pub data: Bytes, | ||
| /// Account owner program. | ||
| pub owner: Pubkey, | ||
| } | ||
|
|
||
| /// A `solana.*` row that has not yet reached `finalized` commitment — the kind | ||
| /// picked up by the aged-row sweep, where `commitment = 'confirmed'` and the | ||
| /// row's slot is at least one finalization window behind `LATEST_CHAIN_SLOT`. | ||
| #[derive(Debug, Clone)] | ||
| pub(crate) struct UnfinalizedRow { | ||
| /// Table the row lives in. | ||
| pub table: &'static str, | ||
| /// Transaction signature. | ||
| pub signature: Signature, | ||
| /// Slot the row was inserted at. | ||
| pub slot: Slot, | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,47 @@ | ||
| #![expect(dead_code)] | ||
| //! Dead-letter types: events that failed to persist and were diverted to | ||
| //! `solana.dead_letter` for operator follow-up. | ||
|
|
||
| use { | ||
| crate::types::{Signature, slot::Slot}, | ||
| bytes::Bytes, | ||
| }; | ||
|
|
||
| /// A decoded event whose write to `solana.*` failed and was diverted to | ||
| /// `solana.dead_letter`. | ||
| #[derive(Debug, Clone)] | ||
| pub(crate) struct DeadLetterEntry { | ||
| /// Slot the event was observed at. | ||
| pub slot: Slot, | ||
| /// Transaction signature, if the failure was per-transaction. | ||
| pub signature: Option<Signature>, | ||
| /// Why the event landed in the dead-letter table. | ||
| pub reason: DeadLetterReason, | ||
| /// Original raw bytes for replay. | ||
| pub raw_bytes: Bytes, | ||
| } | ||
|
|
||
| /// Why a row landed in the dead-letter table. | ||
| #[derive(Debug, Clone, Copy, PartialEq, Eq)] | ||
| pub(crate) enum DeadLetterReason { | ||
| /// Decoder received both halves but couldn't parse them. | ||
| DecoderError, | ||
| /// Watchdog gave up: account-update half never arrived. | ||
| AccountUpdateMissing, | ||
| /// Watchdog gave up: transaction-update half never arrived. | ||
| TxUpdateMissing, | ||
| /// Settlement landed but no `proposed_solutions` row matched. | ||
| SolutionUidUnmatchable, | ||
| } | ||
|
|
||
| impl DeadLetterReason { | ||
| /// String label used in `solana.dead_letter.reason`. | ||
| pub fn as_label(self) -> &'static str { | ||
| match self { | ||
| Self::DecoderError => "decoder_error", | ||
| Self::AccountUpdateMissing => "account_update_missing", | ||
| Self::TxUpdateMissing => "tx_update_missing", | ||
| Self::SolutionUidUnmatchable => "solution_uid_unmatchable", | ||
| } | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,58 @@ | ||
| #![expect(dead_code)] | ||
| //! Error types used across the indexer's domain. | ||
|
|
||
| use {crate::types::slot::Slot, thiserror::Error}; | ||
|
|
||
| /// Failures surfaced from the decoder. | ||
| #[derive(Debug, Error, PartialEq, Eq)] | ||
| pub(crate) enum DecodeError { | ||
| /// The discriminator byte(s) at the start of the instruction data did not | ||
| /// match any known instruction on either program. | ||
| #[error("unknown instruction discriminator")] | ||
| UnknownDiscriminator, | ||
| /// The ALT (Address Lookup Table) loaded-address list could not be resolved | ||
| /// against the full account list. | ||
| #[error("alt resolution failed")] | ||
| AltResolutionFailed, | ||
| /// The instruction was recognised but its schema did not match the on-chain | ||
| /// layout. | ||
| #[error("schema mismatch")] | ||
| SchemaMismatch, | ||
| } | ||
|
|
||
| /// Failures surfaced from the persistence boundary. | ||
| #[derive(Debug, Error, PartialEq, Eq)] | ||
| pub(crate) enum StoreError { | ||
| /// The SQL `ON CONFLICT` clause rejected the write (e.g. watermark | ||
| /// regression). | ||
| #[error("store conflict")] | ||
| Conflict, | ||
| /// The store is temporarily unavailable (e.g. connection lost, pool | ||
| /// exhausted). The caller is expected to retry. | ||
| #[error("store unavailable")] | ||
| Unavailable, | ||
| } | ||
|
|
||
| /// Failures surfaced from the stream boundary. | ||
| #[derive(Debug, Error)] | ||
| pub(crate) enum StreamError { | ||
| /// The stream has been disconnected by the server. | ||
| #[error("stream disconnected")] | ||
| Disconnected, | ||
| /// The internal mpsc send timed out (backpressure on the decoder). | ||
| #[error("stream send timeout")] | ||
| SendTimeout, | ||
| /// The resume slot is outside the provider's replay window. The caller | ||
| /// should reset `from_slot` to `LATEST_CHAIN_SLOT − replay_window`, | ||
| /// record the lost range, and retry the subscription. | ||
| #[error( | ||
| "replay window exceeded: attempted slot {attempted_slot}, earliest replayable \ | ||
| {earliest_replayable_slot}" | ||
| )] | ||
| ReplayWindowExceeded { | ||
| /// The slot the subscriber attempted to resume from. | ||
| attempted_slot: Slot, | ||
| /// The earliest slot the provider can still serve. | ||
| earliest_replayable_slot: Slot, | ||
| }, | ||
| } |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The doc comment for
PartialEventstates that "each delivery carries the half that just landed," but the struct definition is missing thehalf: PartialHalffield. Without this field, the watchdog cannot receive or hold the actual data halves to match and reconstruct the full event. Additionally, sincePartialHalfcontains heap-allocatedBoxtypes,PartialEventcannot deriveCopyand should only deriveDebug, Clone.