-
Notifications
You must be signed in to change notification settings - Fork 176
Isolate pool-indexer flyway migrations from the shared DB set #4555
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
AryanGodara
wants to merge
2
commits into
main
Choose a base branch
from
aryan/pool-indexer-migration-isolation
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+236
−90
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,99 @@ | ||
| # Pool-indexer migrations | ||
|
|
||
| Flyway migrations for the pool-indexer's own per-network database (e.g. | ||
| `ink_pool_indexer`), kept out of the shared `../sql/` set so they don't run | ||
| against the autopilot/orderbook main DBs. | ||
|
|
||
| The migration image ships both dirs; init containers pick one via `-locations`: | ||
|
|
||
| | DB | location | | ||
| |---------------------|-----------------------------------------------------| | ||
| | autopilot/orderbook | `/flyway/sql` (default) | | ||
| | pool-indexer | `-locations=filesystem:/flyway/sql-pool-indexer` | | ||
|
AryanGodara marked this conversation as resolved.
|
||
|
|
||
| New pool-indexer migrations go here, never in `../sql/`. `V110` is duplicated | ||
| from `../sql/` on purpose: the shared copy can't be deleted (Flyway checksums | ||
| applied migrations) so it's cancelled there by `../sql/V111`. | ||
|
|
||
| ## Schema | ||
|
|
||
| The tables below live in the indexer's own per-network database (e.g. | ||
| `ink_pool_indexer`), created by the migrations in this directory. | ||
|
|
||
| ### pool\_indexer\_checkpoints | ||
|
|
||
| Highest finalized block processed per `contract_address` by `pool-indexer`. `contract_address` is the factory address. The indexer runs one process per network against its own DB, so there's no `chain_id` column. | ||
|
|
||
| Column | Type | Nullable | Details | ||
| --------------------|--------|----------|-------- | ||
| contract\_address | bytea | not null | Factory address (20 bytes) | ||
| block\_number | bigint | not null | | ||
|
|
||
| Indexes: | ||
| - PRIMARY KEY: btree (`contract_address`) | ||
|
|
||
| ### uniswap\_v3\_pools | ||
|
|
||
| One row per pool discovered from a `PoolCreated` event. `token{0,1}_{decimals,symbol}` are nullable and filled in by the backfill task. `factory` partitions the table when multiple V3-compatible factories run on the same network so each indexer touches only its own rows. | ||
|
|
||
| Column | Type | Nullable | Details | ||
| -------------------|----------|----------|-------- | ||
| address | bytea | not null | Pool address (20 bytes) | ||
| factory | bytea | not null | Address of the V3 factory that emitted `PoolCreated` | ||
| token0 | bytea | not null | | ||
| token1 | bytea | not null | | ||
| fee | int | not null | Hundredths of a basis point (500 = 0.05%, 3000 = 0.3%, 10000 = 1%). `CHECK (fee > 0)`. | ||
| token0\_decimals | smallint | nullable | `NULL` = not yet fetched. `-1` = sentinel for "fetched but call failed" | ||
| token1\_decimals | smallint | nullable | | ||
| token0\_symbol | text | nullable | `NULL` = not yet fetched. `""` = sentinel for "fetched but call failed" | ||
| token1\_symbol | text | nullable | | ||
| created\_block | bigint | not null | Block in which the pool was created on-chain | ||
|
|
||
| Indexes: | ||
| - PRIMARY KEY: btree (`address`) | ||
| - Four partial indexes on `(token{0,1})` with predicate `token{0,1}_{symbol,decimals} IS NULL` to power the backfill scan. | ||
|
|
||
| ### uniswap\_v3\_pool\_states | ||
|
|
||
| Current state per pool: `sqrt_price_x96` and `tick` come from the latest `Swap`/`Initialize`; `liquidity` and `block_number` also update on in-range `Mint`/`Burn`. FK → `uniswap_v3_pools`. | ||
|
|
||
| **Uniswap V3 pool-state primer.** Three values capture a pool's instantaneous state: | ||
|
|
||
| - `sqrt_price_x96` — `sqrt(price) * 2^96` where `price = token1/token0`, stored in Q64.96 fixed-point. The square-root form keeps swap math additive and bounds precision loss over the uint160 range. Mirrors on-chain `slot0.sqrtPriceX96`. | ||
| - `tick` — `floor(log_{1.0001}(price))`. Each tick is a ~0.01% price step; the current tick is the bucket the live price falls into. Routers use it to decide which positions are in-range. | ||
| - `liquidity` — sum of every position's liquidity whose `tickLower <= current_tick < tickUpper`. This is the `L` in V3's invariant `Δsqrt_price = Δamount / L`. Updates on `Swap` (the event carries the new value) and on `Mint`/`Burn` whose range spans the current tick. | ||
|
|
||
| The per-tick deltas that move `liquidity` when the price crosses a tick boundary live in [`uniswap_v3_ticks`](#uniswap_v3_ticks). | ||
|
|
||
| Column | Type | Nullable | Details | ||
| -------------------|---------|----------|-------- | ||
| pool\_address | bytea | not null | FK → `uniswap_v3_pools(address)` | ||
| block\_number | bigint | not null | Block of the most recent state-changing event (`Swap`, `Initialize`, or in-range `Mint`/`Burn`). | ||
| sqrt\_price\_x96 | numeric | not null | uint160 — see primer above | ||
| liquidity | numeric | not null | uint128 — see primer above | ||
| tick | int | not null | signed int24 — see primer above | ||
|
|
||
| Indexes: | ||
| - PRIMARY KEY: btree (`pool_address`) | ||
|
|
||
| ### uniswap\_v3\_ticks | ||
|
|
||
| Per-tick liquidity deltas. Rows with `liquidity_net = 0` are pruned. FK → `uniswap_v3_pools`. | ||
|
|
||
| **Why deltas instead of per-tick totals.** A V3 position covers `[tickLower, tickUpper)` and contributes to pool liquidity only when the current tick is in that range. We store the entering / exiting deltas at the bounds: | ||
|
|
||
| - At `tickLower`: `liquidity_net += position.liquidity` (entering) | ||
| - At `tickUpper`: `liquidity_net -= position.liquidity` (exiting) | ||
|
|
||
| When a swap crosses a tick boundary, the pool's `liquidity` shifts by `± tick.liquidity_net`. This encoding makes the per-tick aggregate O(1) at swap time — no per-position iteration. | ||
|
|
||
| Quoters consult these to predict liquidity changes at tick crossings during swap simulation. Without them, large swaps would be priced as if the liquidity stayed flat, producing wildy wrong quotes | ||
|
|
||
| Column | Type | Nullable | Details | ||
| ----------------|---------|----------|-------- | ||
| pool\_address | bytea | not null | FK → `uniswap_v3_pools(address)` | ||
| tick\_idx | int | not null | Tick coordinate (signed int24); same domain as [`uniswap_v3_pool_states.tick`](#uniswap_v3_pool_states) | ||
| liquidity\_net | numeric | not null | int128, signed — net liquidity entering (+) / exiting (-) at this tick | ||
|
|
||
| Indexes: | ||
| - PRIMARY KEY: btree (`pool_address`, `tick_idx`) | ||
59 changes: 59 additions & 0 deletions
59
database/sql-pool-indexer/V110__pool_indexer_uniswap_v3.sql
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,59 @@ | ||
| -- Tracks the highest finalized block fully processed per factory contract. | ||
| -- A DB instance hosts a single network, so no `chain_id` column is needed. | ||
| CREATE TABLE pool_indexer_checkpoints ( | ||
| contract_address BYTEA NOT NULL, -- factory address | ||
| block_number BIGINT NOT NULL, | ||
| PRIMARY KEY (contract_address) | ||
| ); | ||
|
|
||
| -- One row per pool, discovered from `PoolCreated` events. `factory` | ||
| -- partitions the table so multiple V3-compatible factories on the same | ||
| -- network can coexist (logs are fetched chain-wide, then partitioned at | ||
| -- the write boundary). | ||
| CREATE TABLE uniswap_v3_pools ( | ||
| address BYTEA NOT NULL, -- pool address | ||
| factory BYTEA NOT NULL, | ||
| token0 BYTEA NOT NULL, | ||
| token1 BYTEA NOT NULL, | ||
| fee INT NOT NULL CHECK (fee > 0), -- hundredths of a basis point (500 = 0.05%, 3000 = 0.3%, 10000 = 1%) | ||
| token0_decimals SMALLINT, | ||
| token1_decimals SMALLINT, | ||
| token0_symbol TEXT, | ||
| token1_symbol TEXT, | ||
| created_block BIGINT NOT NULL, | ||
| PRIMARY KEY (address) | ||
| ); | ||
|
|
||
| -- Current state per pool. `sqrt_price_x96` + `tick` come from the latest | ||
| -- Swap/Initialize; `liquidity` + `block_number` also update on in-range | ||
| -- Mint/Burn events. | ||
| CREATE TABLE uniswap_v3_pool_states ( | ||
| pool_address BYTEA NOT NULL, | ||
| block_number BIGINT NOT NULL, | ||
| sqrt_price_x96 NUMERIC NOT NULL, -- uint160 | ||
| liquidity NUMERIC NOT NULL, -- uint128 | ||
| -- `tick` here means the Uniswap V3 *price tick index* (signed | ||
| -- int24), not a database index. See `uniswap_v3_ticks.tick_idx`. | ||
| tick INT NOT NULL, | ||
| PRIMARY KEY (pool_address), | ||
| FOREIGN KEY (pool_address) REFERENCES uniswap_v3_pools(address) | ||
| ); | ||
|
|
||
| -- Active ticks per pool. Rows with `liquidity_net = 0` are pruned. | ||
| -- `tick_idx` is the price tick coordinate (signed int24) — same domain | ||
| -- as `uniswap_v3_pool_states.tick`, one row per active tick boundary. | ||
| CREATE TABLE uniswap_v3_ticks ( | ||
| pool_address BYTEA NOT NULL, | ||
| tick_idx INT NOT NULL, | ||
| liquidity_net NUMERIC NOT NULL, -- int128 (can be negative) | ||
| PRIMARY KEY (pool_address, tick_idx), | ||
| FOREIGN KEY (pool_address) REFERENCES uniswap_v3_pools(address) | ||
| ); | ||
|
|
||
| -- Symbol/decimals backfill hot paths. Partial on `IS NULL` so each | ||
| -- index shrinks to near-empty once most rows are populated (real value | ||
| -- or the `""` / `-1` "tried, failed" sentinel). | ||
| CREATE INDEX ON uniswap_v3_pools (token0) WHERE token0_symbol IS NULL; | ||
| CREATE INDEX ON uniswap_v3_pools (token1) WHERE token1_symbol IS NULL; | ||
| CREATE INDEX ON uniswap_v3_pools (token0) WHERE token0_decimals IS NULL; | ||
| CREATE INDEX ON uniswap_v3_pools (token1) WHERE token1_decimals IS NULL; |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.