Skip to content

feat: Materialized analytics rollup tables from indexed events #559

Description

@joelpeace48-cell

Background & current state

Computing analytics/leaderboards by scanning raw indexed events is expensive and won't scale to thousands of campaigns/participants. There are no precomputed rollups.

Goal

Maintain materialized rollup tables derived incrementally from indexed events so dashboards and leaderboards read precomputed aggregates in <50ms.

Technical design

  • Rollups: participants_by_day, credits_by_day, claims_by_day, campaign_funnel(registered, credited, claimed), top_participants(campaign_id, address, points).
  • Update incrementally as new events are projected (NEW-024) — apply deltas, not full recompute. Support idempotent reprocessing (recompute a day from raw if needed).
  • Index rollup tables for the exact dashboard/leaderboard queries.

Edge cases

  • Late/out-of-order events → upsert deltas keyed by (campaign, day).
  • Reconciliation (NEW-025) recomputes affected rollup buckets.
  • Time zones → store UTC; document.
  • Ties in leaderboards → deterministic tiebreak (e.g., earliest ledger).

Task breakdown

  • Rollup table migrations + indexes.
  • Incremental delta updates in the projection layer.
  • Recompute-from-raw utility for a bucket.
  • Benchmarks vs. raw scans.

Acceptance criteria

  • Dashboard/leaderboard queries hit rollups (<50ms p95) instead of scanning raw events.
  • Rollups remain correct after reconciliation/late events.

Testing & verification

  • Benchmark before/after; correctness test vs. recompute-from-raw.

Out of scope

Dependencies / related

  • Builds on NEW-024/025; feeds NEW-027, NEW-099.

Difficulty: medium · Effort: M · indexer + performance

Metadata

Metadata

Assignees

Labels

Stellar WaveIssues in the Stellar wave programarea: backendBackend API (Node/Express)difficulty: mediumModerate complexityenhancementNew feature or requestindexerOn-chain event indexingperformancePerf and scalability

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions