Skip to content

Backend: Persist transaction backfill jobs with resumable progress checkpoints #863

Description

@Junirezz

Context

transactionBackfill.ts replays Soroban contract events for a ledger range to repair indexer gaps. Jobs and progress are tracked in in-memory Map structures.

Problem / Gap

Long-running backfill jobs are lost on process restart. Operators cannot resume partially completed ledger scans or audit historical backfill runs after deploys.

Proposed approach

  • Persist backfill job metadata and progress checkpoints in Prisma.
  • Resume from the last completed ledger batch on restart.
  • Expose job status via existing admin backfill endpoints with durable history.
  • Enforce existing MAX_LEDGER_RANGE and MAX_BATCH_SIZE guards.

Acceptance criteria

  • A running backfill survives restart and resumes from the last checkpoint.
  • Completed and failed jobs remain queryable for at least a configurable retention window.
  • Dry-run jobs are persisted separately and never mutate DB state.
  • Tests cover start, mid-run restart simulation, completion, and failure paths.

Files/areas affected

  • backend/src/transactionBackfill.ts
  • backend/src/eventPollingService.ts
  • backend/prisma/schema.prisma

Metadata

Metadata

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions