Skip to content

feat(runner): saturated-window GPS metric + always-emit sequencer results#205

Draft
meyer9 wants to merge 1 commit into
testing/local-zfs-runnerfrom
fix/basebench-saturated-metrics
Draft

feat(runner): saturated-window GPS metric + always-emit sequencer results#205
meyer9 wants to merge 1 commit into
testing/local-zfs-runnerfrom
fix/basebench-saturated-metrics

Conversation

@meyer9

@meyer9 meyer9 commented Jul 20, 2026

Copy link
Copy Markdown
Collaborator

Summary

Two related changes that make sequencer benchmark results trustworthy and always reported. Motivated by running the harness against very large state (hundreds of millions of accounts), where the post-run block-replay phase reliably times out and the current code discards the whole run's metrics as a result.

Always-emit sequencer results

  • A load-test payload worker that exits non-zero after producing a valid result is now treated as a workload failure, not a fatal error: the run keeps its collected sequencer metrics and records a FailureReason instead of throwing everything away.
  • service.runTest salvages GetResult() when the run errors (e.g. the post-run newPayload phase hits a context deadline on huge state), so the collected per-block metrics are still emitted with success=false.
  • RunResult gains FailureReason; GetResult derives Success from the workload outcome instead of hardcoding true.

Saturated-window gas/sec

  • gasPerSecond is now averaged only over saturated blocks (gas ≥ 0.5× the median block gas), so warm-up/drain blocks no longer dilute the headline number. It no longer keys off the nominal gas limit (real blocks never reach it, which previously produced a zero).
  • Adds a regime label (cadence-limited / overloaded / unsaturated) derived from mean getPayload latency vs. block time, plus a SaturatedBlockCount, so a reader can tell whether the builder actually kept up.
  • Records a per-block duration/block_building metric used by the aggregate.

Notes for review

Draft for review.

…ults

Two related changes to make sequencer benchmark results trustworthy and
always reported, motivated by running the harness against very large
(hundreds of millions of accounts) state where the post-run block-replay
phase reliably times out.

Always-emit:
- The load-test payload worker exiting non-zero after producing a valid
  result is now treated as a workload failure, not a fatal error: the run
  keeps its collected sequencer metrics and records a FailureReason
  instead of discarding everything.
- service.runTest salvages GetResult() when the run errors (e.g. the
  post-run newPayload phase hits a context deadline) so the collected
  per-block metrics are still emitted with success=false.
- RunResult gains FailureReason; GetResult sets Success from the workload
  outcome rather than hardcoding true.

Saturated-window gas/sec:
- gasPerSecond is now averaged only over saturated blocks (gas >= 0.5x
  the median block gas), so warmup/drain blocks no longer dilute the
  headline throughput number, and it no longer keys off the nominal gas
  limit (real blocks never reach it).
- Adds a regime label (cadence-limited vs overloaded vs unsaturated)
  derived from mean getPayload latency vs block time, plus a
  SaturatedBlockCount, so a reader can tell whether the builder kept up.
- Records a per-block duration/block_building metric for the aggregate.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant