feat(runner): saturated-window GPS metric + always-emit sequencer results#205
Draft
meyer9 wants to merge 1 commit into
Draft
feat(runner): saturated-window GPS metric + always-emit sequencer results#205meyer9 wants to merge 1 commit into
meyer9 wants to merge 1 commit into
Conversation
…ults Two related changes to make sequencer benchmark results trustworthy and always reported, motivated by running the harness against very large (hundreds of millions of accounts) state where the post-run block-replay phase reliably times out. Always-emit: - The load-test payload worker exiting non-zero after producing a valid result is now treated as a workload failure, not a fatal error: the run keeps its collected sequencer metrics and records a FailureReason instead of discarding everything. - service.runTest salvages GetResult() when the run errors (e.g. the post-run newPayload phase hits a context deadline) so the collected per-block metrics are still emitted with success=false. - RunResult gains FailureReason; GetResult sets Success from the workload outcome rather than hardcoding true. Saturated-window gas/sec: - gasPerSecond is now averaged only over saturated blocks (gas >= 0.5x the median block gas), so warmup/drain blocks no longer dilute the headline throughput number, and it no longer keys off the nominal gas limit (real blocks never reach it). - Adds a regime label (cadence-limited vs overloaded vs unsaturated) derived from mean getPayload latency vs block time, plus a SaturatedBlockCount, so a reader can tell whether the builder kept up. - Records a per-block duration/block_building metric for the aggregate.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two related changes that make sequencer benchmark results trustworthy and always reported. Motivated by running the harness against very large state (hundreds of millions of accounts), where the post-run block-replay phase reliably times out and the current code discards the whole run's metrics as a result.
Always-emit sequencer results
FailureReasoninstead of throwing everything away.service.runTestsalvagesGetResult()when the run errors (e.g. the post-runnewPayloadphase hits a context deadline on huge state), so the collected per-block metrics are still emitted withsuccess=false.RunResultgainsFailureReason;GetResultderivesSuccessfrom the workload outcome instead of hardcodingtrue.Saturated-window gas/sec
gasPerSecondis now averaged only over saturated blocks (gas ≥ 0.5× the median block gas), so warm-up/drain blocks no longer dilute the headline number. It no longer keys off the nominal gas limit (real blocks never reach it, which previously produced a zero).cadence-limited/overloaded/unsaturated) derived from meangetPayloadlatency vs. block time, plus aSaturatedBlockCount, so a reader can tell whether the builder actually kept up.duration/block_buildingmetric used by the aggregate.Notes for review
testing/local-zfs-runner(this change builds on that branch; retarget tomainonce it lands).go build ./...andgo vet ./...clean; my files are gofmt-clean (two unrelated pre-existing unformatted files on the branch are left untouched).Draft for review.