Espresso 3b: TEE batcher (re-hosted)#459
Conversation
| return err | ||
| } | ||
| bs.TxManager = txManager | ||
| if err := bs.initChainSigner(); err != nil { |
There was a problem hiding this comment.
Mirrored from #447 (unresolved review thread) to keep tracking it here after the re-host. Original location: op-batcher/batcher/service.go:390.
Original transcript:
@chatgpt-codex-connector — 2026-05-26:
Guard ChainSigner initialization behind Espresso mode
initTxManagernow always callsinitChainSigner, butbs.TxManageris created as*txmgr.SimpleTxManager, which does not implementopcrypto.ChainSigner; this makes service startup fail with"tx manager does not implement ChainSigner"even when--espresso.enabled=false. Because this path runs during normal batcher boot, it becomes a universal startup regression rather than an Espresso-only failure.Useful? React with 👍 / 👎.
There was a problem hiding this comment.
To use Codex here, create a Codex account and connect to github.
| } | ||
|
|
||
| func (c CLIConfig) Check() error { | ||
| if c.Enabled { |
There was a problem hiding this comment.
Mirrored from #447 (unresolved review thread) to keep tracking it here after the re-host. Original location: espresso/cli.go:200.
Original transcript:
@chatgpt-codex-connector — 2026-05-26:
Validate fallback-auth lead time outside Espresso-enabled block
FallbackAuthLeadTimeis validated only whenc.Enabledis true, but fallback mode (--espresso.enabled=false) still uses this field inisFallbackAuthRequired; a negative duration therefore bypasses validation and is later cast touint64, producing a huge lead that can force premature/always-on fallback auth gating. This creates hard-to-diagnose behavior for fallback batchers with a mis-set duration flag.Useful? React with 👍 / 👎.
There was a problem hiding this comment.
To use Codex here, create a Codex account and connect to github.
| batcherAddr := l.Txmgr.From() | ||
|
|
||
| isActive := (activeIsEspresso && l.Config.Espresso.Enabled) || | ||
| (!activeIsEspresso && !l.Config.Espresso.Enabled) |
There was a problem hiding this comment.
Mirrored from #447 (unresolved review thread) to keep tracking it here after the re-host. Original location: op-batcher/batcher/espresso_active.go:52.
Original transcript:
@chatgpt-codex-connector — 2026-05-26:
Check configured signer address in active-batcher decision
isBatcherActivedetermines activity solely fromactiveIsEspressovsEspressoEnabled, without verifying that this node’sTxmgr.From()matches the currently authorized batcher address for that mode. In fallback mode, a node with the wrong sender key will still consider itself active and keep attempting publishes that revert atBatchAuthenticator, causing avoidable failure loops and no progress if this is the only running batcher.Useful? React with 👍 / 👎.
There was a problem hiding this comment.
To use Codex here, create a Codex account and connect to github.
| // sendTxWithEspresso uses the txmgr queue to send the given transaction candidate after setting | ||
| // its gaslimit. It will block if the txmgr queue has reached its MaxPendingTransactions limit. | ||
| func (l *BatchSubmitter) sendTxWithEspresso(txdata txData, isCancel bool, candidate *txmgr.TxCandidate, queue TxSender[txRef], receiptsCh chan txmgr.TxReceipt[txRef]) { | ||
| transactionReference := txRef{id: txdata.ID(), isCancel: isCancel, isBlob: txdata.daType == DaTypeBlob} |
There was a problem hiding this comment.
Mirrored from #447 (unresolved review thread) to keep tracking it here after the re-host. Original location: op-batcher/batcher/espresso.go:1344.
Original transcript:
@jcortejoso — 2026-05-29:
Would be possible to include here daType and tx size?:
type txRef struct { id txID isCancel bool isBlob bool daType DaType size int }
f840eee to
12b46a6
Compare
c78ed9f to
319b3ab
Compare
12b46a6 to
f8480f8
Compare
319b3ab to
3e7dcd9
Compare
9330de1 to
1616378
Compare
6cf6a1f to
eb6ff32
Compare
1616378 to
25a6c63
Compare
| // | ||
| // This method compares the batcher's own role (Config.Espresso.Enabled) | ||
| // against the contract's activeIsEspresso flag. | ||
| func (l *BatchSubmitter) isBatcherActive(ctx context.Context) (bool, error) { |
| // ChainSigner interface and stores the embedded ChainSigner on the service. | ||
| // Espresso uses ChainSigner to sign batch authentication payloads sent to the | ||
| // BatchAuthenticator contract; the cast is required by every Espresso path. | ||
| func (bs *BatcherService) initChainSigner() error { |
There was a problem hiding this comment.
I think this breaks batcher startup entirely. initTxManager (op-batcher/batcher/service.go:389) always calls initChainSigner, which asserts bs.TxManager to opcrypto.ChainSigner. But bs.TxManager is always a *txmgr.SimpleTxManager, which has neither Sign nor SignTransaction, so the assertion always fails and every batcher start — espresso or not — errors with "tx manager does not implement ChainSigner". The driver tests miss it because they build BatchSubmitter directly and skip service init.
The only ChainSigner implementations here (clientSigner, privateKeySigner in op-service/crypto/espresso.go) aren't wired into production — their only caller is a test.
Fix: build a real ChainSigner instead of casting the tx manager, or at least gate initChainSigner behind cfg.Espresso.Enabled so non-espresso batchers still start.
There was a problem hiding this comment.
i just added a flag for non espresso mode
700f549
| go l.receiptsLoop(l.wg, receiptsCh) // ranges over receiptsCh channel | ||
| go l.publishingLoop(l.killCtx, l.wg, receiptsCh, publishSignal) // ranges over publishSignal, spawns routines which send on receiptsCh. Closes receiptsCh when done. | ||
| go l.blockLoadingLoop(l.shutdownCtx, l.wg, unsafeBytesUpdated, publishSignal) // sends on unsafeBytesUpdated (if throttling enabled), and publishSignal. Closes them both when done | ||
| if l.Config.Espresso.Enabled { |
There was a problem hiding this comment.
throttlingLoop starts by default (LowerThreshold defaults to 3,200,000) and just ranges over unsafeBytesUpdated with no context select. The non-espresso path sends on that channel and closes it on exit, but the espresso loops never touch it.
Two consequences: DA throttling silently does nothing in TEE mode, and on shutdown throttlingLoop blocks forever on the never-closed channel, so the wg.Wait in StopBatchSubmitting (op-batcher/batcher/driver.go:298) never returns — admin_stopBatcher or SIGTERM hangs until SIGKILL.
Fix: feed and close unsafeBytesUpdated from the espresso path like the non-espresso one
There was a problem hiding this comment.
This i may need you to take a close look:
abb7ee1
After every block loaded from the streamer I send it to the throttling loop now to check if it should throttle.
| func (l *BlockLoader) reset(ctx context.Context) { | ||
| l.prevSyncStatus = nil | ||
| l.queuedBlocks = nil | ||
| l.batcher.clearState(ctx) |
There was a problem hiding this comment.
This looks like a data race on the streamer. reset runs in the queueing loop and calls clearState, which resets the streamer via resetEspressoStreamer (writes readPos). Meanwhile the loading loop calls Peek/Update/Next on the same streamer without holding channelMgrMutex, and BufferedEspressoStreamer has no internal locking. So the two goroutines touch readPos and the batches slice concurrently — -race will flag it.
Beyond the race, a reset landing mid-iteration rewinds readPos, so the loading loop re-peeks already-consumed batches, re-adds a duplicate block, and AddL2Block returns ErrReorg — which triggers another clearState, churning in a loop.
Fix: serialize streamer access. Either don't reset the streamer from the queueing loop (signal the loading loop to do it), or guard every streamer call in both loops with one mutex.
There was a problem hiding this comment.
I made the espressoBatchQueueingLoop requests the clearState as suggested.
| go l.publishingLoop(l.killCtx, l.wg, receiptsCh, publishSignal) // ranges over publishSignal, spawns routines which send on receiptsCh. Closes receiptsCh when done. | ||
| go l.blockLoadingLoop(l.shutdownCtx, l.wg, unsafeBytesUpdated, publishSignal) // sends on unsafeBytesUpdated (if throttling enabled), and publishSignal. Closes them both when done | ||
| if l.Config.Espresso.Enabled { | ||
| if err := l.startEspressoLoops(receiptsCh, publishSignal); err != nil { |
There was a problem hiding this comment.
The throttling loop is started here, before startEspressoLoops runs its fallible setup (registerBatcher, resolveTEEVerifierAddress). If startEspressoLoops returns an error, StartBatchSubmitting returns without unwinding, so this goroutine is left on the waitgroup. running also stays true, so admin_startBatcher won't restart, and StopBatchSubmitting's wg.Wait blocks on this leaked loop (it ranges over unsafeBytesUpdated, which the espresso path never closes — see the other comment), so stop deadlocks too. The batcher is wedged until a process restart.
The earlier error returns (waitForL2Genesis, waitNodeSync) don't have this problem because they happen before any goroutine is started, so StopBatchSubmitting's wg.Wait returns immediately and recovers cleanly.
Fix: start the throttling loop after the espresso/non-espresso loops instead of before them. The send to it is non-blocking (select with a default in sendToThrottlingLoop), so it doesn't need to be running first. Then an error return from startEspressoLoops leaves no goroutine on the waitgroup, matching the earlier returns and keeping the failure recoverable.
|
|
||
| // NOTE: This function MUST guarantee no transient errors. It is allowed to fail only on | ||
| // invalid batches or in case of misconfiguration of the batcher, in which case it should fail | ||
| // for all batches. |
There was a problem hiding this comment.
ToBlock doesn't validate the batch, and that turns a malformed-but-signed batch into a livelock for every TEE batcher on the namespace.
The encode side (BlockToEspressoBatch) checks that the first tx is an L1-info deposit, but the decode side never re-checks it: UnmarshalEspressoTransaction only recovers the signer and RLP-decodes, and ToBlock just re-inserts b.L1InfoDeposit as txs[0] without checking IsDepositTx() or that the header matches the batch body. An authorized signer — which includes the non-TEE fallback batcher key, a hot key on an ordinary host — can post a batch that is validly signed but semantically malformed (first tx not a deposit, or L1-info that doesn't parse). It passes the signature check and the header-only peek.
In the loading loop this hits the wrong error path. When ToBlock fails, the loop calls Next() and continues — it skips the bad batch. When AddL2Block fails, it calls clearState + streamer Reset and breaks — it rewinds. Because ToBlock does no validation it succeeds on the malformed batch, so the failure only surfaces one step later in AddL2Block (via BlockToSingularBatch rejecting the non-deposit tx), which takes the reset path. Reset rewinds to the safe head without advancing past the batch, so the next tick re-peeks the same batch, ToBlock succeeds again, AddL2Block fails again, reset again — forever. The safe head never advances, and since the namespace is shared every TEE batcher consuming it is stuck the same way until someone intervenes.
This is exactly what the NOTE here already asks for — ToBlock should fail only on invalid batches, so a bad batch takes the skip path rather than the reset path. The implementation just doesn't do the checking.
Fix: validate in ToBlock (or UnmarshalEspressoTransaction) — reject the batch when !b.L1InfoDeposit.IsDepositTx(), when the L1-info data fails to parse, or when BatchHeader is inconsistent with the decoded SingularBatch (parent hash, number, timestamp). Then a malformed-but-signed batch fails in ToBlock and the loop steps over it instead of resetting onto it.
There was a problem hiding this comment.
let me know what you think about this fix + tests
b531ec9
Bring in op-service/crypto/espresso.go (ChainSigner interface unifying SignTransaction and arbitrary-data signing) and op-service/signer/espresso.go (SignerClient.Sign wrapper around eth_sign). Required by the Espresso batcher to sign batch-authentication payloads with either a remote signer or a local private key, in addition to the existing transaction-signing path. Co-authored-by: OpenCode <noreply@opencode.ai>
Introduce op-node/rollup/derive/espresso_batch.go defining EspressoBatch (a SingularBatch with block number, L1 info deposit transaction, and signer address attached), along with BlockToEspressoBatch and the unmarshaler used by the streamer. Also pulls in the github.com/EspressoSystems/espresso-network/sdks/go dependency, which provides the Espresso transaction and namespace types. Consumed by the Espresso batcher (next commits) to convert L2 blocks into batches submitted to Espresso, and to round-trip those batches back through the streamer. Co-authored-by: OpenCode <noreply@opencode.ai>
Bring in the parts of the espresso/ shared package that are TEE-only: - espresso/cli.go: full CLI flag set for --espresso.enabled, query service URLs, light-client/L1 endpoints, batch-authenticator address, receipt-verification tuning, namespace/origin-height parameters used to construct the espresso-streamer. (The --espresso.fallback-auth-lead-time flag lives in op-batcher/flags/flags.go and was added by the fallback PR.) - espresso/interface.go: EspressoStreamer[B] interface that wraps github.com/EspressoSystems/espresso-streamers/op.BatchStreamer. - espresso/ethclient.go: AdaptL1BlockRefClient adapter (used by cli.go to construct the streamer) and FetchEspressoBatcherAddress helper. Also adds the EspressoSystems/espresso-network/sdks/go and EspressoSystems/espresso-streamers Go module dependencies. The regenerated BatchAuthenticator bindings already live in the fallback PR's espresso/bindings/. Co-authored-by: OpenCode <noreply@opencode.ai>
Bring in op-batcher/enclave/attestation.go: a thin wrapper around the hf/nsm library that obtains an AWS Nitro NSM attestation document over a given public key. Used by the Espresso batcher (next commit) to attach a TEE attestation to its registration with the BatchAuthenticator. Adds the github.com/hf/nsm dependency. Builds on all platforms; NSM device access is only attempted at runtime when invoked from inside a Nitro enclave. Co-authored-by: OpenCode <noreply@opencode.ai>
Add the Espresso TEE batcher write-path on top of the fallback batcher: - op-batcher/batcher/espresso.go: Espresso submission loop (peeks the channel manager, converts each L2 block to an EspressoBatch, submits it to Espresso, waits for inclusion, and then posts the batch txs to L1 with TEE-attested BatchAuthenticator.authenticateBatchInfo calls). - op-batcher/batcher/espresso_service.go: EspressoBatcherConfig, initEspresso (Espresso client / light-client construction, optional TEE attestation gathering), and the initChainSigner hook that wraps the txmgr into a opcrypto.ChainSigner. - op-batcher/batcher/espresso_helpers_test.go and espresso_transaction_submitter_test.go: unit tests for the helpers and the TEE transaction submitter. Extends the existing fallback wiring: - op-batcher/batcher/espresso_driver.go: adds EspressoDriverSetup fields (Client/LightClient/ChainSigner/SequencerAddress/Attestation), batcherL1Adapter, setupEspressoStreamer, startEspressoLoops, resetEspressoStreamer; extends dispatchAuthenticatedSendTx with the TEE branch (always authenticates when Espresso.Enabled). - op-batcher/batcher/espresso_active.go: adds isBatcherActive (queries BatchAuthenticator.activeIsEspresso to gate publishing against this batcher's role). - op-batcher/batcher/driver.go: extends DriverSetup with the Espresso EspressoDriverSetup field; adds espressoSubmitter / espressoStreamer / teeVerifierAddress / degradedLog fields on BatchSubmitter; calls setupEspressoStreamer in NewBatchSubmitter; branches StartBatchSubmitting on Espresso.Enabled to call startEspressoLoops; calls resetEspressoStreamer in clearState. - op-batcher/batcher/service.go: BatcherConfig.Espresso field; EspressoClient / EspressoLightClient / ChainSigner / Attestation runtime fields; initEspresso / initChainSigner / applyEspressoDriverSetup call-outs. - op-batcher/batcher/config.go: thread Espresso espresso.CLIConfig through CLIConfig. - op-batcher/flags/flags.go: register espresso.CLIFlags (TEE-only flags; the --espresso.fallback-auth-lead-time flag added by the fallback PR continues to live in op-batcher/flags/flags.go). Also adds op-service/log/repeat_state.go (RepeatStateLogger) and its test, used by the Espresso submission loop's tick-driven warnings. A safeTestRecorder helper is inlined into the test to avoid pulling in the unrelated debouncer. Adds the github.com/hf/nitrite dependency (transitively required by hf/nsm for attestation document parsing). Co-authored-by: OpenCode <noreply@opencode.ai>
The TEE batcher's Espresso submission path called Txmgr.Send directly for both the authenticateBatchInfo tx and the batch inbox tx, and ran inside authGroup, so it bypassed MaxPendingTransactions, assigned nonces nondeterministically (violating Holocene's in-order L1 inclusion requirement), and never checked whether the auth tx reverted — a reverted authenticateBatchInfo emits no event, so the verifier would silently drop the batch. Submit both txs through the ordered queue.Send path on the publishing-loop goroutine (auth first, batch second) so the auth tx takes the lower nonce and is mined first, and both stay under MaxPendingTransactions. A watcher goroutine (tracked by authGroup) collects both receipts, fails the pair if the auth tx reverted, runs the lookback-window check, and emits a single synthetic receipt. This is the same fix already applied to the fallback path; extract the shared submission + receipt-watching flow into submitAuthenticatedBatch / watchAuthReceipts so both paths reuse it and differ only in how the authenticateBatchInfo calldata is built (TEE-attested signature vs empty signature). Rename fallback_auth.go to espresso_auth.go to reflect the shared scope, and restructure the tests: one suite drives the shared flow directly, plus per-path tests asserting the distinguishing auth calldata (empty sig vs a recoverable EIP-712 signature). Co-authored-by: OpenCode <noreply@opencode.ai>
Port TestBatchRoundtrip from the integration branch and fold it into espresso_batch_test.go. It is the only test covering ToEspressoTransaction and the batcher->derivation serialization path; it asserts the decoded batch matches the original and that the recovered signer is the batcher. Also drop the decodedBlock.ExecutionWitness() comparison in TestEspressoBatchConversion: that method does not exist on the op-geth types.Block pinned in the rebase-18 base, so go vet of the derive_test package failed to build. EspressoBatch/ToBlock carries no execution witness. Co-authored-by: OpenCode <noreply@opencode.ai>
eb6ff32 to
ab13906
Compare
Based on #448
Pulls in the Espresso/TEE batcher .
op-batcher/enclave/attestation.goand modifies the driver to add an Espresso path. Bulk of the changes is inespresso_-prefixed files.This is #447, re-hosted from an in-repo branch (now properly stacked).