Skip to content

Reduce idle RPC usage#1440

Open
Jurij89 wants to merge 1 commit into
mainfrom
codex/rpc-call-reduction
Open

Reduce idle RPC usage#1440
Jurij89 wants to merge 1 commit into
mainfrom
codex/rpc-call-reduction

Conversation

@Jurij89

@Jurij89 Jurij89 commented Jul 4, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Reduces idle JSON-RPC volume by giving counted ethers providers a static network after one live eth_chainId validation, eliminating repeated steady-state chain-id detection while still catching wrong-chain or unavailable RPC endpoints.
  • Splits chain event polling into independent lanes for publish confirmation, context graph discovery, VM reconciliation, collection updates, allow-list updates, and profile events; ChainEventLaneRunner owns cadence, cursor persistence, lazy lane restoration, and conservative legacy cursor compatibility.
  • Adds explicit cache primitives for hot chain reads: TtlValueCache retains bounded positive values, KeyedSingleFlight coalesces in-flight work with stale-write guards, PCA reverse-agent authorization is single-flight only for publish-critical freshness, and PCA write paths centralize post-success invalidation.
  • Adds a targeted devnet/rpc-quiet-window suite for idle rpc_usage budget validation.

Expected RPC impact

The investigation baseline from the Base mainnet node was 62,820 counted raw RPCs over about 218.5 minutes, or roughly 288 calls/minute. The largest steady-state contributors were eth_getLogs (26,819, about 123/minute), eth_chainId (16,777, about 77/minute), and eth_call (15,079, about 69/minute).

Expected quiet-window behavior after this PR:

  • eth_chainId: near-zero steady-state calls for numeric EVM chain configs after one validation per endpoint; the devnet budget is <=5/minute, a ~93% reduction versus the Base average and ~98% versus the observed 333/minute peak window.
  • eth_getLogs: most idle 12s ticks return before getBlockNumber; only due lanes scan, and context graph discovery moves to a slower independent cadence. The devnet budget is <=20/minute, an ~84% reduction versus the Base average and ~91% versus the observed 215/minute peak window.
  • Total quiet-window RPCs: the devnet gate budgets <=120 calls/minute per idle daemon, at least ~58% below the Base average before accounting for workload-specific eth_call cache wins.
  • eth_call: exact reduction is workload-dependent, but duplicate identity/PCA reads collapse to one in-flight request per key and short TTL window; RPC failures are not cached.

The Gnosis sample remains useful failover evidence, but its malformed hosts were an intentional failover test rather than the expected production baseline.

Scope note

This PR does not change random-sampling-specific code or the random sampling prover/bind implementation. Random sampling may indirectly benefit from shared chain-adapter provider behavior, but its core challenge/proof flow and random-sampling RPC logic are not intentionally modified here.

Related

Follow-up issues intentionally left out of this PR to avoid late churn:

Files changed

File What
packages/chain/src/rpc-usage.ts Allows counted providers to receive an optional static network
packages/chain/src/evm-adapter-base.ts Parses configured chain ids, initializes static-network providers, validates chain id once, owns identity/PCA cache state and named PCA invalidators
packages/chain/src/keyed-ttl-single-flight-cache.ts Adds separate TtlValueCache and KeyedSingleFlight primitives with non-storing zero-TTL semantics and epoch-guarded in-flight invalidation
packages/chain/src/evm-adapter-identity.ts Adds positive identity caching, negative in-flight coalescing, and local seed/clear hooks
packages/chain/src/evm-adapter-conviction.ts Composes PCA account-info caching, single-flight reverse-agent reads, and centralized PCA write invalidation
packages/publisher/src/chain-event-lane-runner.ts Extracts lane scheduling, head seeding, lazy cursor restore, scanning, and persistence mechanics out of the poller
packages/publisher/src/chain-event-poller.ts Defines lane descriptors and keeps event-domain dispatch/handler logic
packages/chain/test/*, packages/publisher/test/* Adds static-network, cache primitive, identity/PCA cache, op-wallet mutation, poller lane, cursor, lazy-restore, legacy-save, and misc dispatch regression tests
devnet/rpc-quiet-window/*, devnet/suites.json, package.json, pnpm-* Adds registered idle devnet RPC budget suite and workspace metadata

Note: investigation/planning artifacts remain local under ignored agent-docs/ for this Codex run, but are intentionally not part of the repository diff.

Test plan

  • pnpm --filter @origintrail-official/dkg-core --filter @origintrail-official/dkg-storage --filter @origintrail-official/dkg-chain --filter @origintrail-official/dkg-publisher run build
  • pnpm --dir packages/chain exec vitest run --config vitest.unit.config.ts test/keyed-ttl-single-flight-cache.unit.test.ts test/rpc-usage.unit.test.ts test/evm-adapter.unit.test.ts test/evm-adapter-pca-rpc.unit.test.ts test/evm-adapter-op-wallet.unit.test.ts --reporter dot
  • pnpm --dir packages/publisher exec vitest run test/chain-event-poller-v10-only.test.ts test/chain-event-poller-ka-registered.test.ts test/chain-event-poller-ka-created.test.ts test/chain-event-poller-extra.test.ts --reporter dot
  • pnpm test:devnet:manifest -- --reporter dot
  • pnpm exec vitest run --config devnet/rpc-quiet-window/vitest.config.ts -t "parses the committed" --reporter dot
  • GitHub PR checks are passing or intentionally skipped, including Bura: cli, Kosava: node-ui e2e, and Tornado: core + storage + chain
  • Live WSL devnet quiet-window gate: RPC_QUIET_WINDOW_MS=185000 pnpm test:devnet:rpc-quiet-window on an idle publisher-enabled devnet
  • Windows testnet validation using isolated DKG_HOME, the prepared funded config bundle, and sanitized rpc_usage evidence

Comment thread packages/chain/src/evm-adapter-base.ts Outdated
Comment thread packages/publisher/src/chain-event-poller.ts Outdated
Comment thread agent-docs/rpc-call-reduction-team-prompt.md Outdated
Comment thread packages/publisher/src/chain-event-poller.ts Outdated
Comment thread packages/chain/src/evm-adapter-base.ts Outdated
Comment thread packages/chain/src/evm-adapter-base.ts Outdated
Comment thread packages/publisher/src/chain-event-poller.ts Outdated
Comment thread packages/chain/src/evm-adapter-base.ts Outdated
@Jurij89 Jurij89 force-pushed the codex/rpc-call-reduction branch from e2c3412 to 53f98b0 Compare July 4, 2026 20:47
Comment thread packages/chain/src/evm-adapter-identity.ts
Comment thread packages/chain/src/evm-adapter-conviction.ts Outdated
Comment thread packages/publisher/src/chain-event-poller.ts Outdated
@Jurij89 Jurij89 force-pushed the codex/rpc-call-reduction branch from 53f98b0 to 97fbbff Compare July 4, 2026 20:57
Comment thread packages/chain/src/rpc-usage.ts
Comment thread packages/chain/src/keyed-ttl-single-flight-cache.ts Outdated
Comment thread packages/chain/src/evm-adapter-conviction.ts Outdated
@Jurij89 Jurij89 force-pushed the codex/rpc-call-reduction branch from 97fbbff to 76acdd8 Compare July 4, 2026 21:11
Comment thread packages/chain/src/keyed-ttl-single-flight-cache.ts Outdated
@Jurij89 Jurij89 force-pushed the codex/rpc-call-reduction branch from 76acdd8 to 4c6d4d7 Compare July 4, 2026 21:25
Comment thread packages/chain/test/rpc-usage.unit.test.ts
Comment thread packages/publisher/src/chain-event-poller.ts Outdated
Comment thread packages/chain/src/evm-adapter-base.ts Outdated
@Jurij89 Jurij89 force-pushed the codex/rpc-call-reduction branch from 4c6d4d7 to 36e2874 Compare July 4, 2026 21:47
Comment thread packages/publisher/src/chain-event-poller.ts Outdated
Comment thread packages/chain/src/evm-adapter-conviction.ts Outdated
Comment thread packages/publisher/src/chain-event-lane-runner.ts Outdated
@Jurij89 Jurij89 force-pushed the codex/rpc-call-reduction branch from 36e2874 to 1a1828e Compare July 4, 2026 22:04
Comment thread packages/publisher/src/chain-event-lane-runner.ts Outdated
Comment thread packages/publisher/src/chain-event-lane-runner.ts Outdated
@Jurij89 Jurij89 force-pushed the codex/rpc-call-reduction branch from 1a1828e to 615fa99 Compare July 4, 2026 22:21
Comment thread packages/chain/src/evm-adapter-base.ts
@Jurij89 Jurij89 force-pushed the codex/rpc-call-reduction branch from 615fa99 to 36a6397 Compare July 4, 2026 22:55
Comment thread packages/publisher/src/chain-event-lane-cursor-store.ts Outdated
Comment thread packages/chain/src/evm-adapter-base.ts Outdated
Comment thread packages/publisher/src/chain-event-lane-cursor-store.ts Outdated
Comment thread packages/publisher/test/chain-event-poller-ka-created.test.ts Outdated
@Jurij89 Jurij89 force-pushed the codex/rpc-call-reduction branch from 36a6397 to c2aaa9b Compare July 4, 2026 23:16
Comment thread devnet/rpc-quiet-window/automated.test.ts Outdated
Comment thread packages/publisher/test/chain-event-poller-ka-created.test.ts Outdated
@Jurij89 Jurij89 force-pushed the codex/rpc-call-reduction branch from c2aaa9b to a170194 Compare July 4, 2026 23:33
Comment thread packages/chain/src/keyed-ttl-single-flight-cache.ts Outdated
Comment thread packages/chain/src/keyed-ttl-single-flight-cache.ts
@Jurij89 Jurij89 force-pushed the codex/rpc-call-reduction branch from a170194 to c232660 Compare July 5, 2026 00:02
Comment thread packages/publisher/src/chain-event-lane-runner.ts
Comment thread packages/publisher/test/chain-event-lane-runner.unit.test.ts
Comment thread packages/publisher/src/chain-event-poller.ts Outdated
@Jurij89 Jurij89 force-pushed the codex/rpc-call-reduction branch from c232660 to 130dd5a Compare July 5, 2026 00:18
Comment thread packages/publisher/test/chain-event-lane-runner.unit.test.ts
Comment thread packages/publisher/src/chain-event-poller.ts Outdated
@Jurij89 Jurij89 force-pushed the codex/rpc-call-reduction branch 2 times, most recently from 63fb8e2 to 2dba6cb Compare July 5, 2026 01:04
Comment thread packages/chain/src/evm-adapter-base.ts Outdated
@Jurij89 Jurij89 force-pushed the codex/rpc-call-reduction branch from 2dba6cb to 008a791 Compare July 5, 2026 01:15
Comment thread packages/chain/src/evm-adapter-conviction.ts Outdated
Comment thread packages/publisher/src/chain-event-lane-cursor-store.ts Outdated
@Jurij89 Jurij89 force-pushed the codex/rpc-call-reduction branch 3 times, most recently from e201ee9 to 89b725f Compare July 5, 2026 02:57
Comment thread packages/chain/src/evm-adapter.ts Outdated
@Jurij89 Jurij89 force-pushed the codex/rpc-call-reduction branch from 89b725f to 1078681 Compare July 5, 2026 03:14
Comment thread packages/chain/src/pca-read-cache.ts Outdated
Comment thread packages/chain/src/pca-read-cache.ts
@Jurij89 Jurij89 force-pushed the codex/rpc-call-reduction branch 2 times, most recently from 2ec1364 to b4ed271 Compare July 5, 2026 13:56
Comment thread packages/chain/src/evm-adapter-base.ts
Comment thread packages/publisher/src/chain-event-poller.ts
Comment thread packages/chain/src/rpc-failover-client.ts
Comment thread packages/publisher/src/chain-event-poller.ts
@Jurij89 Jurij89 force-pushed the codex/rpc-call-reduction branch from 0e81242 to 07473e4 Compare July 5, 2026 15:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants