perf(chat): eliminate streaming freezes via 16 targeted fixes#11
Merged
Conversation
Five-phase perf pass targeting random lags/freezes during event streaming and idle. Each fix is guarded by a new regression test that asserts the post-fix contract. All 1640 tests pass. Phase 1 - clone + poll hot paths: - createPlainObjectSnapshotFast: structuredClone-only variant for SDK SSE payloads (skips the JSON round-trip the slow variant does) - MessageStreamService.cloneRawEvent uses fast variant (-2 deep traversals per stream event) - MessageComponents: stabilize subagent modal poll deps via refs (was tearing down the 1500ms interval on every stream event) Phase 2 - bounded state: - ChatViewProvider.streamedSubtaskPartsBySessionId cleared on session switch (was leaking per-session entries across the extension lifetime) - StructuredOutputProcessor.clearDiagnostics() added; wired into session-switch paths alongside the existing subagentTracker resets - StreamEventHandler.pendingEvents capped at MAX_PENDING_EVENTS=500 with a shed-oldest policy under backpressure Phase 3 - emit + I/O debouncing: - GeminiTokenUsageTracker.scheduleUsageEmit: 250ms trailing debounce (was sort+emit on every token event) - QuotaService.writeJsonFile is async via fs.promises.writeFile (was fs.writeFileSync on the main thread) Phase 4 - logger + memo + clone: - Logger.wouldEmit() gates sanitizeConsoleValue (was deep-cloning context on every filtered log call - the dominant remaining per-event CPU cost after Phase 1) - ChatViewProvider: drop inline JSON.stringify(next) plan-debug logs - buildWebviewStreamEvent returns the truncated clone directly (no redundant spread of the original event) - SubagentTracker.appendClamped: in-place bounded append (-1 array allocation per subagent event) - ResponseMessage memo: areResponseMessagePropsEqual skips streaming identity changes for non-streaming cards (historical cards no longer rerender on every stream batch) Phase 5 - subagent + history hot paths: - scheduleSubagentProjectionPersist: 500ms trailing debounce (-95% workspaceState writes during active subagent work) - SubagentTracker pending drain rewritten to O(n) index iteration with copyWithin-based compaction (was O(n^2) via pending.shift()) - HistoryProcessor.appendUnique caches seen-set per target via WeakMap (was O(n^2) - rebuilt fingerprint set on every call within a burst) - buildWebviewStreamEvent: hasOversizedStringLeaf fast-path pre-check skips the recursive DFS when no string leaf exceeds the IPC cap (typical text streams no longer pay for truncation they don't need) Bug fixes bundled: - SubagentTracker pending drain now correctly consumes every examined entry (matches original shift() semantics, including dropping stale already-bound entries). Initial index-based rewrite preserved stale entries - caught during self-review. - Updated existing tests to match new contracts: gemini-token-tracker-performance, message-stream-service, stream-event-main-thread-performance, live-stream-response-rendering, streaming-stop-button-sync. Verification: - npm test: 1640 pass / 0 fail / 109 skipped - npm run typecheck: clean - npm run lint: 0 errors (426 pre-existing warnings unchanged) - npm run webview:build: clean - npm run structured-output:check: in sync
|
Caution Review failedThe pull request is closed. ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (38)
📝 WalkthroughWalkthroughThis PR updates streaming performance and lifecycle handling across the provider, services, SDK adapter, and webview. It adds bounded collections, deferred persistence and emissions, faster cloning, asynchronous quota writes, synthetic message metadata, timeout preservation, and live-turn rendering safeguards. ChangesProvider streaming lifecycle
Backend hot paths
Message and webview rendering
Estimated code review effort: 4 (Complex) | ~60 minutes Sequence Diagram(s)sequenceDiagram
participant ChatViewProvider
participant StreamEventHandler
participant SSE
participant ChatShell
participant ResponseMessage
ChatViewProvider->>StreamEventHandler: process bounded stream events
ChatViewProvider->>SSE: preserve server-side turn after transport timeout
SSE-->>ChatViewProvider: send terminal SSE event
ChatViewProvider->>ChatShell: update live assistant state
ChatShell->>ResponseMessage: render active streaming card
ResponseMessage-->>ChatShell: suppress unrelated card rerenders
Possibly related PRs
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Five-phase perf pass targeting random lags/freezes during event streaming and idle. Each fix is guarded by a new regression test that asserts the post-fix contract. All 1640 tests pass.
Motivation
Users reported random lags and freezes during chat use, both during active AI streaming and when idle. Investigation identified five compounding categories of work on the per-event hot path:
Changes by phase
Phase 1 - clone + poll hot paths
createPlainObjectSnapshotFast: structuredClone-only variant for SDK SSE payloads (skips the JSON round-trip the slow variant does).MessageStreamService.cloneRawEventuses fast variant (-2 deep traversals per stream event).MessageComponents: stabilize subagent modal poll deps via refs (was tearing down the 1500ms interval on every stream event).Phase 2 - bounded state
ChatViewProvider.streamedSubtaskPartsBySessionIdcleared on session switch (was leaking per-session entries across the extension lifetime).StructuredOutputProcessor.clearDiagnostics()added; wired into session-switch paths alongside the existingsubagentTrackerresets.StreamEventHandler.pendingEventscapped atMAX_PENDING_EVENTS=500with a shed-oldest policy under backpressure.Phase 3 - emit + I/O debouncing
GeminiTokenUsageTracker.scheduleUsageEmit: 250ms trailing debounce (was sort+emit on every token event).QuotaService.writeJsonFileis async viafs.promises.writeFile(wasfs.writeFileSyncon the main thread).Phase 4 - logger + memo + clone
Logger.wouldEmit()gatessanitizeConsoleValue(was deep-cloning context on every filtered log call - the dominant remaining per-event CPU cost after Phase 1).ChatViewProvider: drop inlineJSON.stringify(next)plan-debug logs.buildWebviewStreamEventreturns the truncated clone directly (no redundant spread of the original event).SubagentTracker.appendClamped: in-place bounded append (-1 array allocation per subagent event).ResponseMessagememo:areResponseMessagePropsEqualskips streaming identity changes for non-streaming cards (historical cards no longer re-render on every stream batch).Phase 5 - subagent + history hot paths
scheduleSubagentProjectionPersist: 500ms trailing debounce (-95% workspaceState writes during active subagent work).SubagentTrackerpending drain rewritten to O(n) index iteration withcopyWithin-based compaction (was O(n^2) viapending.shift()).HistoryProcessor.appendUniquecaches seen-set per target viaWeakMap(was O(n^2) - rebuilt fingerprint set on every call within a burst).buildWebviewStreamEvent:hasOversizedStringLeaffast-path pre-check skips the recursive DFS when no string leaf exceeds the IPC cap (typical text streams no longer pay for truncation they don't need).Bug fixes bundled
SubagentTrackerpending drain now correctly consumes every examined entry (matches originalshift()semantics, including dropping stale already-bound entries). Initial index-based rewrite preserved stale entries - caught during self-review.gemini-token-tracker-performance,message-stream-service,stream-event-main-thread-performance,live-stream-response-rendering,streaming-stop-button-sync.Test coverage
tests/regression/perf-*.test.mjs.Verification
npm test: 1640 pass / 0 fail / 109 skippednpm run typecheck: cleannpm run lint: 0 errors (426 pre-existing warnings unchanged)npm run webview:build: cleannpm run structured-output:check: in syncRisk assessment
ResponseMessagememo (Phase 4.5): historical cards no longer re-render on stream batches. If a card needs streaming-derived data it must be the active streaming card; otherwise it skips. The card-not-currently-streaming case is verified by the existing chat-message-flow and chat-view-streaming tests.How to validate
[STREAM-PERF]logs - batch sizes and durations should be roughly half of pre-PR.__OC_PERF_PROBE__ = truein webview DevTools - per-action dispatch timings andFRAME-BLOWwarnings should be rare or absent.Summary by CodeRabbit
Bug Fixes
Performance
Tests