Skip to content

Use the editor's syntax tree instead of re-parsing the full document#41

Open
moudy wants to merge 1 commit into
mainfrom
zerg/use-editor-syntax-tree
Open

Use the editor's syntax tree instead of re-parsing the full document#41
moudy wants to merge 1 commit into
mainfrom
zerg/use-editor-syntax-tree

Conversation

@moudy

@moudy moudy commented Jul 17, 2026

Copy link
Copy Markdown

Why

TextState.updateImpl re-parses the entire document synchronously — parser.parse(state.doc.toString(), treeFragments) — inside the ViewPlugin update path. This runs unbudgeted on the main thread on mount and again on every qualifying update. On large documents, or when the language parser has pathological cases (e.g. @lezer/markdown < 1.3.2's quadratic GFM autolink scan on long unbroken tokens), a single parse takes multiple seconds and the editor freezes at ~100% CPU whenever the minimap is enabled.

Measured against a real workspace (31KB markdown file containing one 30k-char token, minimap displayText: 'characters'): editor mount blocked the main thread for 7.1s + 3.6s + 3.5s ≈ 15s with a blank pane. With this change the minimap's own parse cost drops to zero (only the editor's budgeted parser runs).

This is the amplifier behind replit/repl-it-web AGI-1427 (Zendesk #469883): CodeMirror's own language integration parses asynchronously with work budgets, so the editor alone survives slow parsers — the minimap's synchronous full parse is what turns a slow parse into a hard freeze, and it double-parses every document even in the happy path.

References AGI-1427

What changed

  • TextState now reads the editor's incrementally-maintained, work-budgeted tree via syntaxTree(state) instead of running its own full parser.parse (removes the _previousTree/TreeFragment machinery and the per-update doc.toString() for parsing).
  • shouldUpdate also fires when syntaxTree(update.state) !== syntaxTree(update.startState), so minimap highlighting converges as the editor's background parse advances (including right after mount, when the tree may only cover part of the document).
  • Behavior with no language configured is unchanged in effect: syntaxTree returns an empty tree, which yields no highlight spans (previously tree was undefined and highlighting was skipped).

Verified in the repl-it-web workspace with this build: pathological file opens instantly, minimap renders with correct highlighting in both blocks and characters modes, and highlighting fills in as background parsing progresses.

~ written by Zerg 👾 (mutated-thor-88c5)

@moudy
moudy marked this pull request as ready for review July 17, 2026 01:46
@moudy
moudy requested a review from Monkatraz July 17, 2026 01:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant