Fix OOM in aspire-version-placeholders build hook#1318
Merged
Conversation
The astro:build:done hook re-walked the entire dist tree with a single recursive Promise.all, holding the contents of every .html/.md/.txt file (tens of thousands across all locales, including the large llms-full.txt assets) in memory at once. On the full production build that exhausted the default ~4 GB Node heap and crashed with 'JavaScript heap out of memory'. That broad walk was almost entirely redundant. The remarkAspireVersionPlaceholders remark plugin is already wired into markdown.remarkPlugins, so placeholders are replaced before render: .html pages are correct, and llms*.txt is sourced from rendered HTML via render(entry). The reference/**/*.md endpoints come from API/sample data, not docs content. The only generated artifact that still contains raw placeholders is the per-page .md copies emitted by starlight-page-actions, which viteStaticCopy's raw src/content/docs/** through a regex-only transform that bypasses the remark pipeline. Scope the post-build pass to .md files only and stream them through a bounded worker pool (default concurrency 16). Peak memory is now proportional to the concurrency limit, and the bulk of dist (.html plus the large .txt assets) is no longer re-read. Replacement semantics are unchanged. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR addresses an out-of-memory failure during pnpm build:production by narrowing and throttling the aspire-version-placeholders post-build replacement pass so it no longer reads (and holds) large portions of the dist output concurrently.
Changes:
- Scope the post-build replacement pass to
.mdfiles (intended to targetstarlight-page-actionsMarkdown copies). - Replace unbounded recursive
Promise.alltraversal with a bounded worker-pool concurrency model. - Extend unit tests to lock in the scoping decision and exercise bounded traversal across nested directories.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| src/frontend/config/aspire-version-placeholders-integration.mjs | Restricts placeholder replacement to .md files and processes them with bounded concurrency to avoid OOM. |
| src/frontend/tests/unit/aspire-version-placeholders.vitest.test.ts | Adds tests covering extension scoping and recursive traversal beyond the concurrency limit. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Guard against a non-finite/0/negative concurrency value: normalize to a finite positive integer (falling back to the default) before computing the worker count, so a stray NaN can't collapse the pool to an empty array and silently skip every file. Adds a regression test passing NaN. - Reword the scoping test comment so it no longer implies the seeded .html/.txt/.mdx fixtures were already replaced; clarify the assertion is that this pass intentionally leaves every non-.md extension untouched. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Contributor
Frontend HTML artifact readyThe latest frontend build uploaded the This comment updates automatically when a new frontend build artifact is uploaded. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
pnpm build:productiononrelease/13.5(e.g. CI for #1233) crashes during theaspire-version-placeholdersintegration'sastro:build:donehook:Example run: https://github.com/microsoft/aspire.dev/actions/runs/28447182778/job/84299577028
Root cause
replaceAspireVersionPlaceholdersInDirectorywalked the entiredisttree with a single recursivePromise.all. Every directory recursion and every filereadFilestarted concurrently, so the contents of every.html/.md/.txtfile across all locales — tens of thousands of files, including the largellms-full.txtassets — were held in memory simultaneously, exhausting the default ~4 GB Node heap.Why the walk was mostly redundant
The replacement work itself is trivial (only two placeholders). Tracing where placeholders actually need replacing:
dist?.htmlpagesmarkdown.remarkPlugins(remarkAspireVersionPlaceholdersruns before expressive-code renders code blocks)llms*.txtstarlight-llms-txtsources rendered HTML viarender(entry)reference/**/*.md*.mdcopiesstarlight-page-actionsviteStaticCopys rawsrc/content/docs/**through a regex-only transform that bypasses remarkThe
remarkAspireVersionPlaceholdersplugin is already wired in and is the canonical, render-time, memory-safe mechanism. The only generated artifact that still carries raw%ASPIRE_VERSION%placeholders is the per-page.mdcopies emitted bystarlight-page-actions.Fix
.mdfiles only (the page-actions copies). The bulk ofdist(.html+ the large.txtassets) is no longer re-read..mdfiles through a small worker pool (default 16) instead of an unbounded recursivePromise.all. Peak memory is now proportional to the concurrency limit, not the total file count.Replacement semantics are unchanged.
Tests
Extended
tests/unit/aspire-version-placeholders.vitest.test.ts:.mdcopies are replaced;.html/.txt/.mdx/.jsonare left untouched (locks in the scoping decision).vitest4/4 pass; ESLint clean.Co-authored-by: Copilot App 223556219+Copilot@users.noreply.github.com