Repository navigation
fix(parsers): yield the event loop during PDF extraction and CSV row coercion - #9311
Conversation
…coercion pdf.js runs its fake worker on the request thread and resolves every message as a microtask, so the page loop and the text-stream read loop never let timers or I/O run. A shared time-sliced check now yields a macrotask once 10ms have passed, across page reads, stream reads, and page assembly. Table CSV append/replace coerces rows in slices of 2,000 with a yield between slices, and checks the plan row limit right after parsing for both modes, before coercion and before claiming the table job slot.
|
The latest updates on your projects. Learn more about Vercel for GitHub. |
 [Medium impact] The changes appear safe to merge; no blocking issues were found. SummaryThis PR lets other requests and timers run during PDF extraction and large CSV imports.
No actionable new issues were found. No previous review threads were supplied. Reviews (2) · Last reviewed commit: "fix(parsers): yield between PDF assembly..." · Reviewed by Greptile |
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
All reported issues were addressed across 4 files
Heads up: you’re close to your flex budget. Increase your flex budget so reviews don’t pause.
Turn on auto-fix | Re-trigger cubic
|
@cubic-dev-ai review this PR |
@waleedlatif1 I have started the AI code review. It will take a few minutes to complete. |
There was a problem hiding this comment.
No issues found across 4 files
Confidence score: 5/5
- Automated review surfaced no issues in the provided summaries.
- No files require special attention.
Heads up: you’re close to your flex budget. Increase your flex budget so reviews don’t pause.
Turn on auto-fix | Re-trigger cubic
Summary
streamTextContentread loop ever let timers or I/O run. One shared time-sliced check (createEventLoopSlicer,setImmediatefromnode:timers/promises) now yields a macrotask once 10ms have passed since the last yield, in the page loop, inside the stream read loop (covers one page with a huge content stream), and in page assembly, between its whole-document passes and per page (replaces the fixedsleep(0)every 32 pages).coerceRowsForTableis a 1:1 map, so rows andcellsRejectedare identical).User-visible changes:
Type of Change
Testing
pdf-parser-cancellation.test.ts: a timer runs between CPU-bound pages; a timer runs while one page streams many chunks (red with only the inner read-loop yield removed)import.test.ts: a timer runs during coercion of a 50,000-row file before the write; an over-limit append/replace returns the row-limit validation error while the slot is held by another job (a within-limit replace still getsconflict)pdf-parser-cancellation,pdf-parser,pdf-parser-structure,importtests (32 passed),bun run lint. Full type-check/audits/test suite left to CI.cellsRejectedidenticalThe remaining PDF block is a single whole-document assembly pass (furniture suppression or word collection); the CSV residual includes the bench's own hashing of the rows.
Checklist
test-auditauthoring gate)