feat(backend): add extract-text node for PDF/DOCX documents#29
Open
LukasHirt wants to merge 3 commits into
Open
feat(backend): add extract-text node for PDF/DOCX documents#29LukasHirt wants to merge 3 commits into
LukasHirt wants to merge 3 commits into
Conversation
Pure-Go, BSD-3-Clause-licensed PDF parser needed by the upcoming Extract Text node's PDF support. DOCX extraction needs no new dependency (archive/zip + encoding/xml, both stdlib, are enough for that format), so this is the only new module required. Signed-off-by: Lukas Hirt <info@hirt.cz>
Executor.Run has always done vars["file.content"] = string(content)
unconditionally, for every file type. That's correct for .txt/.md, but
for a PDF or .docx an LLM Prompt node referencing {{file.content}}
silently got that binary format's raw bytes stringified instead of the
document's actual text.
Add a new "extractText" node kind (pkg/textextract) that detects a
file's format by extension, falling back to content sniffing, and
converts PDF/DOCX into plain text. Plain-text and unrecognized formats
pass through unchanged. Wired into the executor as runExtractText,
writing to vars["file.text"] by default or the node's configured
outputVariable, mirroring the pattern where llm.output is also written
into result.Output.
Image OCR (scanned documents/photos) is explicitly out of scope: it
needs a bundled OCR engine or an external service call, a meaningfully
bigger lift than parsing an already-textual document format. Left as a
natural follow-up.
Signed-off-by: Lukas Hirt <info@hirt.cz>
Modeled as a new CanvasNodeKind ('extractText') rather than a new
actionType: it has no meaningful "target file" side effect the way
tag/move/comment/etc. do, it's a content-transform step like the LLM
Prompt node, and the LLM node itself is already precedent for a
single-purpose top-level kind with no sub-types. This keeps the
picker, canvas node component, and NDV wiring consistent with how llm
already works instead of overloading the action-node switch.
Adds an AI-category picker entry, a canvas node component
(ExtractTextNode.vue, mirroring LlmNode.vue), and reuses the existing
(previously unused) outputVariable field on WorkflowNodeData for the
optional output-variable override, wired into NodeDetailsPanel's NDV.
Signed-off-by: Lukas Hirt <info@hirt.cz>
LukasHirt
force-pushed
the
feat/extract-text-node
branch
from
July 24, 2026 21:00
1ccbcf9 to
32a191c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements the "Extract Text" node (previously just a design proposal in
docs/superpowers/specs/2026-07-24-new-workflow-node-types-design.md) end to end.Bug this fixes:
Executor.Run(backend/pkg/executor/executor.go) has always donevars["file.content"] = string(content)unconditionally for every file type. That's correct for.txt/.md, but for a PDF or.docx, an LLM Prompt node referencing{{file.content}}silently got that binary format's raw bytes stringified instead of the document's actual text — a real, user-facing gap for a perfectly reasonable "summarize this uploaded PDF" workflow.What's in scope: plain-document text extraction (PDF and DOCX). OCR is explicitly out of scope — recovering text from an image (a scanned PDF, a photo of a page) needs either a bundled OCR engine or a call out to an external service, a meaningfully bigger lift than parsing an already-textual document format. That remains a natural, separate follow-up; nothing here attempts it.
Backend (
backend/pkg/textextract,backend/pkg/executor)textextractpackage:DetectFormat(by extension, falling back to content-sniffing magic bytes/zip structure) andExtract, which converts PDF/DOCX to plain text and passes through anything else (including plain text) unchanged — no behavior change for existing.txt/.mdworkflows.github.com/ledongthuc/pdf— pure Go, BSD-3-Clause (a maintained fork of the Go team's ownrsc.io/pdf), no cgo, license compatible with this project's existing dependencies. License text and NOTICE.md/REUSE.toml entries added;reuse lintpasses.archive/zip+encoding/xml) — a.docxis just a zip with an XML part, and plain-text extraction needs nothing beyond unzip-and-walk-the-XML, so no new dependency was needed for that half.extractText(runExtractText, parallel torunLLM/runAction): reads the samefile.content/file.namevarsRunalready loads, extracts text, and writes it tovars["file.text"]by default ornode.Data["outputVariable"]when set.result.Outputgets a short"N characters extracted"summary (per spec, not the full text — unlikellm.output, which can be arbitrarily large here).Frontend
CanvasNodeKind('extractText'), not a newactionType. Rationale: it has no "target file" side effect the waytag/move/comment/etc. do — it's a content-transform step, like the LLM Prompt node. The LLM node is already precedent in this codebase for a single-purpose top-level kind with no sub-types, so this keeps the picker/canvas/NDV wiring consistent with howllmalready works instead of overloading the action-node switch.file-texticon), a new canvas node component (ExtractTextNode.vue, mirroringLlmNode.vue), and the optional output-variable-override field wired intoNodeDetailsPanel.vue.outputVariablefield already declared (but unused) onWorkflowNodeDatarather than adding a new field — same name, same shape, for consistency. (The LLM node's own NDV doesn't read it yet — this PR doesn't touch that, only extractText does.)Test plan
pkg/textextracttests first (confirmed they failed to compile with no implementation), then implementedDetectFormat/Extract/PDF/DOCX extraction. Tests build minimal-but-valid PDF/DOCX fixtures in-process (real byte-accurate PDF with xref table; real DOCX zip withword/document.xml) rather than checking in binary fixtures.pkg/executor/executor_test.gowithrunExtractTexttests — default output var, custom output var, real end-to-end PDF extraction through the executor, plain-text passthrough, and failure propagation on a corrupt document (execution stops, downstreamllmnode doesn't run).cd backend && go build ./... && go vet ./... && go test ./...— all green.tests/unit/nodeTypes.spec.ts(Extract Text registered inNODE_TYPESunderAI_CATEGORY) andtests/unit/NodeDetailsPanel.spec.ts(output-variable field renders and emits correctly), confirmed both fail before implementation, pass after.pnpm test:unit,pnpm check:types,pnpm lint,pnpm build— all green.reuse lint— compliant (addedLICENSES/BSD-3-Clause.txt,NOTICE.md/REUSE.tomlentries forledongthuc/pdf).🤖 Generated with Claude Code