docs: correct multi-worker scaling claims in deployment docs

[arabold]

Summary

• deployment-modes.md and pipeline-architecture.md claimed you can load-balance across multiple worker replicas, or run embedded + external workers together ("hybrid deployment"). Neither is implemented, and it directly contradicted ARCHITECTURE.md's "shared worker" language.
• Verified against the code: PipelineManager's job queue is in-process memory (PipelineManager.ts), only one worker process owns the SQLite-backed document store at a time, and enableWorker/--server-url are mutually exclusive (mcp.ts, web.ts, worker.ts) — so a coordinator always talks to exactly one worker.
• Rewrote the scaling sections in both docs to reflect what's actually true: the stateless web/mcp coordinator tier scales horizontally behind a load balancer; the worker does not (scale it vertically via maxConcurrency instead), in part because SQLite's WAL locking assumes one host's local filesystem, not multiple hosts sharing a file over a network volume.

Test plan

• npm ci (Node 22) + npx biome check on the two changed files — both are excluded from Biome's markdown checks, so no formatting drift
• Pre-commit hook (lint-staged) ran clean
• Manually cross-checked every claim in the reworded sections against PipelineManager.ts, PipelineFactory.ts, mcp.ts, web.ts, worker.ts, and AppServer.ts

Fixes #443

🤖 Generated with Claude Code

[Copilot]

Pull request overview

Updates deployment documentation to remove inaccurate claims about multi-worker/hybrid deployments and align the docs with the current single-worker architecture (in-memory job queue + SQLite-backed store), emphasizing that only the coordinator tier scales horizontally.

Changes:

• Rewrites distributed-mode architecture and scaling guidance to describe horizontal scaling for web/mcp coordinators only.
• Clarifies that the worker process does not scale horizontally and should be scaled vertically (e.g., via maxConcurrency and resources).
• Removes/rewrites “hybrid deployment” guidance to reflect that embedded-worker and external-worker modes are treated as mutually exclusive in normal deployments.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File	Description
docs/infrastructure/deployment-modes.md	Reframes distributed-mode architecture and scaling guidance around a stateless coordinator tier + single worker.
docs/concepts/pipeline-architecture.md	Updates scaling patterns section to clarify horizontal scaling applies to coordinators, not workers, and removes hybrid deployment claims.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.