refactor(interpreter): model stdin as a single shared StdinStream

[a0preetham]

Fixes stdin position tracking in loops and compound commands:

while IFS= read -r line; do
  echo "$line" | cat   # cat consumed all remaining stdin — loop stopped after first line
done <<< $'a\nb\nc'

# Before: a <-- stops here
# After:  a
#         b
#         c

groupStdin was a string snapshot. This fix makes stdin stream-like: one shared object that tracks the offset already read, so any consumer advances the position for everyone.

What

Replaces the interpreter's IOState.groupStdin?: string snapshot with a single StdinStream object — the only representation of stdin anywhere in the interpreter. It works like a bash file descriptor: content bytes plus a read offset, shared by reference. Reading is consuming: a command that drains stdin advances the offset for every other holder of the stream, including across subshell and pipeline-stage boundaries.

Note: this supersedes the earlier StdinCursor design on this branch. That version kept stdin in two parallel forms (a threaded stdin: string parameter plus a cursor in interpreter state) with precedence rules at every consumer, and required each of ~30 commands to remember to call consumeStdin() — a convention-enforced invariant. The redesign makes the invariant structural: ctx.stdin.readAll() is the only way to get the content, so "read stdin but forgot to advance it" is impossible by construction.

Why

The old string-snapshot approach had a recurring bug class: commands inside a while read loop body (pipelines, cat, grep, tr, …) consumed the loop's stdin without advancing the shared position, so the next iteration re-read the same input. The fallback rules (stdin || groupStdin) were duplicated across builtin-dispatch, read, mapfile, and the pipeline glue, and pipeline-execution.ts cleared groupStdin without restoring it.

Design

• src/stdin-stream.ts — StdinStream with readAll() (consume), peek() (forward without consuming; used by timeout/time/env/bash), advance(n) (partial consumers: read, mapfile).
• src/interpreter/stdin-redirect.ts — one resolver for <, <<, <<-, <<<, <&, shared by simple commands, all compound commands (if/for/while/until/case/subshell/group), and function-def redirects; plus withStdin() for scoped install/restore. This replaces four diverged copies of the redirect-processing loop and fixes the heredoc UTF-8 byte-encoding those copies were missing.
• Pipeline stages install a fresh stream per stage; the first stage inherits the enclosing stream by reference (bash fd-inheritance semantics). Empty pipe output can no longer fall back to the enclosing scope's stdin.
• CommandContext.stdin is now a StdinStream (breaking, TypeScript): commands call ctx.stdin.readAll(); StdinStream is exported from the package root. A changeset (minor) documents the migration.

Behavior fixes verified against real bash: subshells share the stdin offset with the parent; function definition redirects apply per call and win over piped stdin; a missing redirect file on a function errors without running the body; diff - - compares stdin against itself; cat - - reads stdin once.

Test coverage

• src/interpreter/while-loop-stdin.test.ts — 40 tests covering stdin redirects (<, <<, <<-, <<<, file-not-found) on all six compound commands, pipelines inside loop bodies, and read position tracking. Carried over from the previous design and passing unchanged — the rewrite preserves all of its behavior.
• src/comparison-tests/stdin-consumption.comparison.test.ts — 15 new comparison tests with fixtures recorded from real bash: consumption by cat/grep/tr in loop and group bodies, subshell offset sharing, pipeline-stage isolation, piped-loop position keeping, function redirect-vs-pipe precedence, missing-file redirect errors, cat - -, diff - -, empty-pipe no-fallback, nested loop redirects, and eval consuming group stdin.
• src/stdin-stream.test.ts — 12 unit tests for the stream primitive itself (consume/peek/advance/exhaustion/shared-reference semantics).
• Full suite: pnpm typecheck, lint, knip, spec tests (3825 passed), comparison tests (553 passed), and unit tests (13k+) all green. The only failures in a full run are pre-existing python3/WASM tests caused by unmaterialized git-LFS blobs in the local checkout (identical on main).

🤖 Generated with Claude Code

[vercel]

@mysorepreetham is attempting to deploy a commit to the Vercel Labs Team on Vercel.

A member of the Team first needs to authorize it.

[a0preetham]

@cramforce Claude seems to write loops and the pipes within a loop do not work currently.

[cramforce]

Please run changeset, sign the commit, and revisit test coverage

[a0preetham]

Done!

✅ Changeset — .changeset/while-loop-stdin-cursor.md with patch bump
✅ Signed commit — SSH-signed, GitHub shows verified: true
✅ Test coverage — 4 new tests + grep cursor-advancement fix

[a0preetham]

Updated with the requested changes. Here's a summary of what's in the latest push:

Changeset — .changeset/while-loop-stdin-cursor.md added (patch bump).

Additional fixes surfaced by test coverage:

• executeUntil was missing all stdin/redirection handling — until ... done < file now works correctly
• executeFor had the same gap — for i in ...; do read ...; done < file now works
• grep (and by extension rg) read ctx.stdin directly without advancing the shared cursor — added ctx.stdinCursor?.readAll() to the stdin path

Test coverage (25 tests across 2 files, all passing):

• src/stdin-cursor.test.ts — 11 unit tests covering StdinCursor directly (advance, readAll, exhausted, edge cases)
• src/interpreter/while-loop-stdin.test.ts — 14 integration tests covering: simple pipelines, multi-stage pipelines, nested loops, group commands, heredocs, stdin API option, cat/grep reading from loop stdin, for loop with stdin redirect, until loop

Coverage for changed files (unit suite, WASM tests excluded):

File	Stmts	Branch
stdin-cursor.ts	100%	100%
utils/file-reader.ts	100%	100%
commands/grep/grep.ts	81.4%	80.8%
interpreter/control-flow.ts	83.8%	70.0%

The gaps in grep.ts and control-flow.ts are in pre-existing unrelated paths (regex error branches, case/if handling), not in the stdin-cursor code paths.

[a0preetham]

Follow-up push: extended stdin-redirect support to all remaining compound commands and fixed a subtle async bug.

What changed

New: stdin redirect support for if, C-style for, word-list for, until, case

All five now process <, <<, <<< redirections and install a StdinCursor, so read inside the body sees the redirected content:

# Now works correctly in all compound commands
if true; then
  IFS= read -r a
  IFS= read -r b
  echo "$a $b"
fi < /file          # "line1 line2"

for ((i=0; i<3; i++)); do
  IFS= read -r line
  echo "$i: $line"
done < /file        # "0: x\n1: y\n2: z"

case 1 in
  1) IFS= read -r a; IFS= read -r b; echo "$a $b";;
esac < /file        # "hello world"

Bug fix: return await in executeIf

executeIf had return executeStatements(...) (no await) inside a try-finally. In JS, return promise inside try-finally fires the finally synchronously — before the promise resolves — so the cursor was restored to undefined before read could run. Changed to return await executeStatements(...).

Coverage (8141 tests, 377 files)

Metric	Value
Statements	61.25% (26318/42963)
Branches	56.77% (17426/30695)
Functions	64.19% (2456/3826)
Lines	61.85% (25257/40835)

Key files:

• stdin-cursor.ts: 100% statements/branches/lines (11 unit tests)
• control-flow.ts: 76% statements, 63% branches

18 new tests added covering: if condition/body with stdin redirect, C-style for body, case body, word-list for body, until loop with pipeline.

[a0preetham]

Coverage before/after comparison (unit suite, WASM/security tests excluded):

Key files

File	Stmts before	Stmts after	Branch before	Branch after
stdin-cursor.ts	100%	100%	100%	100%
utils/file-reader.ts	100%	100%	100%	100%
commands/grep/grep.ts	81.4%	81.4%	80.8%	80.8%
interpreter/control-flow.ts	83.8%	76.1%	70.0%	63.1%

control-flow.ts coverage % dropped because the follow-up push added ~200 lines of stdin-redirect handling for if, for, until, case, and C-style for. The 18 new tests cover the < file-redirect path in each; << and <<< paths in those constructs are untested (pre-existing branches in while aren't fully covered either).

Overall suite (tests only, not line counts changed)

	Before	After
Test files	359	377
Tests passing	7940	8141
New tests	—	+201

[a0preetham]

Updated coverage — added << (heredoc) and <<< (herestring) tests for every compound command (if, word-list for, C-style for, until, case).

Key files — before → after

File	Stmts before	Stmts after	Branch before	Branch after
stdin-cursor.ts	100%	100%	100%	100%
utils/file-reader.ts	100%	100%	100%	100%
commands/grep/grep.ts	81.4%	81.4%	80.8%	80.8%
interpreter/control-flow.ts	83.8%	82.75%	70.0%	74.76%

Branch coverage on control-flow.ts is now higher than before the PR (70% → 74.76%). Statement % is slightly down (83.8% → 82.75%) because the new redirect-handling code added more lines than the new tests cover — remaining gaps are <<- (strip-tabs heredoc) and error paths for missing files in the new constructs.

Overall

	Initial PR	+fix commit	+heredoc/herestring tests
Tests passing	7940	8141	8151
Test files	359	377	377

[a0preetham]

Final coverage update — added <<- (strip-tabs) and file-not-found error path tests for all six compound commands.

control-flow.ts progression

Stage	Stmts	Branches	Functions
Before this PR	83.8%	70.0%	—
After StdinCursor refactor	76.1%	63.1%	—
+ if/for/until/case fix + </<</<<< tests	82.75%	74.76%	—
+ <<- strip-tabs + file-not-found tests	90.18%	78.5%	100%

Overall (8163 tests, 377 files)

Metric	Value
Statements	61.38% (26372/42963)
Branches	56.88% (17460/30695)
Functions	64.34% (2462/3826)
Lines	61.98% (25311/40835)

Remaining uncovered branches in control-flow.ts are pre-existing edge cases unrelated to this PR: failglob mode in for loops, break/continue inside a while condition expression, and the loopError="error" path in nested loop error handling.

[vercel]

Additional Suggestion:

sed, awk, head/tail, and tr read ctx.stdin directly without calling ctx.stdinCursor?.readAll(), so they never advance the shared stdin cursor, causing loops/compound commands that feed off the cursor to re-process the same input.

[a0preetham]

@vercel run a review