Skip to content

pr: stream simple layouts for non-file inputs#11244

Open
mattsu2020 wants to merge 3 commits into
uutils:mainfrom
mattsu2020:pr_zero
Open

pr: stream simple layouts for non-file inputs#11244
mattsu2020 wants to merge 3 commits into
uutils:mainfrom
mattsu2020:pr_zero

Conversation

@mattsu2020

@mattsu2020 mattsu2020 commented Mar 9, 2026

Copy link
Copy Markdown
Contributor

Summary

This PR adds a streaming path for simple pr layouts so inputs such as
character devices and FIFOs can be processed incrementally instead of buffering
the entire input in memory.

The main goal is to avoid OOM-like behavior for inputs such as /dev/zero,
where pr can produce output line by line without waiting for a full read.

What changed

  • add a streaming implementation for simple pr layouts
  • use the streaming path for supported non-regular-file inputs
  • align streaming output with the existing buffered behavior, including page trailer formatting
  • fall back to the buffered path when the selected page layout leaves no printable content lines
  • add regression tests for:
    • character-device input
    • FIFO/file parity for layouts that actually exercise the streaming path
    • FIFO/file parity in zero-content-line fallback cases

Related

@sylvestre

Copy link
Copy Markdown
Contributor

the list of commits isn't ideal, could you please improve it? thanks

@github-actions

github-actions Bot commented Mar 9, 2026

Copy link
Copy Markdown

GNU testsuite comparison:

Skip an intermittent issue tests/date/date-locale-hour (fails in this run but passes in the 'main' branch)
Skip an intermittent issue tests/tail/inotify-dir-recreate (fails in this run but passes in the 'main' branch)
Skipping an intermittent issue tests/pr/bounded-memory (passes in this run but fails in the 'main' branch)
Congrats! The gnu test tests/rm/many-dir-entries-vs-OOM is now passing!
Congrats! The gnu test tests/tail/pipe-f is now passing!

@codspeed-hq

codspeed-hq Bot commented Mar 9, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 339 untouched benchmarks
⏩ 46 skipped benchmarks1


Comparing mattsu2020:pr_zero (339f865) with main (6e28633)

Open in CodSpeed

Footnotes

  1. 46 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@github-actions

Copy link
Copy Markdown

GNU testsuite comparison:

Skip an intermittent issue tests/tail/inotify-dir-recreate (fails in this run but passes in the 'main' branch)
Skipping an intermittent issue tests/cut/bounded-memory (passes in this run but fails in the 'main' branch)
Skipping an intermittent issue tests/pr/bounded-memory (passes in this run but fails in the 'main' branch)
Skipping an intermittent issue tests/tty/tty-eof (passes in this run but fails in the 'main' branch)
Note: The gnu test tests/tail/pipe-f is now being skipped but was previously passing.

@github-actions

github-actions Bot commented May 23, 2026

Copy link
Copy Markdown

GNU testsuite comparison:

Skip an intermittent issue tests/date/date-locale-hour (fails in this run but passes in the 'main' branch)
Skip an intermittent issue tests/tail/tail-n0f (fails in this run but passes in the 'main' branch)
Skipping an intermittent issue tests/pr/bounded-memory (passes in this run but fails in the 'main' branch)
Note: The gnu test tests/cut/bounded-memory is now being skipped but was previously passing.
Congrats! The gnu test tests/tail/pipe-f is now passing!

Comment thread src/uu/pr/src/pr.rs Outdated
Comment thread tests/by-util/test_pr.rs
Comment thread src/uu/pr/src/pr.rs Outdated
Comment thread src/uu/pr/src/pr.rs Outdated
Comment thread src/uu/pr/src/pr.rs
@mattsu2020
mattsu2020 force-pushed the pr_zero branch 2 times, most recently from e65dcbb to 4668cda Compare June 7, 2026 15:10
Replace the whole-buffer page parser with a chunk-aware PageBuilder while preserving the existing buffered output path. This isolates page boundary and line-number handling before the streaming path is enabled.
Process simple layouts from pipes, stdin, and character devices incrementally instead of buffering unbounded input. Reuse PageBuilder for pagination, preserve buffered fallback behavior, and cover streaming devices, FIFO parity, invalid UTF-8, and zero-content-line cases.
@oech3

oech3 commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Conflicting

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants