Skip to content

feat(base-tar): add TarInputStream with GNU/PAX compatibility, plus hardlink and sparse file support - #97

Merged
deer merged 1 commit into
mainfrom
tar_input
Jul 20, 2026
Merged

feat(base-tar): add TarInputStream with GNU/PAX compatibility, plus hardlink and sparse file support#97
deer merged 1 commit into
mainfrom
tar_input

Conversation

@deer

@deer deer commented Jul 17, 2026

Copy link
Copy Markdown
Collaborator

base-tar previously had a TarOutputStream but no way to read tar archives back, and the writer had a latent bug where names over 100 characters were truncated at an arbitrary UTF-16 offset instead of a valid path boundary, risking silent corruption of long or multi-byte paths.

This adds TarInputStream, a full POSIX ustar reader with the extensions needed to read real-world archives: PAX extended headers (x/g typeflags, including global-header cancel-on-empty semantics), GNU long-name/long-link extensions (L/K), and the GNU base-256 fallback for numeric fields too large for their fixed-width octal encoding (e.g. sizes at or above 8GB). TarHeader now measures and splits names in UTF-8 bytes rather than UTF-16 characters, fixing the original truncation bug, and falls back to a PAX path record when no valid ustar prefix/name split exists.

Two entry types that were previously unwritable are now supported end to end: hard links (createHardlinkHeader, typeflag 1) and GNU old-style sparse files (createSparseHeader, typeflag S), the latter including multi-block sparse maps for entries with more than four stored chunks. TarOutputStream's PAX writer also now emits size/uid/gid records alongside path/linkpath whenever a PAX header is already being written and one of those fields would otherwise only be recoverable via base-256, for better compatibility with PAX-only readers.

Getting sparse round-trips correct with real tar took two fixes beyond the initial implementation: a missing trailing zero-length terminator entry that GNU tar relies on to truncate extracted files out to their full logical size, and a discovery that GNU tar's own extractor can mishandle sparse chunks at arbitrary byte offsets it did not write itself, so chunk offsets should stay aligned to a filesystem block boundary (as any real sparse-file detector like SEEK_HOLE/SEEK_DATA would naturally produce); this is documented on TarHeader.SparseChunk.

@deer
deer merged commit 624a9ce into main Jul 20, 2026
3 checks passed
@deer
deer deleted the tar_input branch July 20, 2026 06:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant