Merged
Conversation
…ardlink and sparse file support
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
base-tarpreviously had aTarOutputStreambut no way to read tar archives back, and the writer had a latent bug where names over 100 characters were truncated at an arbitrary UTF-16 offset instead of a valid path boundary, risking silent corruption of long or multi-byte paths.This adds
TarInputStream, a full POSIX ustar reader with the extensions needed to read real-world archives: PAX extended headers (x/gtypeflags, including global-header cancel-on-empty semantics), GNU long-name/long-link extensions (L/K), and the GNU base-256 fallback for numeric fields too large for their fixed-width octal encoding (e.g. sizes at or above 8GB).TarHeadernow measures and splits names in UTF-8 bytes rather than UTF-16 characters, fixing the original truncation bug, and falls back to a PAXpathrecord when no valid ustar prefix/name split exists.Two entry types that were previously unwritable are now supported end to end: hard links (
createHardlinkHeader, typeflag1) and GNU old-style sparse files (createSparseHeader, typeflagS), the latter including multi-block sparse maps for entries with more than four stored chunks.TarOutputStream's PAX writer also now emitssize/uid/gidrecords alongsidepath/linkpathwhenever a PAX header is already being written and one of those fields would otherwise only be recoverable via base-256, for better compatibility with PAX-only readers.Getting sparse round-trips correct with real
tartook two fixes beyond the initial implementation: a missing trailing zero-length terminator entry that GNU tar relies on to truncate extracted files out to their full logical size, and a discovery that GNU tar's own extractor can mishandle sparse chunks at arbitrary byte offsets it did not write itself, so chunk offsets should stay aligned to a filesystem block boundary (as any real sparse-file detector likeSEEK_HOLE/SEEK_DATAwould naturally produce); this is documented onTarHeader.SparseChunk.