Skip to content

feat(wal): recover a torn tail on open - #104

Merged
hmdsefi merged 44 commits into
plan/v0.1from
task/E2.5
Oct 20, 2026
Merged

hmdsefi merged 44 commits into
plan/v0.1from
task/E2.5

Conversation

@hmdsefi

@hmdsefi hmdsefi commented Oct 20, 2026 •

Copy link
Copy Markdown
Owner

wal.Open now recovers the tail of an interrupted write instead of refusing the log. After a crash during Append, the last segment can end with part of a record. Until now Open reported that as corruption forever. Stacked on #102 (the segmented file log).

chainlog/wal/recover.go: the bad record that ends the scan of the last segment is cut when one of these holds:

  1. fewer than 8 bytes remain, or the record's declared end is past the end of the file;
  2. its declared end is exactly the end of the file (it is the last record);
  3. every byte from it on is zero.

Cutting means: Open truncates the file to the end of the last whole record, logs wal torn tail cut at Warn with segment, position and bytes, and fsyncs it with SyncAlways. The cut runs only after the checkpoint check passes, so an Open that fails changes no file. Any other bad record, and any bad record in an earlier segment, stays ErrCorrupt, and no file changes: a bad record whose declared end lies inside the file and is followed by more data is damage, not a torn write.

How it was verified:

  • TestRecoverEveryCut: 30 blocks in 1 KiB segments. For every cut length of the last segment, Open succeeds with Head equal to the whole records kept, and the entries equal the originals byte for byte. When bytes were cut, the file ends at the last whole record and the warning carries the segment, position and removed bytes. One more Append then works.
  • TestTornVersusCorrupt, fourteen cases:
    • cut: 4,096 zero bytes after the last record (case 3); a flipped byte in the last record; the last record re-encoded with a wrong offset;
    • ErrCorrupt with the files unchanged: a flipped byte in the second-to-last record, in the last record of a sealed segment, or in the first segment; trailing bytes whose first byte is non-zero; zeros with one byte past the first 64 KiB; a zero header followed by a record; a damaged last record followed by one byte; a length flip that would cut records the checkpoint covers; a renamed segment; an empty sealed segment; no segments left.
  • TestRecoverSync: with SyncAlways, the cut segment is fsynced after the cut, and a failed fsync fails Open with the I/O error. TestRecoverOneSegment: a log that never rolled.
  • FuzzRecover: append up to 40 blocks, then cut the last segment or flip a byte in it or in any segment, and reopen. Open never panics and fails only with ErrCorrupt. Whatever it keeps equals the originals. A cut always opens, with Head equal to the records wholly kept. A flipped byte is corruption, unless it is in the very last record, which is cut; a damaged length field in the last segment is a cut when its declared end is at or past the end of the file. The 4 seeds run in every go test; go test -run '^$' -fuzz '^FuzzRecover$' -fuzztime 10s ./chainlog/wal found nothing.
  • Review fixes are in the fix: commits.
  • Coverage of the module with -race: 96.4% (CI requires 95%).
  • Gate: gofmt, go mod tidy -diff, go vet (also for windows/amd64), build and tests with and without -race, golangci-lint v2.14.0. All clean.

Risk: cutting too much drops entries that Append already returned. The every-cut test and the fuzz target pin the decision to the three cases. A bit flip in a length field of the last segment can look like case 1 and drop the records after it; the spec accepts that, and the checkpoint check stops it from dropping records the checkpoint covers.

Closes #10

Base automatically changed from task/E2.4 to plan/v0.1 October 20, 2026 14:33
@hmdsefi
hmdsefi merged commit 3f2edd2 into plan/v0.1 Oct 20, 2026
9 checks passed
@hmdsefi
hmdsefi deleted the task/E2.5 branch October 20, 2026 14:43
hmdsefi added a commit that referenced this pull request Oct 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant