Commit d4b421d
authored
feat(session): add JSONL-based session persistence with resume and listing (#13)
## Summary
Add session persistence so conversations survive across process exits.
Sessions are stored as JSONL files (append-only, crash-safe) under
`$XDG_DATA_HOME/ox/sessions/{project}/`. Users can `ox --list` to see
past sessions and `ox -c [<prefix>]` to resume one; `--all` / `-a`
widens both flags across every project. Resumed conversations restore in
the TUI with full fidelity — text, tool calls + results, and (when
`show_thinking` is on) thinking blocks. Works in TUI, bare REPL, and
headless modes. Session ID flows through to the
`x-claude-code-session-id` API header.
### Schema and storage
- **Forward-compatible JSONL** with discriminated entry types — `header`
(carries a `version` readers reject when newer), `message` (UUID +
`parent_uuid` chain), `title` (re-appendable, with `source`:
`first_prompt` / `ai_generated` / `user_provided`), `summary` (exit
marker), and an `Unknown` catch-all so new variants land additively
without a schema bump.
- **Project-scoped layout** — `sessions/{project}/{epoch}-{uuid}.jsonl`.
`{project}` derives from CWD with reserved chars collapsed to `-`; long
paths truncate and append an xxh64 hash of the original `OsStr` bytes so
distinct paths cannot collide after truncation. The epoch prefix keeps
`ls` chronological; lookup by UUID suffix stays O(readdir). One-time
startup sweeps migrate any legacy flat-layout / unprefixed files.
- **Lazy file creation** — `start()` only allocates the session ID and
stages the header in memory; the on-disk file is materialized by the
first `record_message`. A session that exits without recording any
message leaves no artifact behind, so launching `ox` and quitting
without typing does not pollute `--list`. Mirrors claude-code's
`sessionStorage` behaviour.
- **`0o600` perms on Unix** — verbatim tool output may include secrets,
so files stay owner-only.
### Concurrent resume
- **No file-level lock.** Two processes resuming the same session both
acquire append handles immediately and record new messages with
`parent_uuid` pointing at whichever tip they saw at load time. The
resulting fork is resolved on the *next* load: `load_session_data` walks
the recorded UUID DAG, picks the newest-timestamped leaf, and walks back
via `parent_uuid` to the root. Cycles (seen-set) and orphan
`parent_uuid` references are chain terminators, so a corrupted file
yields a trusted prefix rather than hanging or erroring; duplicate UUIDs
resolve last-append-wins. Fork losers stay in the file for audit but are
invisible to later resumes. Matches claude-code's
`loadMessagesFromJsonlPath` + `--fork-session` semantics.
- An in-process per-write `Mutex<SessionManager>` keeps tasks within a
single process from interleaving writes; the lock is never held across
API streaming or tool execution, so future tasks (telemetry, slash
commands) can share the session without waiting for a turn to finish.
### Resume sanitization and crash safety
- On load, the transcript runs through a fixed pipeline so a mid-turn
crash never produces an API-invalid history: strip trailing `thinking` →
drop unresolved assistant `tool_use` and orphan user `tool_result`
blocks → drop emptied messages → merge any adjacent same-role survivors
→ prepend a synthetic user sentinel if the head is now `assistant` →
append an assistant sentinel if the tail is a user turn of only tool
results.
- `load_session_data` reads byte-by-line with warn-skip on non-UTF8 /
malformed lines, so a session SIGKILLed mid-`writeln!` (possibly
truncated mid-UTF8 codepoint, missing trailing `\n`) still resumes.
Interleaved fragments from concurrent large (> `PIPE_BUF`) writes go
through the same path.
### Graceful shutdown
`shutdown_signal()` awaits the first of SIGINT (portable) / SIGTERM
(Unix) / SIGHUP (Unix). All three modes race their main work against
this future via `tokio::select!`, so `finish()` runs on `kill`, SSH
disconnect, or Ctrl+C in cooked mode. (TUI raw mode consumes SIGINT
before it reaches us, but SIGTERM / SIGHUP still arrive.) Mid-stream
Ctrl+C in the TUI aborts the streaming future before the assistant
message is recorded; the next resume sees a clean user-ending transcript
with nothing for sanitization to fix — pinned by a regression test.
**Bare-REPL / headless signal exit.** `tokio::io::stdin()` uses an
uncancellable blocking thread that keeps the runtime alive until stdin
EOFs, so without care a signal-induced exit would hang until the user
pressed Enter. After `finish()` has run, we `std::process::exit(0)` to
bypass runtime Drop — explicit cleanup already ran, so there's nothing
left to lose. Normal EOF / error paths still return through `main` so
the exit code reflects the result.
### CLI surface
- **`--list` / `-l`** — reads the header + first-prompt line and the
last 4 KiB of each file (never parses the full transcript), sorted by
file mtime so resumed sessions bubble to the top. Under `--all`, a
`Project` column shows the tildified `cwd` so cross-project rows stay
disambiguable. Long titles truncate with `...` to fit detected terminal
width (`unicode-width`-aware, CJK / emoji billed correctly); piped
output and undetectable widths render untruncated so downstream tools
can wrap at their own width.
- **`--continue` / `-c`** — bare flag resumes the most-recent session;
`-c <prefix>` matches by ID prefix with explicit ambiguous-prefix and
empty-prefix errors (the latter hints at the bare form). `--all` widens
scope; a clap `ArgGroup` enforces that `--all` is only accepted
alongside `--list` or `--continue` so `ox --all` no longer silently
no-ops.
- List rendering extracted to `session::list_view` and resume resolution
to `session::resolver`. `main.rs` is excluded from coverage, so both
modules now have direct unit tests (~14 and ~12 respectively).
### TUI history rendering
Resumed sessions render text, tool calls (`ToolUse` / `ServerToolUse`)
paired with their results via a per-load `tool_use_id → label` map (with
a `(result)` fallback for orphans), and thinking blocks gated by
`show_thinking` (our analogue of claude-code's `verbose` /
`isTranscriptMode`). `RedactedThinking` is always dropped (opaque
ciphertext). The result matches the live view block-for-block.
### TUI wrap fixes
Three pre-existing TUI wrap bugs surfaced while reviewing resumed
sessions and are fixed in the same PR:
- **Blockquote continuation lines lost their marker.** Long blockquoted
paragraphs wrapped with a plain 2-space indent instead of repeating `>
`, because the markdown renderer called `wrap_line` without a styled
continuation prefix. Now the flattened `indent_stack` (which already
stores the correct continuation form — `> ` for blockquotes, spaces for
lists) is fed in as the styled prefix.
- **Tight list items never wrapped at all.** pulldown-cmark emits tight
`Tag::Item` blocks directly as `Text` events with no paragraph wrapper,
so the existing `end_paragraph → wrap_last_line` path never ran and long
list lines got clipped at the right edge of the terminal. Fixed by
wrapping in `TagEnd::Item` before the indent is popped; idempotent on
loose items that already wrapped via `end_paragraph`.
- **Tool call and tool result labels never wrapped.**
`push_tool_call_line` and `push_tool_result_line` emitted the border
prefix + icon + label as a single `Line`, so long shell commands (e.g.
`cd /Users/... && ls
"${XDG_DATA_HOME:-$HOME/.local/share}/ox/sessions/"`) ran off the
viewport. Now both wrap with a styled continuation prefix aligned under
the label body (`TOOL_RESULT_PREFIX` for calls, `TOOL_OUTPUT_PREFIX` for
results).
Along the way, the two `tui::wrap::wrap_line` / `wrap_line_styled` entry
points were consolidated into a single `wrap_line(line, max_width,
continuation_indent, continuation_prefix: Option<&[Span]>)` so there's
one wrap path for every consumer.
### Cross-cutting
- **First write failure per session is surfaced to the user via
`AgentEvent::Error`**, then warn-logs only — a temporary disk hiccup
informs the user once instead of flooding the UI.
- **`LOCAL_OFFSET` is captured before the tokio runtime starts** and
cached in a `OnceLock` (`time`'s `current_local_offset()` is unsound
under multi-threaded runtimes on Linux).
- Adjacent housekeeping: `rust-toolchain.toml` pinning the stable
channel; clippy fixes for Rust 1.95 lints surfaced alongside.
## Changes
| Area | Files | Notes |
| ------------------ |
------------------------------------------------------------------------------------------------------------------------------------------------
|
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|
| Schema | `session/entry.rs` | JSONL `Entry` enum (`Header` / `Message`
/ `Title` / `Summary` / `Unknown`); `version`, UUID + `parent_uuid`;
`SessionInfo` / `TitleInfo` / `ExitInfo` listing metadata |
| Storage | `session/store.rs`, `session/path.rs` | `SessionStore` /
`SessionWriter`; XDG path resolution; project subdirectories (xxh64 hash
on overlong paths); `{epoch}-{uuid}.jsonl`; mtime sort; UUID-DAG
`load_session_data`; byte-by-line crash-safe reads; legacy-layout
migration sweeps |
| Lifecycle | `session/manager.rs`, `message.rs` | `start` / `resume` /
`record_message` / `finish`; first-prompt title capture (≤ 80 chars);
`sanitize_resumed_messages` pipeline; first-failure surfacing;
mid-stream-abort regression test; `strip_trailing_thinking` exposed for
sanitization reuse |
| CLI dispatch | `main.rs`, `session/resolver.rs`,
`session/list_view.rs` | `--list` / `--continue` / `--all` with
`ArgGroup` coupling; `ResumeMode` + `resolve_session`;
`record_session_message` per-write mutex helper; `shutdown_signal()` +
`tokio::select!` arms; surface session write errors via
`AgentEvent::Error`; list-table renderer with Project column +
width-aware truncation |
| TUI integration | `tui/app.rs`, `tui/components/chat.rs`,
`tui/markdown/render.rs`, `tui/wrap.rs` | Share session manager with
agent loop; full-fidelity `load_history` (text, tool calls + paired
results, gated thinking); new `ChatEntry::Thinking` variant. Wrap fixes:
blockquote markers repeat on wrapped continuations, tight list items
wrap, tool call / result labels wrap; `wrap_line` API consolidated |
| Client integration | `client/anthropic.rs`, `config/oauth.rs` |
`Client::new` accepts session ID for the `x-claude-code-session-id`
header; OAuth lock ported to the shared `util::lock::retry_acquire` |
| Shared utilities | `util.rs`, `util/lock.rs`, `util/path.rs` |
`retry_acquire` advisory-lock helper; `tildify` (`$HOME` → `~/`) shared
by `--list` and the TUI status bar |
| Build / deps | `Cargo.toml` (workspace + member),
`rust-toolchain.toml` | Add `fs4` (oauth lock helper) and `xxhash-rust`
(project subdir hash); broaden `time` features for header
(de)serialization; add `serde` to `uuid`; pin stable toolchain |
| Docs | `docs/guide/sessions.md` (+ guide index),
`docs/research/session-persistence.md` (+ research index),
`docs/roadmap.md`, `README.md`, `CLAUDE.md` | User guide; design notes
(schema, sanitization, fork-friendly DAG, comparison vs. claude-code /
codex); roadmap entries (Working Today + Current Focus + Later); crate
structure refresh |
## Test plan
### Automated (verified on the current tip)
- [x] `cargo fmt --all --check` — clean
- [x] `cargo build` — compiles cleanly
- [x] `cargo clippy --all-targets -- -D warnings` — zero warnings
- [x] `cargo test` — 575 tests pass
- [x] `cargo llvm-cov --ignore-filename-regex 'main\.rs'` — 90.94% line
/ 91.93% function / 91.56% region overall. Session-module floors:
- `session/path.rs` — 100.00%
- `session/manager.rs` — 99.50%
- `tui/components/chat.rs` — 98.63%
- `tui/markdown/render.rs` — 98.00%
- `tui/wrap.rs` — 98.11%
- `session/resolver.rs` — 97.01%
- `session/entry.rs` — 96.00%
- `session/list_view.rs` — 94.33%
- `util/lock.rs` — 93.06%
- `session/store.rs` — 93.24%
- `util/path.rs` — 91.30%
### Manual smoke tests (re-run on the current tip)
- [x] `XDG_DATA_HOME=tmpdir ox --no-tui </dev/null` then `ox -l` —
listing reports "No sessions found in this project"; the project subdir
is empty (lazy-creation regression check from PR review).
- [x] `ox --all` alone and `ox --all -p hi` — clap rejects both with
`<--list|--continue [<SESSION_ID>]>` usage hint.
- [x] `ox -l` and `ox -la` against constructed sessions — header / mtime
sort / message count / title shown correctly; `-la` adds the `Project`
column with tildified `~/GitHub/oxide-code`.
- [x] `ox -c ""` and `ox -c " "` — both error with `empty session ID
prefix` and hint at using bare `ox -c` to resume the latest session.
- [x] `ox -c zzzz` (no match) — errors with `no session matching prefix
'zzzz'`.
- [x] `ox -c 9` (ambiguous, 2 sessions starting with `9`) — errors with
`ambiguous prefix '9' matches 2 sessions: <id>, <id>`.
- [x] `ox -c 27 -p hi` — clap rejects (`--continue` and `--prompt` are
mutually exclusive).
- [x] `ox -l | cat` — full title rendered untruncated when stdout is
piped.
### Pinned by automated tests (not separately re-run by hand)
These behaviors are exercised end-to-end by unit / integration tests;
the test plan items map to specific `cargo test` cases so reviewers can
audit the coverage:
- `0o600` perms on session files —
`session::store::tests::create_sets_user_only_file_permissions_on_unix`.
- Title truncation in narrow terminals —
`session::list_view::tests::render_sessions_truncates_title_when_term_width_too_narrow`
and the `truncate_to_width_*` cluster.
- Resume sanitization (mid-tool-turn crash, sentinel injection, orphan
tool_result drop, collapse same-role) — 12 tests in
`session::manager::tests::resume_*`.
- Mid-stream-quit clean transcript —
`session::manager::tests::resume_after_mid_stream_abort_yields_clean_user_ending_transcript`.
- Concurrent fork resume / UUID DAG selection —
`session::store::tests::load_session_data_picks_newest_leaf_*` and the
rest of the `load_session_data_*` cluster.
- Crash-safe reads (truncated UTF-8, malformed line) —
`session::store::tests::load_session_data_warn_skips_*`.
- Resumed-history rendering (text, tool calls + paired results, gated
thinking, `RedactedThinking` always dropped) —
`tui::components::chat::tests::load_history_*`.
- Markdown wrap regressions —
`tui::markdown::render::tests::blockquote_wraps_with_marker_on_continuations`,
`nested_blockquote_wraps_with_nested_markers`,
`tight_list_item_wraps_without_repeating_marker`,
`blockquote_list_item_wraps_with_blockquote_marker_only`, plus
`tui::wrap::tests::continuation_prefix_spans_applied`.
- Tool call / result label wrap —
`tui::components::chat::tests::push_tool_call_line_wraps_long_label` and
`push_tool_result_line_wraps_long_label`.
### End-to-end live scenarios (verified on the current tip)
- [x] Fresh session + one prompt (headless `ox -p "..."`) — JSONL
created at `sessions/{project}/{epoch}-{uuid}.jsonl` with `0o600` perms;
entries are header + title + 2 messages + summary; `message_count: 2`.
- [x] SIGTERM / SIGHUP / SIGINT against bare REPL (verified via
`expect`) — all three fire the handler arm, run `finish()`, and now exit
cleanly (`eof, ok` in ≤ 1 s) thanks to the `std::process::exit(0)`
post-`finish()` fix in this PR; previously they hung on the tokio-stdin
blocking thread until SIGKILL. JSONL remains intact across all three.
- [x] Fresh session + one prompt in bare REPL + SIGTERM during the next
`> ` wait — process exits cleanly; JSONL has header + title + 2 messages
+ `Summary { message_count: 2 }`.
- [x] Two terminals concurrent resume — `echo "A" | ox --no-tui -c <id>`
and `echo "B" | ox --no-tui -c <id>` run in parallel; both exit 0, both
append; the resulting JSONL has 2 `summary` entries and a genuine fork
(two user messages sharing a `parent_uuid` at the pre-resume tip, each
with its own assistant reply). A third resume picks the
newer-timestamped leaf branch and new messages correctly chain from
there, with the losing branch preserved in the file but invisible to the
API prefix.
### Requires interactive TUI (not run this session)
- [x] `ox` (TUI) with a real conversation, then `ox -c` — visually
verify the TUI renders prior text + tool call markers + tool results +
gated thinking, and that blockquotes / tight lists / tool labels wrap
correctly at narrow widths. The rendering path is pinned by
`tui::components::chat::tests::load_history_*` (6 tests including one
per content block type) and the new wrap regression tests listed above;
the remaining gap is the live render / scroll / terminal paint, which
needs a human reviewer at a real TTY.1 parent ca9e0ad commit d4b421d
34 files changed
Lines changed: 5987 additions & 174 deletions
File tree
- .cspell
- crates/oxide-code
- src
- client
- config
- session
- tool
- tui
- components
- markdown
- util
- docs
- guide
- research
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
10 | 10 | | |
11 | 11 | | |
12 | 12 | | |
| 13 | + | |
13 | 14 | | |
14 | 15 | | |
15 | 16 | | |
| |||
36 | 37 | | |
37 | 38 | | |
38 | 39 | | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
39 | 43 | | |
40 | 44 | | |
| 45 | + | |
41 | 46 | | |
42 | 47 | | |
43 | 48 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
30 | 30 | | |
31 | 31 | | |
32 | 32 | | |
33 | | - | |
| 33 | + | |
34 | 34 | | |
35 | 35 | | |
36 | 36 | | |
37 | 37 | | |
38 | 38 | | |
39 | 39 | | |
40 | 40 | | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
41 | 49 | | |
42 | 50 | | |
43 | 51 | | |
| |||
47 | 55 | | |
48 | 56 | | |
49 | 57 | | |
50 | | - | |
51 | | - | |
52 | | - | |
53 | | - | |
54 | | - | |
55 | | - | |
56 | | - | |
57 | | - | |
58 | | - | |
59 | | - | |
60 | | - | |
61 | | - | |
62 | | - | |
63 | | - | |
64 | | - | |
65 | | - | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
66 | 78 | | |
67 | 79 | | |
68 | 80 | | |
| |||
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
20 | 20 | | |
21 | 21 | | |
22 | 22 | | |
| 23 | + | |
23 | 24 | | |
24 | 25 | | |
25 | 26 | | |
| |||
44 | 45 | | |
45 | 46 | | |
46 | 47 | | |
47 | | - | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
48 | 56 | | |
49 | 57 | | |
50 | 58 | | |
| |||
63 | 71 | | |
64 | 72 | | |
65 | 73 | | |
66 | | - | |
| 74 | + | |
67 | 75 | | |
68 | 76 | | |
69 | 77 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
21 | 21 | | |
22 | 22 | | |
23 | 23 | | |
| 24 | + | |
24 | 25 | | |
25 | 26 | | |
26 | 27 | | |
| |||
38 | 39 | | |
39 | 40 | | |
40 | 41 | | |
| 42 | + | |
41 | 43 | | |
42 | 44 | | |
43 | 45 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
17 | 17 | | |
18 | 18 | | |
19 | 19 | | |
| 20 | + | |
20 | 21 | | |
21 | 22 | | |
22 | 23 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
233 | 233 | | |
234 | 234 | | |
235 | 235 | | |
236 | | - | |
237 | | - | |
| 236 | + | |
| 237 | + | |
238 | 238 | | |
239 | 239 | | |
240 | 240 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
2 | 2 | | |
3 | 3 | | |
4 | | - | |
| 4 | + | |
5 | 5 | | |
6 | 6 | | |
7 | 7 | | |
| 8 | + | |
| 9 | + | |
8 | 10 | | |
9 | 11 | | |
10 | 12 | | |
| |||
17 | 19 | | |
18 | 20 | | |
19 | 21 | | |
20 | | - | |
21 | | - | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
22 | 25 | | |
23 | 26 | | |
24 | 27 | | |
| |||
306 | 309 | | |
307 | 310 | | |
308 | 311 | | |
309 | | - | |
310 | | - | |
| 312 | + | |
| 313 | + | |
| 314 | + | |
| 315 | + | |
311 | 316 | | |
312 | | - | |
313 | | - | |
314 | | - | |
315 | | - | |
316 | | - | |
317 | | - | |
318 | | - | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
| 320 | + | |
| 321 | + | |
319 | 322 | | |
320 | 323 | | |
321 | 324 | | |
322 | | - | |
323 | 325 | | |
324 | | - | |
325 | | - | |
326 | | - | |
327 | | - | |
328 | | - | |
329 | | - | |
330 | | - | |
331 | | - | |
332 | | - | |
333 | | - | |
334 | | - | |
| 326 | + | |
335 | 327 | | |
336 | | - | |
337 | | - | |
338 | | - | |
| 328 | + | |
| 329 | + | |
| 330 | + | |
| 331 | + | |
| 332 | + | |
| 333 | + | |
| 334 | + | |
| 335 | + | |
| 336 | + | |
| 337 | + | |
| 338 | + | |
| 339 | + | |
| 340 | + | |
| 341 | + | |
339 | 342 | | |
340 | 343 | | |
341 | 344 | | |
| |||
0 commit comments