Skip to content

feat: export the current view to a CSV file - #27

Merged
SpollaL merged 3 commits into
mainfrom
feat/export-view
Aug 22, 2026
Merged

SpollaL merged 3 commits into
mainfrom
feat/export-view

Conversation

@SpollaL

@SpollaL SpollaL commented Aug 22, 2026

Copy link
Copy Markdown
Owner

Export the current view to a CSV file

Adds w — write what is on screen to a CSV file — plus a fix for the qa.sh
flake that surfaced while building it.

Branch: feat/export-view → main · 11 files, +812/−18


The feature

w opens a prompt prefilled with a name derived from the source, and writes
App::view — the rows as filtered, sorted and grouped on screen — rather than the
original input. That distinction is the whole point of the feature: App keeps df
(never mutated after load) and view separate, so exporting view is what makes the
file match the display.

 w orders.export.csv_ (writes CSV; .csv added if missing)
 ✓ Wrote 24 rows → orders.export.csv

Nulls are written by Polars as empty fields. The ∅ display glyph and the (null)
filter sentinel are rendering concerns and never reach the file.

Design decisions

CSV only, and the extension is appended rather than parsed. The first draft
supported CSV, Parquet and NDJSON, chosen by the extension you typed. Dropping to
CSV alone removed a Format enum, extension parsing, an unsupported-format error
path, and roughly half the test surface. resolve_path now appends .csv when it is
missing, which deletes the entire "unsupported format" error class from the input
path and stops the extension from lying about the contents.

The write is staged and renamed. File::create truncates the destination before
Polars validates anything, and CsvWriter emits the header before it rejects a
nested dtype. Writing in place therefore replaced a good file with a one-line stub —
reproduced directly:

err=Some("CSV format does not support nested data")
before="original content that must survive\n"
after="a,b\n"

The write now goes to a <dest>.tmp sibling and is renamed into place only on
success; the scratch file is removed on failure. The sibling keeps the rename inside
one filesystem so it cannot silently degrade to a copy.

Exports are always local, and the prompt says so before Enter. browse az://…
hands the viewer the full URI as its file_path, so the suggested name is reduced to
the basename (az://container/data/orders.parquet → orders.export.csv) and lands
in the working directory. A remote destination is flagged in the prompt and refused
before any file is touched:

 w az://c/out.csv_ — ⚠ az:// is not writable; export writes local files
 ✗ Export failed: cannot write to az:// paths — export writes local files only

export::remote_scheme is shared by the prompt and the write guard. A second copy of
the prefix list in ui.rs would silently stop warning the moment a scheme was added
to the guard.

Module shape follows src/clipboard.rs: export.rs is a thin module of free
functions returning std::io::Result, with resolve_path pure (a function of the
typed string and $HOME) and write_csv the only part that touches the filesystem.
app.rs owns the ✓/✗ status formatting. No new error type — std::io::Error::other
wraps the PolarsError, matching the existing precedent in theme.rs.

Also in this PR: the qa.sh startup flake

start_app slept a fixed 0.6s and then sent keys regardless. Under load the app had
not started yet, so keystrokes reached the shell instead — where a filter query like
> 0 is a redirection. That silently created junk files (0, 1f, abcB) in the
repo root and failed suites X4–X6 and Y1.

Polling for a painted frame is not sufficient on its own, because the previous app may
still own the screen: datasight ignores ctrl-c, and browse mode ignores q while a
file is open (browser/events.rs:124). launch therefore respawns the pane outright,
proves the shell is reading input by polling for an echoed token, and then waits for
that token to be replaced by a box-drawing frame.

This exposed X6 as vacuous. It sent cd tests/fixtures as bare keystrokes into
the still-running browse app from X5, then ran $BINARY browse — and $BINARY is the
relative ./target/debug/datasight, which does not exist from that directory. No app
ever launched; the assertion had been passing on X5's leftover screen. It now runs the
cd as part of the launched command with an absolute binary path, and genuinely tests
the no-path default.

The two stdin tests had hand-rolled the same fixed-sleep launch; they use launch now
too.

Testing

  • cargo test — 302 passed. 23 cover export: 13 unit tests in export.rs
    (path resolution, ~ expansion, the failed-write cases, the remote refusal) and 10
    integration tests in app_tests.rs (view-not-source, sort order, grouped view,
    determinism, header-only empty view, real nulls, remote source and destination).
  • qa.sh — 174 passed, 0 failed. Suite P adds 29 checks for the export prompt,
    .csv appending, the overwrite and remote warnings, help discoverability, the
    failed-write-leaves-the-file-intact path, and export from a browse pane.
  • cargo clippy --all-targets -- -D warnings and cargo fmt --check clean.
  • The flake fix was verified with three consecutive qa.sh runs at load average
    3–11, with git status byte-identical before and after.
  • The Azure path was verified live against Azurite: browse an az:// container, open
    a remote parquet, filter it, export to a local CSV, and confirm nothing is written
    back to the container.

Docs

README gains an Export section and a Features bullet; CHANGELOG has an [Unreleased]
entry; CLAUDE.md documents export.rs, the new Mode::Export, STDIN_LABEL, the
launch/start_app requirement for new QA tests, and one gotcha worth repeating:

Any new viewer mode that accepts typed text must be added to App::is_typing, or
browse mode swallows T from the input to open the theme picker.

Notes for review

  • w is export in the DataFrame viewer and word-wrap in the text viewer. The two
    viewers have disjoint keymaps so there is no conflict, but the mnemonic overlap is
    worth a second opinion.
  • Cargo.toml and Cargo.lock are untouched; no new dependencies.

SpollaL and others added 3 commits August 22, 2026 12:04
`w` opens a prompt prefilled with a name derived from the source and writes
`App::view` — the rows as filtered, sorted and grouped on screen — so the file
matches the display rather than the original input.

CSV is the only format. `resolve_path` appends `.csv` when it is missing, which
removes the whole "unsupported format" error class from the input path and keeps
the extension from lying about the contents.

The write is staged through a `<dest>.tmp` sibling and renamed into place on
success. `File::create` truncates before polars validates, and `CsvWriter` emits
the header before rejecting a nested dtype, so writing in place would leave a
one-line stub where the previous file used to be.

Exports are always local. A remote source suggests a local basename, and an
`az://` or `s3://` destination is flagged in the prompt and refused before any
file is touched — `remote_scheme` is shared by the prompt and the write so the
two cannot disagree.

Verified against Azurite: browse an `az://` container, open a remote parquet,
export filtered rows to a local CSV, and confirm nothing is written back.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`start_app` slept a fixed 0.6s and then sent keys regardless. Under load the app
had not started yet, so the keystrokes reached the shell — where a filter query
like `> 0` is a redirection, which silently created junk files (`0`, `1f`,
`abcB`) in the repo root and failed suites X4-X6 and Y1.

Polling for a painted frame is not sufficient on its own, because the previous
app may still own the screen: datasight ignores ctrl-c, and browse mode ignores
`q` while a file is open. `launch` therefore respawns the pane outright, proves
the shell is reading input by polling for an echoed token, and then waits for
that token to be replaced by a box-drawing frame.

This exposed X6 as vacuous. It sent `cd tests/fixtures` as bare keystrokes into
the still-running browse app from X5, then ran `$BINARY browse` — a relative path
that does not exist from that directory. No app ever launched; the assertion was
passing on X5's leftover screen. It now runs the cd as part of the launched
command with an absolute binary path, and genuinely tests the no-path default.

The two stdin tests had hand-rolled the same fixed-sleep launch; they use
`launch` now too.

Verified with three consecutive runs at load average 3-11: 174 passed, 0 failed,
no stray files.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The README described the happy path but was silent on the behaviours a user is
most likely to hit:

- Directories are never created. This is the only error the feature produces in
  normal use, and it surfaces as a bare "No such file or directory (os error 2)"
  that does not explain the export will not mkdir for you. Documented in the
  Export section and given a Troubleshooting entry.
- Nulls are written as empty fields. This mattered enough to note in the
  `export_view` doc comment but never reached the README, and for a data tool the
  null round-trip contract is exactly what a reader needs — `∅` and `(null)` are
  display conventions, not file contents.
- Piped input suggests `export.csv`. Reading from stdin has its own README
  section, so its export default belonged there too.

Also lists Enter/Esc in the keybinding table for consistency with the Search,
Filter and Unique Values sections, notes that absolute paths work and that the
status line reports the resolved path, and records in CLAUDE.md why
`export::remote_scheme` is public — `ui.rs` warns using the same prefix list
`write_csv` refuses on, and a second copy would silently stop warning.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@SpollaL
SpollaL merged commit 05abe1c into main Aug 22, 2026
11 checks passed
@SpollaL
SpollaL deleted the feat/export-view branch August 22, 2026 12:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant