feat: add a ci gpu compress benchmark by joseph-isaacs · Pull Request #8951 · vortex-data/vortex

joseph-isaacs · 2026-07-24T14:42:02Z

Adds a bench-compression GPU variant.

Currently this fails to run. But we can fix that outside of adding the bench.

Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

github-actions · 2026-07-24T14:44:05Z

GPU Compression

┌───────────────────────────────────────────┬────────────────────────┐
│ Benchmark                                 │ vortex-file-compressed │
├───────────────────────────────────────────┼────────────────────────┤
│ decompress time/TPC-H l_comment canonical │ 1676335.00 μs (1.00)   │
└───────────────────────────────────────────┴────────────────────────┘

Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

codspeed-hq · 2026-07-24T14:54:04Z

Merging this PR will improve performance by 17.18%

⚠️

Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚡ 1 improved benchmark
✅ 1891 untouched benchmarks
⏩ 3 skipped benchmarks¹

Performance Changes

	Mode	Benchmark	`BASE`	`HEAD`	Efficiency
⚡	WallTime	`cuda/bitpacked_u8/unpack/3bw[100M]`	351.5 µs	299.9 µs	+17.18%

Tip

Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.

_{Comparing ji/gpu-decompress (f729318) with develop (b15394a)}

3 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

Use direct I/O for the pooled CUDA data-plane reads and black-box the canonicalized batches so the decompression work cannot be optimized away. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WPDxwLypj71WvC9zfusx86

The GPU compression benchmark failed at runtime with: GPU execution for encoding vortex.fsst failed (GPU execution for encoding vortex.patched failed (No CUDA kernel for encoding vortex.patched)) VORTEX_EXPERIMENTAL_PATCHED_ARRAY=1 rewrites interior `Patches` into a top-level `vortex.patched` array. That encoding has no registered CUDA decode kernel, so GPU decompression of FSST-coded columns (whose codes are bitpacked with patches) aborts. Removing the flag keeps patches inline in the bitpacked array, which the CUDA bitpacked kernel decodes. This matches the GPU test workflow (cuda.yaml), which sets only FLAT_LAYOUT_INLINE_ARRAY_NODE. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

0ax1

lgtm. disabled auto-merge and left some nits.

0ax1 · 2026-07-27T16:05:43Z

+      - name: Build GPU compression benchmark
+        shell: bash
+        run: |
+          cargo build --locked --package compress-bench --profile release_debug \


nit: all in one or one flag per line

0ax1 · 2026-07-27T16:06:09Z

+          repo-token: ${{ secrets.GITHUB_TOKEN }}
+          enable-sccache: "true"
+      - uses: ./.github/actions/system-info
+      - name: Display NVIDIA GPU details


nit: probably this should be factored out and re-used across GPU CI tasks

Ill create a follow up

0ax1 · 2026-07-27T16:07:11Z

+          {
+            echo "# GPU Compression"
+            echo
+            echo '```text'


whats the ' after text for?

The single quotes are shell quoting, not part of the output — they stop the shell from treating the backticks in ```text as command substitution. The heredoc emits a plain ```text fence into comment.md.

Generated by Claude Code

0ax1 · 2026-07-27T16:09:22Z

 cargo run -p compress-bench --profile release_debug
 ```
+
+GPU decompression is opt-in and runs only the existing benchmark names allow-listed in


Is this GPU only then?

How do you mean?

0ax1 · 2026-07-27T16:10:17Z

    datasets_filter: Option<Regex>,
    formats: Vec<Format>,
    ops: Vec<CompressOp>,
+    gpu_decompress: bool,


Should this be a vec of [cpu, gpu]?

do you think anything is gpu only?

i mean decompression at least

maybe a flag is the right thing here

0ax1 · 2026-07-27T16:11:32Z

+    /// Run GPU decompression for the allow-listed benchmarks.
+    ///
+    /// This filters the suite to GPU-supported dataset names and runs only Vortex decompression.
+    #[arg(long)]


maybe worth considering handling a list of processors. cpu is default but the user can override

can we do that if and when?

0ax1 · 2026-07-27T16:11:58Z

+        let open_options = SESSION.open_options().with_cuda();
+        #[cfg(target_os = "linux")]
+        let open_options =
+            open_options.with_read_at_options(PooledFileReadAtOptions::default().with_direct_io());


Should direct io be a flag?

Two open review comments on the GPU compression benchmark: - Put the `cargo build` flags for the benchmark on a single line rather than splitting them across a continuation. - Make direct IO a flag instead of hardcoding it. `--gpu-decompress` still defaults to `O_DIRECT` so repeated iterations measure storage bandwidth rather than page-cache hits; `--no-gpu-direct-io` opts back into the page cache. Direct IO remains Linux-only, so the flag is inert elsewhere. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

`compress-bench --gpu-decompress` forces `ops` to `[Decompress]`, so no compressed sizes are recorded and `CompressMeasurements::ratios` stays empty. Rendering the ratios table then indexed `measurements[baseline_target]` for a target with no entry and panicked: thread 'main' panicked at vortex-bench/src/display.rs:50:32: no entry found for key The empty-target list the caller passes when the baseline format is absent hit a second panic in the same function, a divide-by-zero when sizing the map. Skip rendering when there are no measurements or no targets, and cover both cases with regression tests. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

feat: add a ci gpu compress benchmark

846dac3

Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

joseph-isaacs added the action/benchmark-gpu-compress label Jul 24, 2026

github-actions Bot removed the action/benchmark-gpu-compress label Jul 24, 2026

joseph-isaacs added the changelog/feature A new feature label Jul 24, 2026

feat: add a ci gpu compress benchmark

3ce6c56

Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>

joseph-isaacs added the action/benchmark-gpu-compress label Jul 24, 2026

github-actions Bot removed the action/benchmark-gpu-compress label Jul 24, 2026

0ax1 reviewed Jul 24, 2026

View reviewed changes

Comment thread benchmarks/compress-bench/src/gpu_vortex.rs

0ax1 reviewed Jul 24, 2026

View reviewed changes

Comment thread benchmarks/compress-bench/src/gpu_vortex.rs Outdated

joseph-isaacs marked this pull request as ready for review July 24, 2026 15:26

joseph-isaacs force-pushed the ji/gpu-decompress branch from a781d14 to 8cdd1b1 Compare July 24, 2026 15:26

joseph-isaacs requested a review from robert3005 July 24, 2026 16:08

joseph-isaacs enabled auto-merge (squash) July 24, 2026 16:08

joseph-isaacs requested a review from 0ax1 July 27, 2026 13:26

0ax1 disabled auto-merge July 27, 2026 16:08

0ax1 approved these changes Jul 27, 2026

View reviewed changes

0ax1 reviewed Jul 27, 2026

View reviewed changes

joseph-isaacs added the action/benchmark-gpu-compress label Jul 27, 2026 — with Claude

github-actions Bot removed the action/benchmark-gpu-compress label Jul 27, 2026

joseph-isaacs added the action/benchmark-gpu-compress label Jul 28, 2026 — with Claude

github-actions Bot removed the action/benchmark-gpu-compress label Jul 28, 2026

Uh oh!

Conversation

joseph-isaacs commented Jul 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Jul 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

GPU Compression

Uh oh!

codspeed-hq Bot commented Jul 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will improve performance by 17.18%

Performance Changes

Footnotes

Uh oh!

Uh oh!

Uh oh!

0ax1 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

joseph-isaacs commented Jul 24, 2026 •

edited

Loading

github-actions Bot commented Jul 24, 2026 •

edited

Loading

codspeed-hq Bot commented Jul 24, 2026 •

edited

Loading