Skip to content

Specialize VarBin PiecewiseSequence takes#8836

Merged
robert3005 merged 3 commits into
developfrom
codex/piecewise-sequence-varbin-take
Jul 21, 2026
Merged

Specialize VarBin PiecewiseSequence takes#8836
robert3005 merged 3 commits into
developfrom
codex/piecewise-sequence-varbin-take

Conversation

@danking

@danking danking commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Summary

  • move VarBin and VarBinView PiecewiseSequence take specializations out of Use PiecewiseSequence indices for FSL take #8800 into this stacked PR
  • keep the branch based directly on codex/piecewise-sequential-take
  • preserve existing behavior: VarBin copies contiguous byte ranges, VarBinView gathers view ranges while reusing backing data buffers

Validation

  • cargo +nightly fmt --all
  • cargo check -p vortex-array

@danking danking added the changelog/chore A trivial change label Jul 17, 2026
@codspeed-hq

codspeed-hq Bot commented Jul 17, 2026

Copy link
Copy Markdown

Merging this PR will regress 1 benchmark

⚡ 5 improved benchmarks
❌ 1 regressed benchmark
✅ 1792 untouched benchmarks
⏩ 44 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Simulation varbinview_large 390.6 µs 445.4 µs -12.3%
Simulation struct_large 431.1 µs 38.7 µs ×11
Simulation fsl_large 615.6 µs 91.5 µs ×6.7
Simulation list_i32_large 172.7 µs 96.2 µs +79.52%
Simulation i32_large 277.8 µs 235.9 µs +17.75%
Simulation true_count_vortex_buffer[128] 580.6 ns 522.2 ns +11.17%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing codex/piecewise-sequence-varbin-take (5d31540) with develop (2ea51cb)2

Open in CodSpeed

Footnotes

  1. 44 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

  2. No successful run was found on develop (add6944) during the generation of this report, so 2ea51cb was used instead as the comparison base. There might be some changes unrelated to this pull request in this report.

@danking
danking force-pushed the codex/piecewise-sequence-varbin-take branch 2 times, most recently from 79d9e10 to 39ee98e Compare July 20, 2026 16:11
@danking
danking force-pushed the codex/piecewise-sequential-take branch from c94c4ef to 1fdda1c Compare July 20, 2026 16:11
@danking
danking force-pushed the codex/piecewise-sequence-varbin-take branch 3 times, most recently from df091fb to e31db3e Compare July 20, 2026 18:10
Base automatically changed from codex/piecewise-sequential-take to develop July 20, 2026 19:20
@danking
danking force-pushed the codex/piecewise-sequence-varbin-take branch from e31db3e to 9f02349 Compare July 20, 2026 19:24
@danking
danking changed the base branch from develop to codex/piecewise-sequential-run-take-consumers July 20, 2026 19:25
@github-actions

github-actions Bot commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Polar Signals Profiling Results

Latest Run

Status Commit Job Attempt Link
🟢 Done 5d31540 1 Explore Profiling Data
Previous Runs (1)
Status Commit Job Attempt Link
🟢 Done 9f02349 1 Explore Profiling Data

Powered by Polar Signals Cloud

@github-actions

github-actions Bot commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

Benchmarks: Vortex queries 📖

Verdict: No clear signal (medium confidence)
Attributed Vortex impact: -2.2%
Engines: DataFusion No clear signal (+8.9%, medium confidence) · DuckDB Likely improvement (-12.2%, high confidence)
Vortex (geomean): 0.997x ➖
Parquet (geomean): 1.009x ➖
Shifts: Parquet (control) +0.9% · Median polish +0.9%

How to read Verdict and Engines
  • Verdict: Overall PR-level signal after subtracting baseline drift estimated from Parquet control rows. It can be Likely improvement, Likely regression, or No clear signal.
  • Engines: Per-engine attribution. DataFusion is compared against DataFusion/Parquet controls; DuckDB is compared against DuckDB/Parquet controls. This answers whether each engine improved or regressed independently.
  • Confidence: Based on directional consistency, share of rows above the noise floor, and control-run noise.

datafusion / vortex-file-compressed (1.104x ❌, 0↑ 1↓)
name PR 5d31540 (ns) base add6944 (ns) ratio (PR/base)
vortex_q00/datafusion:vortex-file-compressed 🚨 10599312 9338219 1.14
vortex_q01/datafusion:vortex-file-compressed 6495096 6048928 1.07
datafusion / parquet (1.014x ➖, 0↑ 0↓)
name PR 5d31540 (ns) base add6944 (ns) ratio (PR/base)
vortex_q00/datafusion:parquet 20690177 21059503 0.98
vortex_q01/datafusion:parquet 4973716 4750673 1.05
duckdb / vortex-file-compressed (0.881x ✅, 2↑ 0↓)
name PR 5d31540 (ns) base add6944 (ns) ratio (PR/base)
vortex_q00/duckdb:vortex-file-compressed 🚀 10690905 12173747 0.88
vortex_q01/duckdb:vortex-file-compressed 🚀 6600679 7472232 0.88
duckdb / parquet (1.003x ➖, 0↑ 0↓)
name PR 5d31540 (ns) base add6944 (ns) ratio (PR/base)
vortex_q00/duckdb:parquet 23713679 23276573 1.02
vortex_q01/duckdb:parquet 9565286 9680554 0.99

No file size changes detected.

@danking
danking force-pushed the codex/piecewise-sequence-varbin-take branch 5 times, most recently from 9eba049 to 3c5e519 Compare July 20, 2026 20:16
Base automatically changed from codex/piecewise-sequential-run-take-consumers to develop July 20, 2026 22:00
danking added 3 commits July 20, 2026 18:02
Signed-off-by: Daniel King <dan@spiraldb.com>
Signed-off-by: Daniel King <dan@spiraldb.com>
Signed-off-by: Daniel King <dan@spiraldb.com>
@danking
danking force-pushed the codex/piecewise-sequence-varbin-take branch from 3c5e519 to 5d31540 Compare July 20, 2026 22:04
@robert3005
robert3005 marked this pull request as ready for review July 21, 2026 09:18
@robert3005
robert3005 enabled auto-merge (squash) July 21, 2026 09:18
@robert3005
robert3005 merged commit def2791 into develop Jul 21, 2026
112 of 116 checks passed
@robert3005
robert3005 deleted the codex/piecewise-sequence-varbin-take branch July 21, 2026 09:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/chore A trivial change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants