Add PiecewiseSequence take microbenchmarks#8830
Conversation
18df62f to
04a21ec
Compare
9adf56d to
c4a0358
Compare
0906157 to
f5c8555
Compare
80f7b89 to
30870aa
Compare
30870aa to
e777ab4
Compare
f5c8555 to
b59c45f
Compare
e777ab4 to
e4a0543
Compare
b59c45f to
388d234
Compare
|
I think we want benchmarks. I came up with the long term solution in #8825 and we have to make sure we don't regress |
490e9db to
ed7bbe2
Compare
388d234 to
5f0d64e
Compare
ed7bbe2 to
efafe71
Compare
5f0d64e to
245abc0
Compare
Merging this PR will not alter performance
Performance Changes
Comparing Footnotes
|
efafe71 to
56b0394
Compare
245abc0 to
baaab82
Compare
56b0394 to
80bfa43
Compare
0121842 to
4600066
Compare
80bfa43 to
09ca199
Compare
431cbd5 to
4bb6ce4
Compare
4bb6ce4 to
a42849f
Compare
9183265 to
182567a
Compare
87f6283 to
8235e03
Compare
|
I think in order to merge them we have to make them smaller, these should really be fast microbenchmarks |
Signed-off-by: Daniel King <dan@spiraldb.com>
Signed-off-by: Daniel King <dan@spiraldb.com>
Signed-off-by: Daniel King <dan@spiraldb.com>
Signed-off-by: Daniel King <dan@spiraldb.com>
a42849f to
9995892
Compare
Signed-off-by: Daniel King <dan@spiraldb.com>
Polar Signals Profiling ResultsLatest Run
Previous Runs (1)
Powered by Polar Signals Cloud |
Benchmarks: Vortex queries 📖Verdict: No clear signal (medium confidence) How to read Verdict and Engines
datafusion / vortex-file-compressed (1.272x ❌, 0↑ 2↓)
datafusion / parquet (1.186x ❌, 0↑ 2↓)
duckdb / vortex-file-compressed (1.179x ❌, 0↑ 2↓)
duckdb / parquet (1.068x ➖, 0↑ 0↓)
No file size changes detected. |
|
@robert3005 each one is <=500us now but I could bring them down further if you like. |
|
we generally held the bar at 1ms so this is good enough |
…nnel The raw `oneshot` crate has a receiver-waker drop bug (vortex-data#6221/vortex-data#8830 class) that can strand a completed task's wake inside the buffer_unordered(handle.spawn(..)) scan composition, presenting as the never-ready cold-tier freeze (POLLED-THEN-PENDING scan, spawned==completed, backlog>0). Swap Handle::spawn/spawn_io/spawn_cpu/spawn_blocking to tokio::sync::oneshot, whose receiver waker is robust. Raw `oneshot` crate stays for other call sites (single.rs/tokio.rs).
@robert3005 I'm not sure if you want to check these in. These are what I used to reason about various ways to copy slices of data from one spot in memory to another.
Summary
piecewise_sequence_take_primitiveto compare constant-length PiecewiseSequence take against dense lengths and reifying constant lengthstake_slices_to_buffer_matrixto compare extend/cursor-copy loops with and without preverification plus unchecked slicingValidation
cargo +nightly fmt --allcargo clippy -p vortex-array --all-targets --all-features -- -D warningscargo bench -p vortex-array --bench piecewise_sequence_take_primitivecargo bench -p vortex-array --bench take_slices_to_buffer_matrix