Commit 725fbc5
ORC-1961: Support
### What changes were proposed in this pull request?
This PR aims to support `orc.compression.zstd.strategy`.
### Why are the changes needed?
To allow a user to choose a proper strategy based on their data.
https://facebook.github.io/zstd/zstd_manual.html#Chapter5
```
typedef enum { ZSTD_fast=1,
ZSTD_dfast=2,
ZSTD_greedy=3,
ZSTD_lazy=4,
ZSTD_lazy2=5,
ZSTD_btlazy2=6,
ZSTD_btopt=7,
ZSTD_btultra=8,
ZSTD_btultra2=9
/* note : new strategies _might_ be added in the future.
Only the order (from fast to strong) is guaranteed */
} ZSTD_strategy;
```
### How was this patch tested?
Pass the CIs.
```
$ cd java
$ mvn package -DskipTests -Pbenchmark
$ cd bench
$ time java -Dorc.compression.zstd.strategy=1 -jar core/target/orc-benchmarks-core-*-uber.jar generate data -d sales -c zstd -f orc
...
54.51s user 1.28s system 103% cpu 53.984 total
$ time java -Dorc.compression.zstd.strategy=9 -jar core/target/orc-benchmarks-core-*-uber.jar generate data -d sales -c zstd -f orc
...
148.21s user 1.75s system 101% cpu 2:28.13 total
```
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #2338 from dongjoon-hyun/ORC-1961.
Authored-by: Dongjoon Hyun <dongjoon@apache.org>
Signed-off-by: William Hyun <william@apache.org>orc.compression.zstd.strategy
1 parent 09693fe commit 725fbc5
4 files changed
Lines changed: 34 additions & 6 deletions
File tree
- java/core/src/java/org/apache/orc
- impl
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
80 | 80 | | |
81 | 81 | | |
82 | 82 | | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
83 | 87 | | |
84 | 88 | | |
85 | 89 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
429 | 429 | | |
430 | 430 | | |
431 | 431 | | |
| 432 | + | |
432 | 433 | | |
433 | 434 | | |
434 | 435 | | |
| |||
445 | 446 | | |
446 | 447 | | |
447 | 448 | | |
| 449 | + | |
| 450 | + | |
| 451 | + | |
| 452 | + | |
| 453 | + | |
| 454 | + | |
| 455 | + | |
| 456 | + | |
448 | 457 | | |
449 | 458 | | |
450 | 459 | | |
| |||
520 | 529 | | |
521 | 530 | | |
522 | 531 | | |
| 532 | + | |
| 533 | + | |
523 | 534 | | |
524 | 535 | | |
525 | 536 | | |
| |||
Lines changed: 1 addition & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
121 | 121 | | |
122 | 122 | | |
123 | 123 | | |
| 124 | + | |
124 | 125 | | |
125 | 126 | | |
126 | 127 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
29 | 29 | | |
30 | 30 | | |
31 | 31 | | |
32 | | - | |
33 | | - | |
| 32 | + | |
| 33 | + | |
34 | 34 | | |
35 | 35 | | |
36 | 36 | | |
37 | | - | |
| 37 | + | |
38 | 38 | | |
39 | 39 | | |
40 | 40 | | |
| |||
57 | 57 | | |
58 | 58 | | |
59 | 59 | | |
| 60 | + | |
60 | 61 | | |
61 | | - | |
| 62 | + | |
62 | 63 | | |
63 | 64 | | |
| 65 | + | |
64 | 66 | | |
65 | 67 | | |
66 | 68 | | |
67 | 69 | | |
68 | | - | |
| 70 | + | |
69 | 71 | | |
70 | 72 | | |
71 | 73 | | |
| |||
123 | 125 | | |
124 | 126 | | |
125 | 127 | | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
126 | 135 | | |
127 | 136 | | |
128 | 137 | | |
| |||
136 | 145 | | |
137 | 146 | | |
138 | 147 | | |
| 148 | + | |
139 | 149 | | |
140 | 150 | | |
141 | 151 | | |
142 | 152 | | |
143 | 153 | | |
144 | 154 | | |
145 | 155 | | |
| 156 | + | |
146 | 157 | | |
147 | 158 | | |
148 | 159 | | |
149 | 160 | | |
150 | 161 | | |
151 | | - | |
| 162 | + | |
152 | 163 | | |
153 | 164 | | |
154 | 165 | | |
| |||
183 | 194 | | |
184 | 195 | | |
185 | 196 | | |
| 197 | + | |
186 | 198 | | |
187 | 199 | | |
188 | 200 | | |
| |||
0 commit comments