diff --git a/papers/P2728.md b/papers/P2728.md index 4e85ac1..114ab29 100644 --- a/papers/P2728.md +++ b/papers/P2728.md @@ -1777,28 +1777,29 @@ the chunk gets filled up, only for the rest of the chunk to get discarded. The reference implementation's benchmark transcodes the [unicode_lipsum](https://github.com/lemire/unicode_lipsum) corpora from UTF-16 to UTF-8. The SIMD implementation uses a prototype kernel written against C++26 `std::simd` -and a buffer capacity of 128 code units. For comparison, the last column is a single bulk +and a buffer capacity of 64 code units. For comparison, the last column is a single bulk `simdutf` call over the whole corpus, with no view involved. Numbers are GiB/s of input consumed (GCC 16.1, `-O3 -march=native`, x86-64 AVX2, AMD Ryzen 9 5950X). | Corpus | Scalar view | Prototype SIMD view | Bulk `simdutf` | |---|---:|---:|---:| -| Latin | 1.82 | 4.01 | 60.4 | -| Arabic | 0.83 | 1.05 | 10.9 | -| Chinese | 0.73 | 0.99 | 8.9 | -| Japanese | 0.70 | 0.76 | 8.7 | -| Korean | 0.72 | 0.76 | 8.0 | +| Latin | 1.86 | 4.20 | 60.4 | +| Arabic | 0.81 | 0.96 | 10.9 | +| Chinese | 0.82 | 1.00 | 8.9 | +| Japanese | 0.77 | 0.85 | 8.7 | +| Korean | 0.89 | 0.86 | 8.0 | In the current prototype, we see >2x speedup on the most favorable case, which is ASCII -input, and moderate to small speedups on other corpuses. (Note that the prototype is +input, and either parity or small speedups on other corpuses. (Note that the prototype is currently in an incomplete state, and only implements the UTF16-UTF8 direction, so results -with other directions may differ). `simdutf` smokes us, mainly due to benefiting from the -bulk API, and we can't approach its speed with a view; we would need to do an algorithm to -achieve comparable performance.\* +with other directions may differ). 64 is the minimum buffer size at which we get favorable +results, for this particular transcoding direction, in my prototype. `simdutf` smokes us, +mainly due to benefiting from the bulk API, and we can't approach its speed with a view; +we would need to do an algorithm to achieve comparable performance.\* To put these numbers into perspective, the article text of English Wikipedia is roughly -40 GiB in UTF-8, or about 80 GiB in UTF-16, and so would take approximately 44 seconds -to transcode on a single core with the scalar implementation and 20 seconds to +40 GiB in UTF-8, or about 80 GiB in UTF-16, and so would take approximately 43 seconds +to transcode on a single core with the scalar implementation and 19 seconds to transcode with SIMD, assuming its properties are roughly similar to the Latin corpus above.