Zstd announcement

content/_index.md

@@ -30,6 +30,7 @@ supporters = [
 ]
 
 blogposts = [
+    "Announcing Zstandard in Rust",
     "Three years of Rusty sudo",
     "Announcing ntpd-rs in Ubuntu",
     "Video: Securing time with NTS (FOSDEM26)",

content/blog/Announcing Zstandard in Rust.md

@@ -0,0 +1,87 @@
++++
+title = "Announcing Zstandard in Rust"
+slug = "announcing-zstandard-in-rust"
+authors = ["Folkert de Vries"]
+date = "2026-05-28"
+
+[taxonomies]
+tags = ["libzstd-rs", "announcement"] 
+
++++
+
+Over the past year, we've been silently working on our third compression project. After [zlib](https://github.com/trifectatechfoundation/zlib-rs) and [bzip2](https://github.com/trifectatechfoundation/libbzip2-rs) we're now taking on zstd with [libzstd-rs-sys](https://github.com/trifectatechfoundation/libzstd-rs-sys), and are proud to announce its first release.
+
+<!-- more -->
+
+Zstd is a compression format that has been designed with modern CPUs in mind. It is both much faster and able to compress better than gzip. It is already widely used, and we expect that it will continue to gradually replace gzip for web traffic.
+
+## Why though? 
+
+Using zstd in Rust is already supported via the [zstd](https://crates.io/crates/zstd) crate, so why bother with a whole new implementation?
+
+### Portability
+
+One practical advantage of Rust is that it is straightforward to write portable software. Currently the `zstd` crate compiles the C code from source, which requires a C toolchain for the target and for the target to be supported at all. Setting up a C toolchain on Windows or for webassembly can be a challenge, and isn't needed with a pure-Rust implementation. For Rust programmers, using dependencies written in Rust is a superior experience.
+
+### Drop-in replacement
+
+Additionally, like with zlib and bzip2, we support compiling `libzstd-rs-sys` into a drop-in compatible C library. Hence we are, or aim to be, an alternative to the C reference implementation. 
+
+### Strengthening the ecosystem
+
+The [C reference implementation](https://github.com/facebook/zstd) is maintained by Meta, and requires signing a contributor agreement with them in order to contribute. We believe that an independent, performant and compatible implementation strengthens the open source ecosystem.
+
+## Current state
+
+The reference implementation was initially translated using [`c2rust`](https://c2rust.com/), and we have since completed the cleanup work for decompression and the dictionary builder. 
+
+We test our Rust code (compiled into a C static library) with the the reference implementation's test suite. We additionally use fuzz testing and Miri, so we're confident in the correctness of our implementation.
+
+The pre-release is available here: <TODO: link to release>.
+
+This work has also had ecosystem benefits: we've found several limitations of Miri (that are now resolved) and made contributions to Clippy. A more complete write-up of our recent contributions can be found [here](https://trifectatech.org/blog/fixing-our-own-problems-in-the-rust-compiler/). 
+
+### The cost of memory safety
+
+By default decompression performance of our implementation is a few percent slower than the C reference implementation. We benchmark each merge in to `main` in our [benchmark suite](https://trifectatechfoundation.github.io/libzstd-rs-sys-bench/).
+
+We believe we can justify this regression though, because with the `unsafe-performance-experimental` feature flag enabled we match C performance. This feature flag disables 4 bounds checks where data from the input is used to index into a data structure. For most users a ~3% performance regression is likely an acceptable price to pay for increased memory safety. If you really do need that last bit of performance, you can enable the flag at your own risk. Its behavior in these four locations matches C which also does not check the bounds and appears to run just fine in many production systems.
+
+## Future
+
+We are looking for funding of the compression portion of this library. 
+
+Because of code sharing between compression and decompression, we have looked at the compression code a bit, but most of the cleanup work still needs to be done. We did set up benchmarks to ensure compression performance does not unexpectedly regress, and as mentioned we already use the reference implementation's test suite to check that we produce the correct result. 
+
+### Ecosystem integration
+
+We have our own fork of the `zstd` that uses `libzstd-rs-sys` instead of the C library. We'd like to upstream this at some point. 
+
+For the most commonly-used APIs this is straightforward. For the `experimental` features we run into some mismatches where `zstd-safe` uses an enum but we must use a `struct` for FFI safety.
+
+## Thanks to our sponsors
+
+The work on the decompression side has been funded by [Chainguard](https://www.chainguard.dev/), [Astral](https://astral.sh/) and [NLnet Foundation](https://nlnet.nl/), and we're grateful for their support! [Sovereign Tech Agency](https://sovereign.tech) invested in the dictionary builder; thank you!
+
+---
+
+### About 
+
+[**Trifecta Tech Foundation**](https://trifectatech.org) is a non-profit and a Public Benefit Organisation (501(c)(3) equivalent) that creates open-source building blocks for critical infrastructure software. Our initiatives on data compression, time synchronization, and privilege boundary, impact the digital security of millions of people. If you’d like to support our work, please contact us; see [trifectatech.org/support](https://trifectatech.org/support/).
+
+[**Astral**](https://astral.sh/) builds high-performance developer tools for the Python ecosystem: 
+- Ruff, an extremely fast Python linter, written in Rust.
+- uv, an extremely fast Python package manager, written in Rust.
+
+Astral's mission is to make the Python ecosystem more productive. Learn more at [astral.sh](https://astral.sh/).
+
+[**NLnet Foundation**](https://nlnet.nl/) is a recognised philanthropic non-profit foundation. The foundation stimulates network research and development in the domain of Internet technology. The articles of association for the NLnet Foundation state: "to promote the exchange of electronic information and all that is related or beneficial to that purpose". The prefered instrument of NLnet is awarding microgrants to small, independent projects supporting independent researchers and developers. Read more on [nlnet.nl](https://nlnet.nl/).
+
+[**Chainguard**](https://www.chainguard.dev/) builds trusted open source software for a secure-by-default stack. Read more on [chainguard.dev](https://www.chainguard.dev/)
+
+### Further Information
+
+- [Data compression initiative on Trifecta Tech Foundation](https://trifectatech.org/initiatives/data-compression)
+- [libzstd-rs GitHub Repository](https://github.com/trifectatechfoundation/libzstd-rs-sys)
+
+*For inquiries, please contact Erik Jonkers, [contact@trifectatech.org](mailto:contact@trifectatech.org)*

content/initiatives/data-compression.md

@@ -20,7 +20,8 @@ funders = [
     "chainguard",
     "ngi-zero-core",
     "min-bzk",
-    "astral"
+    "astral",
+    "sta"
 ]
 
 supporters = [
@@ -31,6 +32,7 @@ supporters = [
 ]
 
 blogposts = [
+    "Announcing Zstandard in Rust",
     "Compression compiler contributions",
     "zlib-rs stable API",
     "Emulating avx-512 intrinsics in Miri",

content/initiatives/workplans/workplan-zstd-rs.md

@@ -9,12 +9,32 @@ backLink = "projects/libzstd-rs"
 backTitle = "Back to project: Zstandard in Rust"
 +++
 
-Work on zstd started in July 2025. The first milestone is the decompression portion of the library.
-
-Now the decompression is in development, we're seeking funding for the *encoder*. Please [get in touch with us](/support), if you are interested in financially supporting the development of memory-safe zstd.
+Work on zstd started in July 2025. The first three milestones are now completed.
 
 ### In progress milestones
 
+We're seeking funding for the *encoder*. Please [get in touch with us](/support), if you are interested in financially supporting the development of memory-safe zstd.
+
+---
+
+### Future work
+
+**Milestone 4: Encoder implementation**
+
+This milestone adds API-compatible zstd compression.
+
+- Improving the code quality of the translated code
+- Improving the performance of the translated code
+- Testing & fuzzing
+- Audit
+- Create release and publish crates
+
+Please contact us if you are interested in supporting this milestone; see [trifectatech.org/support](https://trifectatech.org/support/). 
+
+---
+
+### Completed milestones
+
 **Milestone 1: Decoder implementation**
 
 This milestone adds API-compatible zstd decompression functionality.
@@ -44,22 +64,6 @@ This work is supported by [NLnet Foundation](https://nlnet.nl).
 
 ---
 
-### Future work
-
-**Milestone 4: Encoder implementation**
-
-This milestone adds API-compatible zstd compression.
-
-- Improving the code quality of the translated code
-- Improving the performance of the translated code
-- Testing & fuzzing
-- Audit
-- Create release and publish crates
-
-Please contact us if you are interested in supporting this milestone; see [trifectatech.org/support](https://trifectatech.org/support/). 
-
----
-
 
 
 

content/news.md

@@ -5,6 +5,7 @@ template = "news.html"
 
 [extra]
 blogposts = [
+    "Announcing Zstandard in Rust",
     "Three years of Rusty sudo",
     "Tail calls project goal",
     "Compression compiler contributions",

content/projects/libzstd-rs.md

@@ -24,6 +24,7 @@ supporters = [
 ]
 
 blogposts = [
+    "Announcing Zstandard in Rust",
     "Compression compiler contributions"
 ]
 +++
@@ -36,15 +37,15 @@ For the high level goals of our data compression projects, see our [Data compres
 
 ### About
 
-Development of Zstandard in Rust began in July 2025, with the first release of the decoder planned for March 2026.
+Development of Zstandard in Rust began in July 2025. In May of 2026 we released the first version that includes the decoder and the dictionaty builder.
 
 We thank [Chainguard](https://www.chainguard.dev/), [Astral](https://astral.sh/), [Sovereign Tech Agency](https://sovereign.tech) and [NLnet Foundation](https://nlnet.nl) for their support.
 
 ### Roadmap
 
-- 2025 Q3: start of decoder implementation
-- 2025 Q4: implementation of dictBuilder feature
-- 2026 Q1: release of decoder
+- 2025 Q3: start of decoder implementation (*completed*)
+- 2025 Q4: implementation of dictBuilder feature (*completed*)
+- 2026 Q1: release of decoder (*completed*)
 - 2026 Q2: start of encoder implementation (*pending funding*)
 
 For details see [the workplan](/initiatives/workplans/libzstd-rs/).