Skip to content

Commit 121dea5

Browse files
authored
Release v1.1.0
2 parents 10928ec + dda9489 commit 121dea5

14 files changed

Lines changed: 779 additions & 249 deletions

.github/workflows/ci.yml

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
name: Elixir CI
2+
3+
on:
4+
pull_request:
5+
branches:
6+
- main
7+
- dev
8+
9+
jobs:
10+
test:
11+
name: Test (Elixir ${{ matrix.elixir }}, OTP ${{ matrix.otp }})
12+
runs-on: ubuntu-latest
13+
strategy:
14+
fail-fast: false
15+
matrix:
16+
include:
17+
- elixir: '1.18.4'
18+
otp: '28.0'
19+
version-type: 'strict'
20+
- elixir: '1.12.0'
21+
otp: '24.x'
22+
version-type: 'loose'
23+
steps:
24+
- name: Checkout Repository
25+
uses: actions/checkout@v4
26+
27+
- name: Setup Elixir
28+
uses: erlef/setup-beam@v1
29+
with:
30+
elixir-version: ${{ matrix.elixir }}
31+
otp-version: ${{ matrix.otp }}
32+
version-type: ${{ matrix.version-type }}
33+
34+
- name: Install Dependencies
35+
run: mix deps.get
36+
37+
- name: Run Tests
38+
run: mix test

CHANGELOG.md

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,29 @@ All notable changes to this project will be documented in this file.
55
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
66
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
77

8+
## [v1.1.0] - 2025-08-09
9+
10+
### Added
11+
12+
- Added a new Streaming API that processes data in chunks, reducing peak memory
13+
usage when handling large datasets or network streams
14+
- Introduced `Msgpack.encode_stream/2` to lazily encode a stream of Elixir
15+
terms one by one
16+
- Introduced `Msgpack.decode_stream/2` to lazily decode a stream of
17+
MessagePack objects, capable of handling data that arrives in multiple
18+
chunks
19+
- Added CI workflow to run tests against supported Elixir versions
20+
21+
### Changed
22+
23+
- Updated minimum supported Elixir version to v1.12
24+
- While the library may work with older versions, StreamData supports a
25+
minimum of v1.12, so it would be missing the property tests
26+
27+
### Fixed
28+
29+
- Updated timestamp decoding to be backwards-compatible with Elixir v1.12
30+
831
## [v1.0.2] - 2025-08-06
932

1033
### Fixed

README.md

Lines changed: 28 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,9 @@ types.
2020
limits to mitigate resource exhaustion from malformed or malicious payloads.
2121
- **Telemetry Integration:** Emits standard `:telemetry` events for integration
2222
with monitoring tools.
23+
- **Streaming API:** Process large collections or continuous data streams with
24+
low memory overhead using `Msgpack.encode_stream/2` and
25+
`Msgpack.decode_stream/2`.
2326

2427
## Installation
2528

@@ -50,6 +53,27 @@ iex> Msgpack.decode!(<<0xC1>>)
5053
** (Msgpack.DecodeError) Unknown type prefix: 193. The byte `0xC1` is not a valid MessagePack type marker.
5154
```
5255

56+
### Streaming Large Collections
57+
58+
For datasets that may not fit in memory, you can use the streaming API, which
59+
processes one term at a time.
60+
61+
```elixir
62+
# Create a lazy stream of terms to be encoded.
63+
iex> terms = Stream.cycle([1, "elixir", true])
64+
65+
# The output is a lazy stream of encoded binaries.
66+
iex> encoded_stream = Msgpack.encode_stream(terms)
67+
68+
# The stream is only consumed when you enumerate it.
69+
iex> encoded_stream |> Stream.take(3) |> Enum.to_list()
70+
[
71+
{:ok, <<1>>},
72+
{:ok, <<166, 101, 108, 105, 120, 105, 114>>},
73+
{:ok, <<195>>}
74+
]
75+
```
76+
5377
## Full Documentation
5478

5579
For detailed information on all features, options, and functions, see the [full
@@ -62,7 +86,10 @@ This section explains how to setup the project locally for development.
6286

6387
### Dependencies
6488

65-
- Elixir `~> 1.7` (OTP 21+)
89+
- Elixir `~> 1.12` (OTP 24+)
90+
- See [Compatibility and
91+
deprecations](https://hexdocs.pm/elixir/1.18.4/compatibility-and-deprecations.html)
92+
for more information
6693

6794
### Get the Source
6895

lib/msgpack.ex

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,8 @@ defmodule Msgpack do
4949

5050
alias Msgpack.Encoder
5151
alias Msgpack.Decoder
52+
alias Msgpack.StreamEncoder
53+
alias Msgpack.StreamDecoder
5254
alias Msgpack.EncodeError
5355
alias Msgpack.DecodeError
5456

@@ -314,4 +316,63 @@ defmodule Msgpack do
314316
raise DecodeError, reason: reason
315317
end
316318
end
319+
320+
@doc """
321+
Encodes a stream of Elixir terms into a stream of MessagePack binaries.
322+
323+
Each term in the input enumerable is encoded individually. The output stream
324+
will contain `{:ok, binary}` tuples for successful encodings or `{:error,
325+
reason}` tuples for failures.
326+
327+
This function delegates to `Msgpack.StreamEncoder.encode/2`.
328+
329+
## Options
330+
331+
Accepts the same options as `Msgpack.encode/2`.
332+
333+
## Examples
334+
335+
```elixir
336+
iex> terms = [1, "elixir", :world]
337+
iex> Msgpack.encode_stream(terms, atoms: :string) |> Enum.to_list()
338+
[
339+
{:ok, <<1>>},
340+
{:ok, <<166, 101, 108, 105, 120, 105, 114>>},
341+
{:ok, <<165, 119, 111, 114, 108, 100>>}
342+
]
343+
```
344+
"""
345+
@spec encode_stream(Enumerable.t(), StreamEncoder.opts_t()) :: StreamEncoder.t()
346+
def encode_stream(enumerable, opts \\ []) do
347+
StreamEncoder.encode(enumerable, opts)
348+
end
349+
350+
@doc """
351+
Decodes a stream of MessagePack binaries into a stream of Elixir terms.
352+
353+
This function provides a streaming, lazy interface for decoding, making it
354+
suitable for handling large datasets that do not fit into memory.
355+
356+
It delegates to `Msgpack.StreamDecoder.decode/2`.
357+
358+
For more detailed information on behavior, see the `Msgpack.StreamDecoder`
359+
module documentation.
360+
361+
## Options
362+
363+
Accepts the same options as `Msgpack.decode/2`.
364+
365+
## Examples
366+
367+
```elixir
368+
iex> objects = [1, "elixir", true]
369+
iex> stream = Enum.map(objects, &Msgpack.encode!/1)
370+
iex> Msgpack.decode_stream(stream) |> Enum.to_list()
371+
[1, "elixir", true]
372+
```
373+
"""
374+
@spec decode_stream(Enumerable.t(binary()), StreamDecoder.opts_t()) :: StreamDecoder.t()
375+
def decode_stream(enumerable, opts \\ []) do
376+
StreamDecoder.decode(enumerable, opts)
377+
end
317378
end

0 commit comments

Comments
 (0)