All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog and this project adheres to Semantic Versioning.
- Fixed
select_replacematch spec input_newer/5andput_all_newer/3when values contain maps (including maps nested inside tuples or lists). Maps in match spec bodies are now wrapped with{:const, map}viams_literal/1, which tells ETS to treat them as opaque literals.
:tablemode for:processing_batch_sizeoption. When set to:table, the ETS table name is passed directly to the processor instead of reading and batching the data. This gives the processor full control over how it reads and processes the table. The processor can optionally keep the table for later processing by calling:ets.rename/2to free the original name and:ets.give_away/3to transfer it to another process. #14.- Added ASCII data flow diagrams to
PartitionedBuffer.Queue,PartitionedBuffer.Map, andPartitionedBuffer.Partitionmodule docs. - Minor documentation and comment fixes (grammar, typos, consistency).
- Migrated CI from CircleCI to GitHub Actions.
PartitionedBuffer.Mapprocessor now receives{key, value, version, updates}tuples instead of{key, {value, updates}}. Theversionfield is the entry version set viaput_newer/5andput_all_newer/3(0for regularput/4entries). Existing processor functions must update their pattern matching accordingly.
- Fixed
select_replacematch spec input_newer/5andput_all_newer/3when keys or values are tuples or lists. Bare tuples in match spec bodies are interpreted as operations by ETS, not as literal data. Addedms_literal/1to wrap tuple/list values using the{{...}}constructor form.
update_options/2forPartitionedBuffer.QueueandPartitionedBuffer.Map. Allows updatingprocessing_interval_ms,processing_timeout_ms, andprocessing_batch_sizeat runtime across all partitions.
- The
:processoroption now accepts an MFA tuple{Module, Function, Args}in addition to anonymous functions. The batch is prepended to the arguments.
PartitionedBuffer.Mapprocessor now receives{key, {value, updates}}tuples instead of{key, value}. Theupdatesfield tracks the number of times an existing key was updated (only for versioned updates viaput_newer/5andput_all_newer/3; regularput/4entries always haveupdatesset to0).
PartitionedBuffer.Map— New key-value map buffer using:setETS tables with last-write-wins semantics. Exposesput/4,put_all/3,get/3,delete/3,size/1,stop/3,start_link/1, etc.PartitionedBuffer.Map—put_newer/5andput_all_newer/3for versioned conditional updates with "newer version wins" semantics. Uses ETSinsert_new+select_replacewith a literal key for O(1) atomic updates.
- Refactored the previous monolithic implementation into
PartitionedBuffer.Queue, extracting it as a concrete buffer implementation behind thePartitionedBufferbehaviour. This sets the foundation for multiple buffer types (e.g.,Map).
- Moved
sizefrom metadata to measurements in the processing Telemetry span. This change prevents high cardinality tags in Datadog, reducing monitoring costs while still allowing size to be tracked as a metric.
- [sc-83498] Initial implementation for the partitioned buffer.