Skip to content

Commit e5e6701

Browse files
committed
Added Compression Class functionality with different compression classes.
- Added first elements to support stream compression and also cleaned up auto-properties by providing [DebuggerStepThrough] attributes. - Added initial MemoryStreamSlim.Compress methods for passing a source stream. - Added Compress method to MemoryStreamSlim for passing byte[] source data. Refactor compression functionality in MemoryStreamSlim - Updated `CompressionType.cs` to include a summary for the `CompressionType` enum, defining the ZLib algorithm. - Enhance string compression in MemoryStreamCompression - Updated `MemoryStreamCompression.cs` to improve string compression functionality. Key changes include reducing the maximum anticipated compression size from 512 MB to 256 MB, adding overloads for `CreateCompressionOutput` to handle string inputs, and modifying Brotli, ZLib, Deflate, and GZip methods to support string data using `StreamWriter`. New public methods for compressing strings with customizable options and encoding were introduced, along with improved documentation for clarity. - Optimize MemoryStreamSlim creation for compression - Introduce new methods for creating MemoryStreamSlim instances with initial capacity based on source data length to improve memory usage during compression. Update existing methods to utilize these new overloads, including support for MemoryStreamSlimOptions. Modify Create methods to accept options by reference, enhancing performance by reducing unnecessary copies. - Added support for asynchronous compression when compressing a source stream. - Introduced `MemoryStreamBrotliCompressionOptions` and `MemoryStreamZLibCompressionOptions` for specific compression settings. - Updated the compression options types to support extended functionality in .NET 9 - Add the dedicated compression extension methods for Deflate, GZip, and ZLib compression types. - Added compression options unit tests - Enhance MemoryStreamSlim functionality and documentation - Updated the MemoryStreamSlimOptions to have the zero buffer behavior be 'init' instead of 'set' to allow initial values on creation of the options. - Added Brotli compression support and updated some MemoryStreamSlim unit tests to support the new options setup configuration. - Implement GZip compression and unit tests - Created or updated `UsingMemoryStreamGZip.cs` to include unit tests for the `MemoryStreamGZip` class, covering multiple scenarios and ensuring data integrity after compression and decompression. Utilized FluentAssertions for improved test readability.
1 parent fbf6145 commit e5e6701

54 files changed

Lines changed: 14085 additions & 267 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

Source/.editorconfig

Lines changed: 243 additions & 111 deletions
Large diffs are not rendered by default.

Source/Docs/articles/memory-management.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@ You can control how the memory buffers are cleared by setting the `MemoryStreamS
2121
using KZDev.PerfUtils;
2222

2323
// Create a new MemoryStreamSlim instance with an expandable initial capacity of 0 bytes, and setting the option to not clear memory buffers
24-
using (Stream stream = MemoryStreamSlim.Create(options => options.ZeroBufferBehavior = MemoryStreamSlimZeroBufferOption.None))
24+
using (Stream stream = MemoryStreamSlim.Create(options => options.WithZeroBufferBehavior(MemoryStreamSlimZeroBufferOption.None))
2525
{
2626
// Read and Write stream operations...
2727
}

Source/Docs/articles/memory-monitoring.md

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -238,7 +238,8 @@ The following table shows the event data.
238238

239239
### MemoryStreamSlimToArray event
240240

241-
This event is raised when the ToArray() method is called on a `MemoryStreamSlim` instance and returns a non-zero length array. This is useful for tracking extra heap memory allocations caused by calling the ToArray() method.
241+
This event is raised when the ToArray() method is called on a `MemoryStreamSlim` instance and returns a non-zero length array. This is useful for tracking extra heap memory allocations caused by calling the ToArray() method. This also records the internal operation of putting all of the
242+
bytes into a contiguous array for decoding the bytes into a string.
242243

243244
The following table shows the task, keyword, level, and opcode.
244245

@@ -256,7 +257,9 @@ The following table shows the event data.
256257

257258
| Name | Type | Description |
258259
| --- | --- | --- |
260+
| StreamId | Guid | The unique identifier for the MemoryStreamSlim instance. |
259261
| ArraySize | Int32 | The size (in bytes) of the returned heap allocated array. |
262+
| StringDecode | Int32 | Indicates if the contiguous array was used for string decoding. 0 = ToArray() called, 1 = Decode() called. |
260263

261264
---
262265

Source/Docs/articles/memorystream-benchmarks.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -89,6 +89,6 @@ This is another reason why the ['Set Loop Count'](./set-loop-count-throughput-be
8989
The benchmarks published here used the following versions of the libraries:
9090

9191
- `BenchmarkDotNet` version: 0.14.0
92-
- `MemoryStreamSlim` version: 1.2.0
92+
- `MemoryStreamSlim` version: 2.0.0
9393
- `RecyclableMemoryStream` version: 3.0.1
94-
- `MemoryStream` version: .NET 8.0.11
94+
- `MemoryStream` version: .NET 9.0.3
Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
# Compression
2+
3+
Compression is often used with memory streams to reduce the size of the data being stored or transmitted. This can be particularly useful when dealing with large amounts of data or when bandwidth is limited.
4+
5+
.NET provides several classes for compression, including `GZipStream` and `DeflateStream`, which can be used in conjunction with memory streams. To simplify the process of compressing and decompressing data, helper classes exist in this library, such as `MemoryStreamGZip` and `MemoryStreamDeflate`. These classes provide a convenient way to handle compression without having to manually manage the underlying streams.
6+
7+
## Helper Classes
8+
9+
The full list of helper classes available for compression are:
10+
11+
- `MemoryStreamGZip`: A helper class that uses GZip compression with a memory stream.
12+
- `MemoryStreamDeflate`: A helper class that uses Deflate compression with a memory stream.
13+
- `MemoryStreamBrotli`: A helper class that uses Brotli compression with a memory stream.
14+
- `MemoryStreamZLib`: A helper class that uses ZLib compression with a memory stream.
Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,14 @@
1+
# Compression
2+
3+
Compression is often used with memory streams to reduce the size of the data being stored or transmitted. This can be particularly useful when dealing with large amounts of data or when bandwidth is limited.
4+
5+
.NET provides several classes for compression, including `GZipStream` and `DeflateStream`, which can be used in conjunction with memory streams. To simplify the process of compressing and decompressing data, helper classes exist in this library, such as `MemoryStreamGZip` and `MemoryStreamDeflate`. These classes provide a convenient way to handle compression without having to manually manage the underlying streams.
6+
7+
## Helper Classes
8+
9+
The full list of helper classes available for compression are:
10+
11+
- `MemoryStreamGZip`: A helper class that uses GZip compression with a memory stream.
12+
- `MemoryStreamDeflate`: A helper class that uses Deflate compression with a memory stream.
13+
- `MemoryStreamBrotli`: A helper class that uses Brotli compression with a memory stream.
14+
- `MemoryStreamZLib`: A helper class that uses ZLib compression with a memory stream.

Source/Docs/articles/memorystreamslim.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,7 @@ Besides creation, the `MemoryStreamSlim` class can be used exactly like the stan
3232
using KZDev.PerfUtils;
3333

3434
// Create a new MemoryStreamSlim instance with an initial capacity of 1024 bytes, and setting the option to not clear memory buffers
35-
using (Stream stream = MemoryStreamSlim.Create(1024, options => options.ZeroBufferBehavior = MemoryStreamSlimZeroBufferOption.None))
35+
using (Stream stream = MemoryStreamSlim.Create(1024, options => options.WithZeroBufferBehavior(MemoryStreamSlimZeroBufferOption.None))
3636
{
3737
// Read and Write stream operations...
3838
}

Source/Docs/articles/stringbuildercache-benchmarks.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -55,5 +55,5 @@ The string segments used to build the strings are pre-allocated (outside of the
5555
The benchmarks published here used the following versions of the libraries:
5656

5757
- `BenchmarkDotNet` version: 0.14.0
58-
- `StringBuilderCache` version: 1.2.0
59-
- `StringBuilder` version: .NET 8.0.11
58+
- `StringBuilderCache` version: 2.0.0
59+
- `StringBuilder` version: .NET 9.0.3

Source/Docs/articles/toc.yml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,8 @@
3131
href: memory-management.md
3232
- name: Monitoring
3333
href: memory-monitoring.md
34+
- name: Compression
35+
href: memorystream-compression.md
3436
- name: Benchmarks
3537
href: memorystream-benchmarks.md
3638
items:

Source/KZDev.PerfUtils.sln

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -85,6 +85,8 @@ Project("{2150E333-8FDC-42A3-9474-1A3956D46DE8}") = "Articles", "Articles", "{B9
8585
Docs\articles\memory-management.md = Docs\articles\memory-management.md
8686
Docs\articles\memory-monitoring.md = Docs\articles\memory-monitoring.md
8787
Docs\articles\memorystream-benchmarks.md = Docs\articles\memorystream-benchmarks.md
88+
Docs\articles\memorystream-compression-benchmarks.md = Docs\articles\memorystream-compression-benchmarks.md
89+
Docs\articles\memorystream-compression.md = Docs\articles\memorystream-compression.md
8890
Docs\articles\MemoryStreamBenchmarks.BulkFillAndReadThroughputBenchmarks-report-github.md = Docs\articles\MemoryStreamBenchmarks.BulkFillAndReadThroughputBenchmarks-report-github.md
8991
Docs\articles\MemoryStreamBenchmarks.BulkFillAndReadThroughputBenchmarks-report.html = Docs\articles\MemoryStreamBenchmarks.BulkFillAndReadThroughputBenchmarks-report.html
9092
Docs\articles\MemoryStreamBenchmarks.ContinuousGrowFillAndReadThroughputBenchmarks-report-github.md = Docs\articles\MemoryStreamBenchmarks.ContinuousGrowFillAndReadThroughputBenchmarks-report-github.md
@@ -155,6 +157,8 @@ Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "KZDev.PerfUtils.Strings.Uni
155157
EndProject
156158
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "StringsBenchmarks", "Perf\StringsBenchmarks\StringsBenchmarks.csproj", "{C648FCE8-1F9F-4E2B-8CC7-52BE3692A8E4}"
157159
EndProject
160+
Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "KZDev.PerfUtils.Memory.Compression.UnitTests", "Tst\KZDev.PerfUtils.Memory.Compression.UnitTests\KZDev.PerfUtils.Memory.Compression.UnitTests.csproj", "{100B56BC-D0CE-4FF0-AD96-C07E412F6689}"
161+
EndProject
158162
Global
159163
GlobalSection(SolutionConfigurationPlatforms) = preSolution
160164
Debug|Any CPU = Debug|Any CPU
@@ -295,6 +299,14 @@ Global
295299
{C648FCE8-1F9F-4E2B-8CC7-52BE3692A8E4}.Profile|Any CPU.Build.0 = Profile|Any CPU
296300
{C648FCE8-1F9F-4E2B-8CC7-52BE3692A8E4}.Release|Any CPU.ActiveCfg = Release|Any CPU
297301
{C648FCE8-1F9F-4E2B-8CC7-52BE3692A8E4}.Release|Any CPU.Build.0 = Release|Any CPU
302+
{100B56BC-D0CE-4FF0-AD96-C07E412F6689}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
303+
{100B56BC-D0CE-4FF0-AD96-C07E412F6689}.Debug|Any CPU.Build.0 = Debug|Any CPU
304+
{100B56BC-D0CE-4FF0-AD96-C07E412F6689}.Package|Any CPU.ActiveCfg = Debug|Any CPU
305+
{100B56BC-D0CE-4FF0-AD96-C07E412F6689}.Package|Any CPU.Build.0 = Debug|Any CPU
306+
{100B56BC-D0CE-4FF0-AD96-C07E412F6689}.Profile|Any CPU.ActiveCfg = Profile|Any CPU
307+
{100B56BC-D0CE-4FF0-AD96-C07E412F6689}.Profile|Any CPU.Build.0 = Profile|Any CPU
308+
{100B56BC-D0CE-4FF0-AD96-C07E412F6689}.Release|Any CPU.ActiveCfg = Release|Any CPU
309+
{100B56BC-D0CE-4FF0-AD96-C07E412F6689}.Release|Any CPU.Build.0 = Release|Any CPU
298310
EndGlobalSection
299311
GlobalSection(SolutionProperties) = preSolution
300312
HideSolutionNode = FALSE
@@ -328,6 +340,7 @@ Global
328340
{2F0EF0E2-B9AC-4BEA-9A1D-3855A76F5C48} = {C95F6A6E-49CC-47F9-AB44-D6904F56D83E}
329341
{F7CED94F-D445-4ACB-AA59-438B3A05068F} = {F8C76B35-0758-433E-953B-0D7E9D63DE13}
330342
{C648FCE8-1F9F-4E2B-8CC7-52BE3692A8E4} = {FC849880-867F-4404-B3A7-A9E0D32B323B}
343+
{100B56BC-D0CE-4FF0-AD96-C07E412F6689} = {F8C76B35-0758-433E-953B-0D7E9D63DE13}
331344
EndGlobalSection
332345
GlobalSection(ExtensibilityGlobals) = postSolution
333346
SolutionGuid = {FE0AB859-B820-470E-AA57-2337B2495B50}

0 commit comments

Comments
 (0)