|
| 1 | +# StorageFlow.Core |
| 2 | + |
| 3 | +StorageFlow is a provider-agnostic object storage framework for .NET. It gives |
| 4 | +application code one fluent API for AWS S3, MinIO, and RustFS while keeping |
| 5 | +provider SDKs outside business logic. |
| 6 | + |
| 7 | +Use StorageFlow when you want typed configuration, reusable policies, streaming |
| 8 | +I/O, consistent results, and the freedom to select a registered provider per |
| 9 | +operation. |
| 10 | + |
| 11 | +## Install |
| 12 | + |
| 13 | +Install Core and at least one provider package: |
| 14 | + |
| 15 | +```bash |
| 16 | +dotnet add package StorageFlow.Core |
| 17 | +dotnet add package StorageFlow.Provider.Minio |
| 18 | +``` |
| 19 | + |
| 20 | +Available providers: |
| 21 | + |
| 22 | +- [`StorageFlow.Provider.S3`](https://www.nuget.org/packages/StorageFlow.Provider.S3) |
| 23 | +- [`StorageFlow.Provider.Minio`](https://www.nuget.org/packages/StorageFlow.Provider.Minio) |
| 24 | +- [`StorageFlow.Provider.RustFs`](https://www.nuget.org/packages/StorageFlow.Provider.RustFs) |
| 25 | + |
| 26 | +## Quick start |
| 27 | + |
| 28 | +Define strongly typed policy keys: |
| 29 | + |
| 30 | +```csharp |
| 31 | +using StorageFlow.Abstractions.Interfaces; |
| 32 | + |
| 33 | +public sealed class DocumentsPolicy : IValidationPolicyKey; |
| 34 | +public sealed class MediaNaming : INamingPolicyKey; |
| 35 | +public sealed class DownloadUrl : IPresignedUrlPolicyKey; |
| 36 | +public sealed class PublicAssets : IDeliveryUrlPolicyKey; |
| 37 | +``` |
| 38 | + |
| 39 | +Register policies and a provider: |
| 40 | + |
| 41 | +```csharp |
| 42 | +using StorageFlow.Core.DependencyInjection; |
| 43 | +using StorageFlow.Provider.Minio; |
| 44 | + |
| 45 | +builder.Services.AddStorageFlow(options => |
| 46 | +{ |
| 47 | + options.Validation.AddPolicy<DocumentsPolicy>(policy => |
| 48 | + { |
| 49 | + policy.MaxFileSizeBytes = 10 * 1024 * 1024; |
| 50 | + policy.AllowedExtensions = [".pdf", ".zip"]; |
| 51 | + policy.AllowedMimeTypes = ["application/pdf", "application/zip"]; |
| 52 | + policy.RequireValidSignature = true; |
| 53 | + }); |
| 54 | + |
| 55 | + options.Naming |
| 56 | + .AddPolicy<MediaNaming>(policy => |
| 57 | + policy.UsePattern("{yyyy}/{MM}/{slug}-{guid}{ext}")) |
| 58 | + .AsDefault(); |
| 59 | + |
| 60 | + options.PresignedUrls.AddPolicy<DownloadUrl>(policy => |
| 61 | + { |
| 62 | + policy.Expiration = TimeSpan.FromMinutes(15); |
| 63 | + policy.HttpMethod = HttpMethod.Get; |
| 64 | + }); |
| 65 | + |
| 66 | + options.DeliveryUrls.AddPolicy<PublicAssets>(policy => policy |
| 67 | + .UseCdn("https://cdn.example.com") |
| 68 | + .WithPathPrefix("assets")); |
| 69 | + |
| 70 | + options.Providers.UseMinio(minio => minio.Configure(config => |
| 71 | + { |
| 72 | + config.Endpoint = builder.Configuration["Storage:Minio:Endpoint"]!; |
| 73 | + config.AccessKey = builder.Configuration["Storage:Minio:AccessKey"]!; |
| 74 | + config.SecretKey = builder.Configuration["Storage:Minio:SecretKey"]!; |
| 75 | + config.UseSSL = true; |
| 76 | + })).AsDefault(); |
| 77 | +}); |
| 78 | +``` |
| 79 | + |
| 80 | +Upload a stream: |
| 81 | + |
| 82 | +```csharp |
| 83 | +var result = await storage |
| 84 | + .Validation<DocumentsPolicy>() |
| 85 | + .CacheControl("private, max-age=0") |
| 86 | + .ContentDisposition("attachment") |
| 87 | + .Metadata("source", "documents-api") |
| 88 | + .FromStream(stream, fileName, contentType, contentLength) |
| 89 | + .UploadAsync("documents", cancellationToken); |
| 90 | + |
| 91 | +if (!result.IsSuccess) |
| 92 | +{ |
| 93 | + logger.LogWarning("{Code}: {Message}", |
| 94 | + result.Error!.Code, |
| 95 | + result.Error.Message); |
| 96 | + return; |
| 97 | +} |
| 98 | + |
| 99 | +var objectKey = result.Value!.ObjectKey; |
| 100 | +var etag = result.Value.ETag; |
| 101 | +``` |
| 102 | + |
| 103 | +The upload stream remains owned by the caller. StorageFlow does not dispose it. |
| 104 | + |
| 105 | +## Validation |
| 106 | + |
| 107 | +Validation policies can enforce: |
| 108 | + |
| 109 | +- minimum and maximum file size; |
| 110 | +- allowed and blocked extensions; |
| 111 | +- allowed MIME types; |
| 112 | +- magic-number/file-signature checks. |
| 113 | + |
| 114 | +Extension and MIME allowlists are configuration driven. You can list any |
| 115 | +extension or MIME type required by your application. |
| 116 | + |
| 117 | +Built-in magic-number validation currently recognizes: |
| 118 | + |
| 119 | +| Format | Supported extensions | Signature | |
| 120 | +|---|---|---| |
| 121 | +| JPEG | `.jpg`, `.jpeg` | JPEG SOI prefix | |
| 122 | +| PNG | `.png` | PNG signature | |
| 123 | +| PDF | `.pdf` | `%PDF` prefix | |
| 124 | +| ZIP | `.zip` | Standard, empty, and spanning ZIP prefixes | |
| 125 | +| MP3 | `.mp3` | MPEG frame prefixes or ID3 tag | |
| 126 | +| MP4 | `.mp4` | ISO base media `ftyp` box | |
| 127 | + |
| 128 | +`RequireValidSignature = true` validates known formats. For an extension not in |
| 129 | +the table, StorageFlow currently skips the signature step; use a custom |
| 130 | +`IFileValidator` when that format must be verified. Extension and MIME checks |
| 131 | +still run normally. |
| 132 | + |
| 133 | +Custom validators are resolved from dependency injection and ordered by their |
| 134 | +`Order` value: |
| 135 | + |
| 136 | +```csharp |
| 137 | +builder.Services.AddStorageFlowValidator<MalwareScanValidator>(); |
| 138 | +``` |
| 139 | + |
| 140 | +## Naming |
| 141 | + |
| 142 | +Naming policies support GUID, SEO, pattern, and custom strategies: |
| 143 | + |
| 144 | +```csharp |
| 145 | +options.Naming.AddPolicy<GuidNaming>(policy => policy.UseGuid()); |
| 146 | +options.Naming.AddPolicy<SeoNaming>(policy => policy.UseSeo()); |
| 147 | +options.Naming.AddPolicy<MediaNaming>(policy => |
| 148 | + policy.UsePattern("{yyyy}/{MM}/{dd}/{slug}-{guid}{ext}")); |
| 149 | +``` |
| 150 | + |
| 151 | +Pattern tokens: `{yyyy}`, `{MM}`, `{dd}`, `{guid}`, `{slug}`, `{ext}`, and |
| 152 | +`{timestamp}`. |
| 153 | + |
| 154 | +An operation uses its explicit naming policy, then the global default, then a |
| 155 | +GUID fallback. Provider-level definitions override a global policy with the |
| 156 | +same key. |
| 157 | + |
| 158 | +## Object operations |
| 159 | + |
| 160 | +Download content as a stream: |
| 161 | + |
| 162 | +```csharp |
| 163 | +var result = await storage |
| 164 | + .Object("documents", objectKey) |
| 165 | + .DownloadAsync(cancellationToken); |
| 166 | + |
| 167 | +if (result.IsSuccess) |
| 168 | +{ |
| 169 | + var download = result.Value!; |
| 170 | + await using var content = download.Content; |
| 171 | + await content.CopyToAsync(destination, cancellationToken); |
| 172 | +} |
| 173 | +``` |
| 174 | + |
| 175 | +`DownloadResult` includes content type, content length, normalized ETag, last |
| 176 | +modified time, and read-only user metadata. The caller owns the returned |
| 177 | +stream. |
| 178 | + |
| 179 | +```csharp |
| 180 | +var exists = await storage |
| 181 | + .Object("documents", objectKey) |
| 182 | + .ExistsAsync(cancellationToken); |
| 183 | + |
| 184 | +var deleted = await storage |
| 185 | + .Object("documents", objectKey) |
| 186 | + .DeleteAsync(cancellationToken); |
| 187 | +``` |
| 188 | + |
| 189 | +## Private and public URLs |
| 190 | + |
| 191 | +Use a presigned URL for temporary private access: |
| 192 | + |
| 193 | +```csharp |
| 194 | +var signed = await storage |
| 195 | + .Object("documents", objectKey) |
| 196 | + .GetPresignedUrlAsync<DownloadUrl>(cancellationToken); |
| 197 | +``` |
| 198 | + |
| 199 | +Use a delivery URL for stable public CDN addresses. Delivery URL generation is |
| 200 | +synchronous and performs no storage or network request: |
| 201 | + |
| 202 | +```csharp |
| 203 | +var publicUrl = storage |
| 204 | + .Object("media", objectKey) |
| 205 | + .GetDeliveryUrl<PublicAssets>(); |
| 206 | +``` |
| 207 | + |
| 208 | +Batch delivery preserves input ordering and duplicate keys: |
| 209 | + |
| 210 | +```csharp |
| 211 | +var urls = storage |
| 212 | + .Objects("media", objectKeys) |
| 213 | + .GetDeliveryUrls<PublicAssets>(); |
| 214 | +``` |
| 215 | + |
| 216 | +## Multiple providers |
| 217 | + |
| 218 | +Register several official providers and mark one default. Select another with |
| 219 | +the autocomplete-friendly `SFProvider` catalog: |
| 220 | + |
| 221 | +```csharp |
| 222 | +var result = await storage |
| 223 | + .Provider(SFProvider.S3) |
| 224 | + .Validation<DocumentsPolicy>() |
| 225 | + .FromStream(stream, fileName, contentType, contentLength) |
| 226 | + .UploadAsync("documents", cancellationToken); |
| 227 | +``` |
| 228 | + |
| 229 | +Provider selection should come from application logic or a typed profile, not |
| 230 | +from an arbitrary client-supplied string. |
| 231 | + |
| 232 | +## Results and errors |
| 233 | + |
| 234 | +Public operations return `StorageResult` or `StorageResult<T>`. Provider SDK |
| 235 | +exceptions do not escape the API. Common error codes include |
| 236 | +`ObjectNotFound`, `BucketNotFound`, `PermissionDenied`, `ProviderError`, |
| 237 | +`ValidationFailed`, and `Unknown`. |
| 238 | + |
| 239 | +ETags are normalized without surrounding quotes, but an ETag is not guaranteed |
| 240 | +to be a content hash. |
| 241 | + |
| 242 | +## More information |
| 243 | + |
| 244 | +- [GitHub repository](https://github.com/mhmtdev/StorageFlow) |
| 245 | +- [Minimal API sample](https://github.com/mhmtdev/StorageFlow/tree/main/samples/StorageFlow.Sample.Api) |
| 246 | +- [Contributing](https://github.com/mhmtdev/StorageFlow/blob/main/CONTRIBUTING.md) |
| 247 | +- [Security policy](https://github.com/mhmtdev/StorageFlow/blob/main/SECURITY.md) |
| 248 | + |
| 249 | +StorageFlow targets .NET 9 and later and is licensed under MIT. |
0 commit comments