feat!: support key deletion via IDeletableXmlRepository

AGENTS.md

@@ -38,15 +38,18 @@ dotnet test --no-build -c Release --filter-method "*.StorageWrapperTests.StoreEl
 ```
 ASP.NET Core Data Protection (IXmlRepository)
     └── StorageWrapper<TStorage>         [Core: bridges IXmlRepository <-> IDataProtectionStorage]
+        (DeletableStorageWrapper<TStorage> also bridges IDeletableXmlRepository <-> IDeletableDataProtectionStorage)
             └── IDataProtectionStorage   [Core: GetAll() / Insert()]
+                └── IDeletableDataProtectionStorage  [Core: adds Delete()]
                     └── IDbDataProtectionStorage  [Dapper: adds InitializeDb() / GetAllAsync()]
                             ├── SQLServerDataProtectionRepository   [Dapper.SQLServer]
                             └── PostgreSQLDataProtectionRepository  [Dapper.PostgreSQL]
 ```
 
 ### Wiring
 
-- **Core:** `PersistKeysToStorage<TStorage>()` registers a singleton `IConfigureOptions<KeyManagementOptions>` that sets `XmlRepository` to a `StorageWrapper<TStorage>`. It does **not** register `TStorage`; the caller must.
+- **Core:** `PersistKeysToStorage<TStorage>()` registers a singleton `IConfigureOptions<KeyManagementOptions>` that sets `XmlRepository` to a `StorageWrapper<TStorage>`, or to a `DeletableStorageWrapper<TStorage>` when `TStorage` is assignable to `IDeletableDataProtectionStorage` (so `XmlKeyManager.CanDeleteKeys` reflects the storage). It does **not** register `TStorage`; the caller must.
+- `DeletableStorageWrapper.DeleteElements` offers the parsed keys (malformed rows skipped), then passes the chosen `GetAll()` instances, ordered by `DeletionOrder`, to a single `Delete()` call. A storage exception is logged (`ErrorDeletingKeysFromStorage`) and returns `false`. Note: in ASP.NET Core 10.0, `XmlKeyManager` exposes `DeleteKeys` but does not implement `IDeletableKeyManager`.
 - `StorageWrapper` (internal) creates a new DI scope and resolves `TStorage` on every `GetAllElements` / `StoreElement` call, so scoped storage implementations work. It trims `FriendlyName`, normalizes empty or whitespace-only names to `null`, and wraps any `Insert` exception in `KeyInsertException`. `GetAllElements` logs and skips rows with malformed XML (`SkippingMalformedKey`) instead of failing the whole key ring.
 - Never log or expose key XML (it may contain unencrypted key material); this includes `XmlException` messages, which can quote the input.
 - **DB providers:** `PersistKeysWithDapperInSqlServer(...)` / `PersistKeysWithDapperInPostgreSQL(...)` configure `IOptions<DapperDataProtectionConfig>`, register the connection source as a singleton (`SqlConnectionFactory` / `NpgsqlDataSource`), register the repository as scoped `IDbDataProtectionStorage`, then call `PersistKeysToStorage<IDbDataProtectionStorage>()`.
@@ -63,16 +66,17 @@ Both repositories use raw SQL via Dapper. They differ in naming conventions:
 | Unique index on friendly name | Filtered NONCLUSTERED (`WHERE ... IS NOT NULL`) | `CREATE UNIQUE INDEX IF NOT EXISTS` (NULLs allowed) |
 
 - The PostgreSQL provider sets the global `Dapper.DefaultTypeMap.MatchNamesWithUnderscores = true` so snake_case columns map to `DataProtectionKeyEntity`. This affects the whole host process (documented in the README).
+- `Delete()` requires `DataProtectionKeyEntity` instances (as returned by `GetAll()`) and deletes by `Id`, row by row in order, in one transaction.
 - Schema and table names are interpolated directly into SQL strings, not parameterized or validated.
 - Defaults are in each provider's `Constants.cs`. Keep the table DDL in sync with the "Database schema" section of `README.md`.
 
 ### Testing
 
 Single test project: `tests/AspNetCore.DataProtection.CustomStorage.Tests` (xUnit v3, FluentAssertions, NSubstitute, Bogus).
 
-- `StorageWrapperTests` and `DapperDataProtectionExtensionsTests` are pure unit tests.
+- `StorageWrapperTests`, `DeletableStorageWrapperTests` and `DapperDataProtectionExtensionsTests` are pure unit tests. `DataProtectionStorageExtensionsTests` runs the real DI container and `XmlKeyManager` against in-memory storages.
 - `SQLServer/` and `PostgreSQL/` tests use an `IClassFixture` (`*ContainerFixture`) that starts a Testcontainers database, calls `InitializeDb()` with the default config, and exposes `ResetDatabaseAsync()` (Respawn) to clear data between tests.
-- `*EndToEndTests` wire each provider through the public extension methods and a real DI container (`AddDataProtection().PersistKeysWithDapperIn...()`, `UseDapperDataProtection()`), then protect/unprotect, including across two service providers sharing one table. Each test uses its own table, so it does not depend on Respawn.
+- `*EndToEndTests` wire each provider through the public extension methods and a real DI container (`AddDataProtection().PersistKeysWithDapperIn...()`, `UseDapperDataProtection()`), then protect/unprotect, including across two service providers sharing one table, and delete keys through `XmlKeyManager.DeleteKeys`. Each test uses its own table, so it does not depend on Respawn.
 
 ### CI/CD (`.github/workflows`)
 

README.md

@@ -33,6 +33,7 @@ The base package lets you plug any backend into the Data Protection key ring by
 - [Configuration](#configuration)
 - [Database schema](#database-schema)
 - [Custom storage backend](#custom-storage-backend)
+- [Deleting keys](#deleting-keys)
 - [How it works](#how-it-works)
 - [Contributing](#contributing)
 - [License](#license)
@@ -224,20 +225,71 @@ builder.Services.AddDataProtection()
 on each read/write, so you must register it yourself (any lifetime works). If an `Insert` throws,
 it is wrapped in a `KeyInsertException`.
 
+To support [key deletion](#deleting-keys), implement `IDeletableDataProtectionStorage` instead:
+
+```csharp
+public class MyDataProtectionStorage : IDeletableDataProtectionStorage
+{
+    // GetAll() and Insert() as above
+
+    public void Delete(IReadOnlyCollection<DataProtectionKey> keys)
+    {
+        // delete every key, in the given order, or none of them (throw on failure)
+    }
+}
+```
+
+The keys passed to `Delete` are instances previously returned by `GetAll()` on the same storage
+instance, so a backend can return a derived record that carries its own identifier (as the Dapper
+providers do with `DataProtectionKeyEntity.Id`) and use it to locate the rows to delete.
+
+## Deleting keys
+
+Deleting a key makes the data it protected permanently unreadable; Microsoft
+[recommends not deleting keys](https://learn.microsoft.com/aspnet/core/security/data-protection/configuration/default-settings#delete-keys)
+except in exceptional cases. When the storage supports it (both database providers do, as does any
+`IDeletableDataProtectionStorage`), old keys can be deleted through the key manager:
+
+```csharp
+using Microsoft.AspNetCore.DataProtection.KeyManagement;
+
+var keyManager = app.Services.GetRequiredService<IKeyManager>();
+if (keyManager is XmlKeyManager { CanDeleteKeys: true } xmlKeyManager)
+{
+    var yearAgo = DateTimeOffset.UtcNow.AddYears(-1);
+    if (!xmlKeyManager.DeleteKeys(key => key.ExpirationDate < yearAgo))
+    {
+        // the deletion failed and no key was deleted; see the ErrorDeletingKeysFromStorage log entry
+    }
+}
+```
+
+> **Note:** in ASP.NET Core 10.0, `XmlKeyManager` exposes `CanDeleteKeys` and `DeleteKeys` but does
+> not implement `IDeletableKeyManager`, so a `keyManager is IDeletableKeyManager` check is `false`
+> even when deletion is supported.
+
+The selected keys are deleted with a single `Delete` call. The database providers delete the rows
+by `Id`, in deletion order, in one transaction: either every selected key is deleted or none is.
+If the storage throws, the error is logged (event `ErrorDeletingKeysFromStorage`) and `DeleteKeys`
+returns `false`. Each deleted key is logged by friendly name (event `DeletingKeyFromStorage`); the
+key XML is never logged. A stored key whose XML is malformed is not offered for deletion.
+
 ## How it works
 
 `PersistKeysToStorage<TStorage>()` registers an internal `IXmlRepository` adapter as the Data
 Protection key repository. The adapter maps the framework's XML-element calls to the simpler
 `IDataProtectionStorage` contract (`GetAll()` / `Insert()`), resolving `TStorage` from a scoped
-service provider for every operation.
+service provider for every operation. When `TStorage` implements `IDeletableDataProtectionStorage`,
+the adapter also implements the framework's `IDeletableXmlRepository` and maps deletions to `Delete()`.
 
 When the key ring is loaded, a stored key whose XML is malformed is skipped and logged as an error
 (event `SkippingMalformedKey`, with the friendly name and the parse position; the XML itself is never
 logged). The remaining keys stay usable, but data protected with the skipped key can no longer be
 unprotected, and if no valid key remains, Data Protection generates a new one.
 
-The database providers add a Dapper layer (`IDbDataProtectionStorage`) on top of that contract,
-implementing the raw SQL for SQL Server and PostgreSQL, plus schema initialization
+The database providers add a Dapper layer (`IDbDataProtectionStorage`, which extends
+`IDeletableDataProtectionStorage`) on top of that contract,
+implementing the raw SQL for SQL Server and PostgreSQL, plus key deletion, schema initialization
 (`InitializeDb()`) and an async read path (`GetAllAsync()`) that returns the stored
 `DataProtectionKeyEntity` rows, including `Id` and `InsertDate`, for tooling such as key auditing.
 

src/AspNetCore.DataProtection.CustomStorage.Dapper.PostgreSQL/PostgreSQLDataProtectionRepository.cs

@@ -1,5 +1,6 @@
 using System;
 using System.Collections.Generic;
+using System.Linq;
 using System.Threading.Tasks;
 using Dapper;
 using Microsoft.Extensions.Options;
@@ -65,6 +66,43 @@ public void Insert(DataProtectionKey key)
             key);
     }
 
+    /// <inheritdoc />
+    /// <remarks>
+    /// The rows are deleted by <see cref="DataProtectionKeyEntity.Id"/>, in the given order, in a single transaction.
+    /// A row that no longer exists is not an error.
+    /// </remarks>
+    /// <exception cref="ArgumentException">a key is not a <see cref="DataProtectionKeyEntity"/> returned by <see cref="GetAll"/></exception>
+    public void Delete(IReadOnlyCollection<DataProtectionKey> keys)
+    {
+        var ids = GetIds(keys);
+        if (ids.Count == 0)
+        {
+            return;
+        }
+
+        using var connection = _dataSource.OpenConnection();
+        using var transaction = connection.BeginTransaction();
+        // Dapper executes the statement once per id, in order.
+        connection.Execute(
+            $"""
+             DELETE FROM {_config.SchemaName}.{_config.TableName}
+             WHERE id = @{nameof(DataProtectionKeyEntity.Id)}
+             """,
+            ids.Select(id => new { Id = id }),
+            transaction);
+        transaction.Commit();
+    }
+
+    private static List<int> GetIds(IReadOnlyCollection<DataProtectionKey> keys)
+    {
+        ArgumentNullException.ThrowIfNull(keys);
+        return keys
+            .Select(key => key as DataProtectionKeyEntity ?? throw new ArgumentException(
+                $"Every key must be a {nameof(DataProtectionKeyEntity)} returned by {nameof(GetAll)}.", nameof(keys)))
+            .Select(entity => entity.Id)
+            .ToList();
+    }
+
     /// <inheritdoc />
     public void InitializeDb()
     {

src/AspNetCore.DataProtection.CustomStorage.Dapper.SQLServer/SQLServerDataProtectionRepository.cs

@@ -1,5 +1,6 @@
 using System;
 using System.Collections.Generic;
+using System.Linq;
 using System.Threading.Tasks;
 using Dapper;
 using Microsoft.Extensions.Options;
@@ -64,6 +65,44 @@ public void Insert(DataProtectionKey key)
              """, key);
     }
 
+    /// <inheritdoc />
+    /// <remarks>
+    /// The rows are deleted by <see cref="DataProtectionKeyEntity.Id"/>, in the given order, in a single transaction.
+    /// A row that no longer exists is not an error.
+    /// </remarks>
+    /// <exception cref="ArgumentException">a key is not a <see cref="DataProtectionKeyEntity"/> returned by <see cref="GetAll"/></exception>
+    public void Delete(IReadOnlyCollection<DataProtectionKey> keys)
+    {
+        var ids = GetIds(keys);
+        if (ids.Count == 0)
+        {
+            return;
+        }
+
+        using var connection = _connectionFactory.CreateConnection();
+        connection.Open();
+        using var transaction = connection.BeginTransaction();
+        // Dapper executes the statement once per id, in order.
+        connection.Execute(
+            $"""
+             DELETE FROM [{_config.SchemaName}].[{_config.TableName}]
+             WHERE [{nameof(DataProtectionKeyEntity.Id)}] = @{nameof(DataProtectionKeyEntity.Id)}
+             """,
+            ids.Select(id => new { Id = id }),
+            transaction);
+        transaction.Commit();
+    }
+
+    private static List<int> GetIds(IReadOnlyCollection<DataProtectionKey> keys)
+    {
+        ArgumentNullException.ThrowIfNull(keys);
+        return keys
+            .Select(key => key as DataProtectionKeyEntity ?? throw new ArgumentException(
+                $"Every key must be a {nameof(DataProtectionKeyEntity)} returned by {nameof(GetAll)}.", nameof(keys)))
+            .Select(entity => entity.Id)
+            .ToList();
+    }
+
     /// <inheritdoc />
     public void InitializeDb()
     {

src/AspNetCore.DataProtection.CustomStorage.Dapper/IDBDataProtectionStorage.cs

@@ -4,7 +4,7 @@
 namespace AspNetCore.DataProtection.CustomStorage.Dapper;
 
 /// <inheritdoc />
-public interface IDbDataProtectionStorage:IDataProtectionStorage
+public interface IDbDataProtectionStorage:IDeletableDataProtectionStorage
 {
     /// <summary>
     /// create the storage/table if needed

src/AspNetCore.DataProtection.CustomStorage/DataProtectionStorageExtensions.cs

@@ -13,7 +13,9 @@ namespace AspNetCore.DataProtection.CustomStorage;
 public static class DataProtectionStorageExtensions
 {
     /// <summary>
-    /// Configures the data protection system to persist keys to a custom storage
+    /// Configures the data protection system to persist keys to a custom storage.
+    /// When <typeparamref name="TStorage"/> implements <see cref="IDeletableDataProtectionStorage"/>,
+    /// keys can also be deleted through <see cref="XmlKeyManager.DeleteKeys"/>.
     /// </summary>
     /// <param name="builder">The <see cref="IDataProtectionBuilder"/> instance to modify.</param>
     /// <returns>The value <paramref name="builder"/>.</returns>
@@ -25,7 +27,9 @@ public static IDataProtectionBuilder PersistKeysToStorage<TStorage>(this IDataPr
             var loggerFactory = services.GetService<ILoggerFactory>() ?? NullLoggerFactory.Instance;
             return new ConfigureOptions<KeyManagementOptions>(options =>
             {
-                options.XmlRepository = new StorageWrapper<TStorage>(services, loggerFactory);
+                options.XmlRepository = typeof(TStorage).IsAssignableTo(typeof(IDeletableDataProtectionStorage))
+                    ? new DeletableStorageWrapper<TStorage>(services, loggerFactory)
+                    : new StorageWrapper<TStorage>(services, loggerFactory);
             });
         });
 

src/AspNetCore.DataProtection.CustomStorage/DeletableStorageWrapper.cs

@@ -0,0 +1,96 @@
+using System;
+using System.Collections.Generic;
+using System.Linq;
+using System.Xml.Linq;
+using Microsoft.AspNetCore.DataProtection.Repositories;
+using Microsoft.Extensions.DependencyInjection;
+using Microsoft.Extensions.Logging;
+
+namespace AspNetCore.DataProtection.CustomStorage;
+
+/// <summary>
+/// handle retrieving, storing and deleting the keys for the DataProtection with a storage
+/// that implements <see cref="IDeletableDataProtectionStorage"/>
+/// </summary>
+/// <typeparam name="TStorage">the registered storage type; it must implement <see cref="IDeletableDataProtectionStorage"/></typeparam>
+internal class DeletableStorageWrapper<TStorage> : StorageWrapper<TStorage>, IDeletableXmlRepository
+    where TStorage : IDataProtectionStorage
+{
+    /// <summary>
+    ///
+    /// </summary>
+    /// <param name="services"></param>
+    /// <param name="loggerFactory"></param>
+    /// <exception cref="ArgumentNullException"></exception>
+    /// <exception cref="ArgumentException"><typeparamref name="TStorage"/> does not implement <see cref="IDeletableDataProtectionStorage"/></exception>
+    public DeletableStorageWrapper(IServiceProvider services, ILoggerFactory loggerFactory)
+        : base(services, loggerFactory)
+    {
+        if (!typeof(TStorage).IsAssignableTo(typeof(IDeletableDataProtectionStorage)))
+        {
+            throw new ArgumentException(
+                $"'{typeof(TStorage).Name}' does not implement '{nameof(IDeletableDataProtectionStorage)}'.",
+                nameof(TStorage));
+        }
+    }
+
+    /// <inheritdoc />
+    /// <remarks>
+    /// A stored key whose XML is malformed is logged and left out of the snapshot, so it is never deleted.
+    /// The chosen keys are deleted with a single <see cref="IDeletableDataProtectionStorage.Delete"/> call;
+    /// if it fails, the error is logged and <c>false</c> is returned.
+    /// </remarks>
+    public bool DeleteElements(Action<IReadOnlyCollection<IDeletableElement>> chooseElements)
+    {
+        ArgumentNullException.ThrowIfNull(chooseElements);
+        using var scope = _services.CreateScope();
+        var storage = (IDeletableDataProtectionStorage)scope.ServiceProvider.GetRequiredService<TStorage>();
+
+        var deletableElements = new List<DeletableElement>();
+        foreach (var key in storage.GetAll())
+        {
+            var element = TryParseKey(key);
+            if (element is not null)
+            {
+                deletableElements.Add(new DeletableElement(key, element));
+            }
+        }
+
+        chooseElements(deletableElements);
+
+        var keysToDelete = deletableElements
+            .Where(e => e.DeletionOrder.HasValue)
+            .OrderBy(e => e.DeletionOrder.GetValueOrDefault())
+            .Select(e => e.Key)
+            .ToList();
+        if (keysToDelete.Count == 0)
+        {
+            return true;
+        }
+
+        foreach (var key in keysToDelete)
+        {
+            _logger.LogDeletingKeyFromStorage(key.FriendlyName, typeof(TStorage).Name);
+        }
+
+        try
+        {
+            storage.Delete(keysToDelete);
+            return true;
+        }
+        catch (Exception e)
+        {
+            _logger.LogErrorDeletingKeysFromStorage(e, keysToDelete.Count, typeof(TStorage).Name);
+            return false;
+        }
+    }
+
+    private sealed class DeletableElement(DataProtectionKey key, XElement element) : IDeletableElement
+    {
+        public DataProtectionKey Key { get; } = key;
+
+        public XElement Element { get; } = element;
+
+        public int? DeletionOrder { get; set; }
+    }
+}

src/AspNetCore.DataProtection.CustomStorage/IDeletableDataProtectionStorage.cs

@@ -0,0 +1,20 @@
+using System.Collections.Generic;
+
+namespace AspNetCore.DataProtection.CustomStorage;
+
+/// <summary>
+/// Storage/repository interface that also supports the deletion of stored keys.
+/// When the storage passed to <see cref="DataProtectionStorageExtensions.PersistKeysToStorage{TStorage}"/>
+/// implements this interface, keys can be deleted through
+/// <see cref="Microsoft.AspNetCore.DataProtection.KeyManagement.XmlKeyManager.DeleteKeys"/>.
+/// </summary>
+public interface IDeletableDataProtectionStorage : IDataProtectionStorage
+{
+    /// <summary>
+    /// delete the given keys, in the given order.
+    /// The keys are instances previously returned by <see cref="IDataProtectionStorage.GetAll"/> on the same instance.
+    /// Either every key is deleted or none is: on failure the implementation must throw and leave the storage unchanged.
+    /// </summary>
+    /// <param name="keys">the keys to delete, in deletion order</param>
+    void Delete(IReadOnlyCollection<DataProtectionKey> keys);
+}