Skip to content

FileTreeStore.delete_entry: non-atomic check-then-act races TTL/concurrent delete → unhandled FileNotFoundError #362

Description

@pvliesdonk

Summary

FileTreeStore (async, key_value.aio.stores.filetree) deletes entries with a check-then-act that is not atomic, so a concurrent delete or a TTL-expiry purge of the same key between the exists() check and the unlink() raises an unhandled FileNotFoundError. A KV delete of a key that is (or becomes) absent should be a no-op, never raise.

Version: py-key-value-aio 0.4.4

The race

DiskCollectionInfo.delete_entry (key_value/aio/stores/filetree/store.py, ~lines 286–298):

async def delete_entry(self, *, key: str) -> bool:
    sanitized_key = self.key_sanitization_strategy.sanitize(value=key)
    key_path: AsyncPath = AsyncPath(self.directory / f"{sanitized_key}.json")
    await self._validate_path_security(path=key_path)

    if not await key_path.exists():   # line ~293  (check)
        return False
    await key_path.unlink()           # line ~296  (act — dispatched to a worker thread)
    return True

exists() and unlink() are two separate awaits, and AsyncPath.unlink() is itself dispatched to a worker thread via anyio. Between the check passing and the unlink running, the file can be removed by:

  • another coroutine/task deleting the same key, or
  • a TTL-expiry purge of the same entry.

The second unlink() then raises FileNotFoundError: [Errno 2] No such file or directory: '.../<key>.json' instead of treating the already-absent key as a successful (idempotent) delete.

How it surfaces in the wild

This bit a production service through FastMCP's resumability EventStore, which is backed by FileTreeStore with a TTL. EventStore.store_event trims an event ring by calling delete(old_id); with long-lived sessions (>100 events, >1h TTL) and concurrent in-flight requests, TTL expiry races the trim-delete. The FileNotFoundError propagates out of the MCP SSE writer / message router and tears down the response stream, so clients see intermittent request timeouts. Full traceback bottoms out exactly at delete_entry -> key_path.unlink().

Expected behaviour

delete of an absent key is a no-op. The store should never raise FileNotFoundError from a delete.

Proposed fix

Drop the non-atomic exists() pre-check and let unlink swallow the missing-file case atomically:

async def delete_entry(self, *, key: str) -> bool:
    sanitized_key = self.key_sanitization_strategy.sanitize(value=key)
    key_path: AsyncPath = AsyncPath(self.directory / f"{sanitized_key}.json")
    await self._validate_path_security(path=key_path)
    try:
        await key_path.unlink()
    except FileNotFoundError:
        return False
    return True

(or await key_path.unlink(missing_ok=True) if the AsyncPath backend supports it — though catching FileNotFoundError is the more portable form and preserves the bool "did it exist" return contract).

The same check-then-act shape appears at ~line 270 (a get/read path) and is worth auditing for an analogous race, though delete is the one observed raising.

Environment

  • py-key-value-aio 0.4.4
  • Python 3.12
  • FileTreeStore on a local filesystem, accessed concurrently with TTL set

— 🤖 Automated post by Claude Code (agent) via the account owner's GitHub token; agent analysis/proposal, not a personal directive from the account owner.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions