Land cache memo, lock and tags in feature/caches and consolidate the cache ADR

docs/adr/AGENTS.md

@@ -32,6 +32,5 @@ Read this index first, then open the records relevant to the change. Before impl
 | [024](024-installer-controller-module.md) | 2026-10-19 | Installer controllers in their own module | `fastapi-startkit init` generates `HomeController` in `app/http/controllers/home_controller.py` (JSON or Inertia variant) instead of inside `routes/api.py`; one shared routes stub imports and registers it, matching the PingCRM layout. |
 | [025](025-model-lifecycle-events.md) | 2026-10-20 | Model lifecycle events, automatic timestamps and datetime casts | `save()` fires `saving → creating/updating → query → created/updated → saved`. Observer changes are persisted, and `updating` fires only when the model is dirty. A new `Model.delete()` fires `deleting → DELETE → deleted`. `observe_events` is async and awaits async handlers. Mass query-builder writes fire no events. Laravel-style auto timestamps: `created_at`/`updated_at` are filled on create unless supplied, `updated_at` moves on update, and `__timestamps__ = False` disables both. `DateCast.set` returns a UTC datetime; `Connection.prepare_binding` passes it unchanged to Postgres/MySQL, and SQLite converts it to a naive UTC string. Postgres registers an asyncpg `timestamp` codec that stores aware values as naive UTC; `timestamptz` keeps asyncpg's codec. `Annotated[T, Cast]` (a `BaseCast` subclass or instance) overrides a column's cast, and `Attribute.__casts__` is removed. |
 | [026](026-schema-column-change.md) | 2026-10-20 | Altering existing columns with `.change()` | `.change()` fully redefines the column, the way Laravel does. MySQL (`MODIFY`) and SQLite (table rebuild) were already correct. Postgres now emits `TYPE ... USING` casts, correct lengths, quoted defaults or `DROP DEFAULT`, `SET`/`DROP NOT NULL`, and replaces the enum check constraint. MSSQL drops the current default constraint (looked up in `sys.default_constraints`), runs one `ALTER COLUMN` per column, then adds `[DF_<table>_<col>]`. Identifiers are bracket- or double-quote-escaped and the MSSQL drop target is built server-side with `QUOTENAME`; changing a Postgres serial column drops its `nextval` default. |
-| [029](029-cache-module.md) | 2026-10-21 | Cache module with callable remember and pluggable stores | New `fastapi_startkit.cache`: async `Cache.get/put/has/forget/flush`, `get` with a callable default, and `remember(key, ttl, factory)` that runs a sync or async factory only on a miss; `remember(key, ttl)` without a factory is a decorator keyed by `key` plus a hash of the call arguments. Stores implement an async contract with a `MISSING` sentinel (cached `None` is a hit): array, file (pickled per-key files), database (masoniteorm table) and redis (named connection from the existing `RedisManager`, `SET PX` TTL, flush by `SCAN` of prefixed keys, never `FLUSHDB`). TTL in seconds or `timedelta`, `None` forever, <=0 forgets. `CacheConfig` (`CACHE_CONNECTION`, `CACHE_PREFIX`, `stores`) via `resolve_config`; `CacheProvider` binds `cache`; facade `remember` resolves lazily. A `with` form is impossible in Python. Shipped in slices into `feature/caches`. |
-| [030](030-cache-memo.md) | 2026-10-21 | Request-scoped cache memoization with Cache.memo() | `Cache.memo(name=None)` returns a `Repository` over `MemoStore`, which wraps the named store and memoizes reads (misses too) for the current asyncio task; `put`/`forget` write through and drop the key, `flush` clears memo and store. State is a `ContextVar` holding `(weakref to owning task, memo)`, and a task that finds another task's memo starts a fresh one, so requests, jobs and child tasks never share memo state. Memoized values ignore TTL for the rest of the task. |
+| [029](029-cache-module.md) | 2026-10-21 | Cache module with remember, memo, locks and tags | New `fastapi_startkit.cache`: async `Cache.get/put/has/forget/flush`, `get` with a callable default, and `remember(key, ttl, factory)` that runs a sync or async factory only on a miss; `remember(key, ttl)` without a factory is a decorator keyed by `key` plus a hash of the call arguments. Stores implement an async contract with a `MISSING` sentinel (cached `None` is a hit): array, file (pickled per-key files), database (masoniteorm table) and redis (named connection from the existing `RedisManager`, `SET PX` TTL, flush by `SCAN` of prefixed keys, never `FLUSHDB`). TTL in seconds or `timedelta`, `None` forever, <=0 forgets. `CacheConfig` via `resolve_config`; `CacheProvider` binds `cache`; facade `remember` resolves lazily. `Cache.memo()` memoizes reads per asyncio task through a `ContextVar` owned by the task, so requests never share memo state. `Cache.lock(name, seconds, owner)` gives atomic, owner-checked, non-re-entrant locks with `get`, `block` (250 ms polling, `LockTimeoutError`), `release`, `force_release` and `async with`, per store: event-loop dict, `fcntl.flock` file, `lock:<name>` row with update-if-expired, Redis `SET NX PX` plus a Lua owner-checked delete. `Cache.tags([...])` uses Laravel's namespaced keys: random tag ids at `<prefix>:tag:<name>:key`, entries at `<prefix>:sha1(ids):<key>`, flush rewrites the ids; tags are sorted and deduplicated. Shipped in slices into `feature/caches`; the former memo, lock and tags ADRs (030, 031, 034) are folded in. |
 | [032](032-migrate-seed-option.md) | 2026-10-21 | db:migrate:fresh and db:migrate:refresh --seed run seeders in the same event loop | `--seed` called the unregistered `seed:run` through Cleo `call` (which also drops the first argument) inside a running event loop, where `db:seed` would call `asyncio.run` again. The seeder selection moves into async `DBSeedCommand.run_seeders`, which `db:seed`, `db:migrate:fresh` and `db:migrate:refresh` all await. `db:migrate:refresh` is now registered. |

fastapi_startkit/src/fastapi_startkit/cache/__init__.py

@@ -1,11 +1,13 @@
 from .array_store import ArrayStore
 from .config import ArrayStoreConfig, CacheConfig, DatabaseStoreConfig, FileStoreConfig, RedisStoreConfig
 from .file_store import FileStore
+from .lock import Lock, LockTimeoutError
 from .manager import CacheManager
 from .memo_store import MemoStore
 from .provider import CacheProvider
 from .repository import Repository
 from .store import MISSING, Store
+from .tagged_store import TaggedStore, TagSet
 
 __all__ = [
     "MISSING",
@@ -17,8 +19,12 @@
     "DatabaseStoreConfig",
     "FileStore",
     "FileStoreConfig",
+    "Lock",
+    "LockTimeoutError",
     "MemoStore",
     "RedisStoreConfig",
     "Repository",
     "Store",
+    "TagSet",
+    "TaggedStore",
 ]

fastapi_startkit/src/fastapi_startkit/cache/array_store.py

@@ -1,11 +1,13 @@
 from typing import Any
 
+from .lock import Lock
 from .store import MISSING, expired, expires_at
 
 
 class ArrayStore:
     def __init__(self) -> None:
         self.storage: dict[str, tuple[float | None, Any]] = {}
+        self.locks: dict[str, tuple[str, float | None]] = {}
 
     async def get(self, key: str) -> Any:
         entry = self.storage.get(key)
@@ -28,3 +30,33 @@ async def forget(self, key: str) -> bool:
     async def flush(self) -> bool:
         self.storage.clear()
         return True
+
+    def lock(self, name: str, seconds: float = 0, owner: str | None = None) -> "ArrayLock":
+        return ArrayLock(self, name, seconds, owner)
+
+
+class ArrayLock(Lock):
+    def __init__(self, store: ArrayStore, name: str, seconds: float = 0, owner: str | None = None):
+        super().__init__(name, seconds, owner)
+        self.store = store
+
+    def holder(self) -> str | None:
+        entry = self.store.locks.get(self.name)
+        if entry is None or expired(entry[1]):
+            return None
+        return entry[0]
+
+    async def acquire(self) -> bool:
+        if self.holder() is not None:
+            return False
+        self.store.locks[self.name] = (self.owner, expires_at(self.ttl))
+        return True
+
+    async def release(self) -> bool:
+        if self.holder() != self.owner:
+            return False
+        del self.store.locks[self.name]
+        return True
+
+    async def force_release(self) -> None:
+        self.store.locks.pop(self.name, None)

fastapi_startkit/src/fastapi_startkit/cache/database_store.py

@@ -1,6 +1,7 @@
 import base64
 import math
 import pickle
+import time
 from collections.abc import Callable
 from typing import Any
 
@@ -9,9 +10,14 @@
 from fastapi_startkit.masoniteorm.connections.connection import Connection
 from fastapi_startkit.masoniteorm.models.builder import QueryBuilder
 
+from .lock import Lock
 from .store import MISSING, expired, expires_at
 
 
+def encode(value: Any) -> str:
+    return base64.b64encode(pickle.dumps(value)).decode()
+
+
 class DatabaseStore:
     def __init__(self, connection: Callable[[], Connection], table: str = "caches"):
         self.connection = connection
@@ -39,10 +45,7 @@ async def get(self, key: str) -> Any:
 
     async def put(self, key: str, value: Any, seconds: float | None) -> bool:
         expiration = expires_at(seconds)
-        record = {
-            "value": base64.b64encode(pickle.dumps(value)).decode(),
-            "expiration": None if expiration is None else math.ceil(expiration),
-        }
+        record = {"value": encode(value), "expiration": None if expiration is None else math.ceil(expiration)}
         if await self.query().where("key", key).update(record):
             return True
         try:
@@ -57,3 +60,35 @@ async def forget(self, key: str) -> bool:
     async def flush(self) -> bool:
         await self.query().delete()
         return True
+
+    def lock(self, name: str, seconds: float = 0, owner: str | None = None) -> "DatabaseLock":
+        return DatabaseLock(self, name, seconds, owner)
+
+
+class DatabaseLock(Lock):
+    def __init__(self, store: DatabaseStore, name: str, seconds: float = 0, owner: str | None = None):
+        super().__init__(name, seconds, owner)
+        self.store = store
+        self.key = f"lock:{name}"
+
+    async def acquire(self) -> bool:
+        expiration = expires_at(self.ttl)
+        record = {"value": encode(self.owner), "expiration": None if expiration is None else math.ceil(expiration)}
+        try:
+            await self.store.query().insert({"key": self.key, **record})
+        except IntegrityError:
+            expired_lock = self.store.query().where("key", self.key).where("expiration", "<=", time.time())
+            return bool(await expired_lock.update(record))
+        return True
+
+    async def release(self) -> bool:
+        held = (
+            self.store.query()
+            .where("key", self.key)
+            .where("value", encode(self.owner))
+            .where(lambda query: query.where_null("expiration").or_where("expiration", ">", time.time()))
+        )
+        return bool(await held.delete())
+
+    async def force_release(self) -> None:
+        await self.store.query().where("key", self.key).delete()

fastapi_startkit/src/fastapi_startkit/cache/file_store.py

@@ -1,16 +1,21 @@
 import asyncio
 import hashlib
+import json
 import os
 import pickle
 import re
 import shutil
 import tempfile
+from collections.abc import Callable
 from pathlib import Path
 from typing import Any
 
-from .store import MISSING, expired, expires_at
+from .lock import Lock
+from .store import MISSING, Missing, expired, expires_at
 
 SHARD = re.compile(r"^[0-9a-f]{2}$")
+Holder = tuple[str, float | None] | None
+Decision = tuple[bool, Holder | Missing]
 
 
 class FileStore:
@@ -76,3 +81,55 @@ def clear(self) -> None:
         for shard in self.directory.iterdir():
             if shard.is_dir() and SHARD.match(shard.name):
                 shutil.rmtree(shard, ignore_errors=True)
+
+    def lock(self, name: str, seconds: float = 0, owner: str | None = None) -> "FileLock":
+        return FileLock(self, name, seconds, owner)
+
+
+class FileLock(Lock):
+    def __init__(self, store: FileStore, name: str, seconds: float = 0, owner: str | None = None):
+        super().__init__(name, seconds, owner)
+        self.path = store.directory / "locks" / hashlib.sha1(name.encode()).hexdigest()
+
+    async def acquire(self) -> bool:
+        def decide(holder: Holder) -> Decision:
+            if holder is not None:
+                return False, MISSING
+            return True, (self.owner, expires_at(self.ttl))
+
+        return await asyncio.to_thread(self.exclusive, decide)
+
+    async def release(self) -> bool:
+        def decide(holder: Holder) -> Decision:
+            if holder is None or holder[0] != self.owner:
+                return False, MISSING
+            return True, None
+
+        return await asyncio.to_thread(self.exclusive, decide)
+
+    async def force_release(self) -> None:
+        await asyncio.to_thread(self.exclusive, lambda holder: (True, None))
+
+    def exclusive(self, decide: Callable[[Holder], Decision]) -> bool:
+        import fcntl
+
+        self.path.parent.mkdir(parents=True, exist_ok=True)
+        with open(self.path, "a+b") as file:
+            fcntl.flock(file, fcntl.LOCK_EX)
+            file.seek(0)
+            result, holder = decide(self.read(file.read()))
+            if holder is not MISSING:
+                file.truncate(0)
+                if holder is not None:
+                    file.write(json.dumps(holder).encode())
+                file.flush()
+            return result
+
+    def read(self, payload: bytes) -> Holder:
+        try:
+            owner, expiration = json.loads(payload)
+        except (ValueError, TypeError):
+            return None
+        if not isinstance(owner, str) or not isinstance(expiration, int | float | None) or expired(expiration):
+            return None
+        return owner, expiration

fastapi_startkit/src/fastapi_startkit/cache/lock.py

@@ -0,0 +1,65 @@
+import asyncio
+import secrets
+import time
+from abc import ABC, abstractmethod
+from collections.abc import Callable
+from typing import Any
+
+from .repository import evaluate
+
+
+class LockTimeoutError(Exception):
+    pass
+
+
+class Lock(ABC):
+    sleep: float = 0.25
+
+    def __init__(self, name: str, seconds: float = 0, owner: str | None = None):
+        self.name = name
+        self.seconds = seconds
+        self.owner = owner or secrets.token_hex(16)
+
+    @property
+    def ttl(self) -> float | None:
+        return self.seconds if self.seconds > 0 else None
+
+    @abstractmethod
+    async def acquire(self) -> bool: ...
+
+    @abstractmethod
+    async def release(self) -> bool: ...
+
+    @abstractmethod
+    async def force_release(self) -> None: ...
+
+    async def get(self, callback: Callable[[], Any] | None = None) -> Any:
+        if not await self.acquire():
+            return False
+        if callback is None:
+            return True
+        return await self.run(callback)
+
+    async def block(self, seconds: float, callback: Callable[[], Any] | None = None) -> Any:
+        deadline = time.monotonic() + seconds
+        while not await self.acquire():
+            if time.monotonic() >= deadline:
+                raise LockTimeoutError(f"Could not acquire lock [{self.name}] within {seconds} seconds.")
+            await asyncio.sleep(self.sleep)
+        if callback is None:
+            return True
+        return await self.run(callback)
+
+    async def run(self, callback: Callable[[], Any]) -> Any:
+        try:
+            return await evaluate(callback)
+        finally:
+            await self.release()
+
+    async def __aenter__(self) -> "Lock":
+        if not await self.acquire():
+            raise LockTimeoutError(f"Lock [{self.name}] is held by another owner.")
+        return self
+
+    async def __aexit__(self, *exc: object) -> None:
+        await self.release()