Skip to content

Commit bcbdb75

Browse files
feat: multi-store architecture with per-store config, persistence, and APIs
- StoreManager: create/drop/list named stores with independent config - Per-store eviction policy, memory limit, TTL, cuckoo filter, persistence - Store config is immutable after creation (drop and recreate to change) - stores.json registry: runtime-created stores survive restarts - Per-store persistence directories: {data_dir}/stores/{name}/ - HTTP APIs: GET/POST /api/stores, GET/DELETE /api/stores/{name}, GET/PUT/DELETE /api/stores/{name}/cache/{key} - RESP: SELECT <store>, STORES commands with per-connection state - Existing /api/cache/{key} and RESP commands unchanged (default store) - Store-aware replication: events include store name, routed correctly - Fix: auto snapshots now fire on schedule (was stub) - Fix: auto AOF compaction now fires when size exceeds limit (was stub) - Fix: corrupt AOF lines skipped instead of blocking startup - Env var overrides: HYPERCACHE_DEFAULT_MEMORY, HYPERCACHE_DEFAULT_TTL, HYPERCACHE_DEFAULT_EVICTION, HYPERCACHE_DEFAULT_CUCKOO, HYPERCACHE_MAX_STORES, HYPERCACHE_CUCKOO_FILTER_FPP, HYPERCACHE_PERSISTENCE_ENABLED, HYPERCACHE_PERSISTENCE_STRATEGY - Default TTL changed to 0 (infinite); only default store ships OOTB - Updated README, config YAML, and architecture docs
1 parent b927d1d commit bcbdb75

10 files changed

Lines changed: 1134 additions & 185 deletions

File tree

README.md

Lines changed: 58 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -463,39 +463,68 @@ network:
463463
gossip_port: 7946
464464

465465
cache:
466-
max_memory: 1GB
467-
default_ttl: 1h
468-
cleanup_interval: 5m
469-
eviction_policy: "session"
466+
max_memory: "8GB"
467+
default_ttl: "0" # 0 = infinite (no expiry); set per-store or per-key
468+
cuckoo_filter_fpp: 0.01 # 1% false positive rate
469+
max_stores: 16 # max stores allowed (1-64)
470470

471471
persistence:
472472
enabled: true
473-
aof_enabled: true
474-
snapshot_enabled: true
475-
snapshot_interval: 300s
473+
strategy: "hybrid" # "aof", "snapshot", "hybrid"
474+
sync_policy: "everysec" # "always", "everysec", "no"
475+
```
476+
477+
### Environment Variable Overrides (Docker / K8s)
478+
Environment variables have highest priority and override both defaults and YAML config:
479+
480+
| Variable | Description | Example |
481+
|----------|-------------|---------|
482+
| `HYPERCACHE_DEFAULT_MEMORY` | Default store max memory | `4GB` |
483+
| `HYPERCACHE_DEFAULT_TTL` | Default store TTL (`0` = infinite) | `0`, `1h`, `30m` |
484+
| `HYPERCACHE_DEFAULT_EVICTION` | Default store eviction policy | `lru`, `lfu`, `fifo`, `ttl` |
485+
| `HYPERCACHE_DEFAULT_CUCKOO` | Default store cuckoo filter toggle | `true`, `false` |
486+
| `HYPERCACHE_MAX_STORES` | Maximum stores allowed | `16` |
487+
| `HYPERCACHE_CUCKOO_FILTER_FPP` | Cuckoo filter false positive rate | `0.01` |
488+
| `HYPERCACHE_PERSISTENCE_ENABLED` | Global persistence toggle | `true`, `false` |
489+
| `HYPERCACHE_PERSISTENCE_STRATEGY` | Global persistence strategy | `hybrid`, `aof`, `snapshot` |
490+
491+
```bash
492+
# Docker example: override default store config without a YAML file
493+
docker run -e HYPERCACHE_DEFAULT_MEMORY=4GB \
494+
-e HYPERCACHE_DEFAULT_TTL=0 \
495+
-e HYPERCACHE_DEFAULT_EVICTION=lru \
496+
-e HYPERCACHE_DEFAULT_CUCKOO=true \
497+
-e HYPERCACHE_PERSISTENCE_ENABLED=true \
498+
-e HYPERCACHE_PERSISTENCE_STRATEGY=hybrid \
499+
rishabhverma17/hypercache
476500
```
477501

478502
### Per-Store Configuration
479503
```yaml
480-
# Independent configuration for each data store
504+
# Only "default" store ships out of the box.
505+
# Store config is immutable — to change settings, drop and recreate the store.
506+
# Additional stores can be defined in YAML or created at runtime via API.
481507
stores:
482-
user_sessions:
483-
eviction_policy: "session" # Session-based eviction
484-
cuckoo_filter: true # Enable probabilistic operations
485-
persistence: "aof+snapshot" # Full persistence
486-
replication_factor: 3
487-
488-
page_cache:
508+
- name: "default"
489509
eviction_policy: "lru" # LRU eviction
490-
cuckoo_filter: false # Disable for pure cache
491-
persistence: "aof_only" # Write-ahead logging only
492-
replication_factor: 2
510+
max_memory: "8GB"
511+
default_ttl: "0" # 0 = infinite
512+
cuckoo_filter: true # Enable probabilistic lookups
513+
persistence: "hybrid" # "hybrid", "aof", "snapshot", "disabled"
514+
515+
- name: "sessions"
516+
eviction_policy: "ttl" # TTL-based eviction
517+
max_memory: "1GB"
518+
default_ttl: "30m"
519+
cuckoo_filter: true
520+
persistence: "aof" # Write-ahead logging only
493521
494-
temporary_data:
522+
- name: "temporary_data"
495523
eviction_policy: "lfu" # Least frequently used
496-
cuckoo_filter: true # Enable for membership tests
497-
persistence: "disabled" # In-memory only
498-
replication_factor: 1
524+
max_memory: "512MB"
525+
default_ttl: "15m"
526+
cuckoo_filter: false # Disable for pure cache
527+
persistence: "disabled" # In-memory only
499528
```
500529

501530
### Monitoring Configuration
@@ -600,22 +629,15 @@ Performance: Batched writes, zero-copy I/O
600629
#### **Per-Store Persistence Settings**
601630
```yaml
602631
stores:
603-
critical_data:
604-
persistence:
605-
mode: "aof+snapshot" # Full durability
606-
fsync: "always" # Immediate disk sync
607-
snapshot_interval: "60s" # Frequent snapshots
632+
- name: "critical_data"
633+
persistence: "hybrid" # Full durability (AOF + snapshots)
634+
# Uses global persistence settings for sync_policy, snapshot_interval, etc.
608635
609-
session_cache:
610-
persistence:
611-
mode: "aof_only" # Write-ahead logging
612-
fsync: "periodic" # Batched sync (1s)
613-
compression: true # Compress log files
636+
- name: "session_cache"
637+
persistence: "aof" # Write-ahead logging only
614638
615-
temporary_cache:
616-
persistence:
617-
mode: "disabled" # In-memory only
618-
# No disk I/O overhead for temporary data
639+
- name: "temporary_cache"
640+
persistence: "disabled" # In-memory only — no disk I/O overhead
619641
```
620642

621643
#### **Durability vs Performance Tuning**

0 commit comments

Comments
 (0)