Skip to content

pilotspace/moon

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

1,066 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Moon Banner

A Redis-compatible in-memory data store, written from scratch in Rust.

Version Rust SDK Python SDK License Status Rust Protocol

Quick startWhy MoonBenchmarksDocsChangelog


⚠ Experimental. Moon is under active development and not recommended for production. Storage formats, APIs, and config flags may change between releases. Please open an issue if something breaks.


Moon speaks the Redis wire protocol (RESP2/RESP3) and implements 230+ commands. It runs on Linux (io_uring via monoio) and macOS (kqueue via monoio) with a thread-per-core, shared-nothing architecture, per-shard WAL, tiered disk offload, an in-process vector search engine with BM25 full-text search, a property graph engine with Cypher subset, cross-store ACID transactions, workspace partitioning, durable message queues, bi-temporal MVCC, and an embedded web console. Any Redis client connects out of the box.

Why Moon

  • Thread-per-core, zero shared state. Each shard owns its own event loop, DashTable, WAL writer, and Pub/Sub registry. No global locks; cross-shard dispatch is a lock-free SPSC channel.
  • Dual runtime. Monoio (io_uring on Linux, kqueue on macOS) for peak throughput; Tokio for portability and CI. Same binary, feature-gated.
  • Forkless persistence. RDB snapshots iterate DashTable segments incrementally — no fork(), no COW memory spike. AOF is a per-shard WAL with batched fsync; the advantage over Redis grows with pipeline depth.
  • Tiered disk offload. Keys evicted under maxmemory spill to NVMe instead of being deleted, with async write and read-through. 100% crash recovery across all tiers.
  • Memory-optimized types. CompactKey (23-byte SSO), CompactValue (16-byte SSO with inline TTL), HeapString, B+ tree sorted sets, and per-request bumpalo arenas — 27–35% less RSS than Redis at 1 KB+ values.

Operator note: Moon's resident memory (RSS) is what you bill for; virtual memory (VSZ) reserved by the allocator is not. See docs/OPERATOR-GUIDE.md#memory-accounting for the full VSZ-vs-RSS guide, MEMORY DOCTOR interpretation, and --memory-arenas-cap / mimalloc-alt tuning knobs.

  • In-process vector search. FT.CREATE / FT.SEARCH with HNSW + TurboQuant 4/8-bit quantization. 12.7K search QPS at 384d COSINE on GCloud x86_64.
  • BM25 full-text search. FT.AGGREGATE with GROUPBY/REDUCE, typo tolerance (Levenshtein fuzzy), TAG and NUMERIC field types, HIGHLIGHT/SUMMARIZE, and three-way RRF hybrid fusion (BM25 + dense + sparse).
  • Property graph engine. 14 GRAPH.* commands with Cypher subset, hybrid graph+vector queries, SIMD-accelerated traversal, and memory-mapped CSR segments. 23× FalkorDB insert, 2.4× Cypher QPS.
  • Cross-store ACID transactions. TXN.BEGIN / TXN.COMMIT / TXN.ABORT for atomic writes across KV, vector, and graph stores with undo-log rollback.
  • Workspace partitioning. WS CREATE / WS AUTH — multi-tenant namespace isolation with transparent key prefixing and per-shard registries.
  • Durable message queues. MQ CREATE / MQ PUSH / MQ POP / MQ ACK — at-least-once delivery with dead-letter queues, debounced triggers, and WAL-backed crash recovery.
  • Bi-temporal MVCC. TEMPORAL.SNAPSHOT_AT / TEMPORAL.INVALIDATE — point-in-time queries across KV (FT.SEARCH AS_OF) and graph (GRAPH.QUERY VALID_AT).
  • Embedded web console. 7-view React UI (Dashboard, Browser, Console, Vectors, Graph, Memory) served at /ui/ — zero deployment, one binary. REST + WebSocket + SSE gateway with Bearer auth and rate limiting.

Moon vs Redis Architecture

Benchmarks

Measured vs Redis 8.6.1, co-located client and server, pipeline depth tuned per row. Full methodology and reproduction steps in BENCHMARK.md and docs/benchmarks.mdx.

Peak throughput (GCloud c3-standard-8, x86_64, monoio io_uring)

Workload Moon Redis Ratio
Peak GET (c=50, p=64) 5.11M 2.98M 1.72×
Peak SET (c=50, p=64) 3.50M 1.82M 1.92×
GET, production defaults (AOF+disk-offload) 4.76M 2.46M 1.93×
GET, max durability (fsync always) 4.85M 2.45M 1.98×
Memory, values ≥ 1 KB 27–35% less
Crash recovery (SIGKILL, 5K keys) 100% 100% parity

ARM64 (GCloud t2a-standard-8, Neoverse-N1)

Workload Moon Redis Ratio
Peak GET (c=50, p=64) 3.47M 1.58M 2.20×
Peak SET (c=50, p=64) 2.42M 1.15M 2.10×
GET, production defaults 3.45M 1.61M 2.14×

Vector search (50K × 384d, COSINE, GCloud x86_64)

Moon Redis (RediSearch) Qdrant
Insert rate 8.2K/s ~4.0K/s ~6.6K/s
Search QPS 12.7K 3.8K 982
Recall@10 (MiniLM) 0.92 0.95 0.96

Graph engine (2K nodes, 6K edges, GCloud x86_64)

Moon FalkorDB
Cypher QPS 2.4×
Native API QPS 19× N/A
Bulk insert 23×

Quick start

Prerequisites

  • Rust stable toolchain (edition 2024)
  • cmake (required by aws-lc-rs for TLS)

Build and run

git clone https://github.com/pilotspace/moon.git
cd moon
cargo build --release

# Defaults: bind 127.0.0.1:6379, shard count = CPU count
./target/release/moon

# Or with production flags
./target/release/moon \
  --port 6379 \
  --shards 8 \
  --appendonly yes --appendfsync everysec \
  --maxmemory 8g --maxmemory-policy allkeys-lfu

Connect with any Redis client

redis-cli -p 6379
127.0.0.1:6379> SET hello world
OK
127.0.0.1:6379> GET hello
"world"
127.0.0.1:6379> HSET user:1 name Alice age 30
(integer) 2
127.0.0.1:6379> FT.CREATE idx ON HASH PREFIX 1 doc: SCHEMA emb VECTOR HNSW 6 DIM 384 TYPE FLOAT32 DISTANCE_METRIC COSINE
OK

Docker

Multi-stage build with cargo-chef caching and a distroless runtime (~41 MB final image):

docker build -t moon .
docker run -d -p 6379:6379 -v moon-data:/data moon \
  moon --bind 0.0.0.0 --appendonly yes

See docs/quickstart.mdx for alternative build configs, TLS setup, and Docker Compose.

Client SDKs

Pythonmoondb on PyPI:

pip install moondb

Rustmoondb on crates.io:

[dependencies]
moondb = "0.1"
tokio = { version = "1", features = ["full"] }

Python SDK (10 lines to vector search)

pip install moondb
from moondb import MoonClient, encode_vector

client = MoonClient(host="localhost", port=6379, decode_responses=True)

# Create a vector index (HNSW, 384 dimensions, cosine similarity)
client.vector.create_index("docs", dim=384, metric="COSINE", prefix="doc:")

# Store a document -- auto-indexed on HSET
client.hset("doc:1", mapping={"title": "Hello", "vec": encode_vector([0.1] * 384)})

# Search by vector similarity
results = client.vector.search("docs", [0.1] * 384, k=5)
for r in results:
    print(r.key, r.score, r.fields)

See examples/ for complete tutorials: RAG Quickstart, Semantic Cache, GraphRAG, AI Agent Tools.

Moon Console (Web UI)

Moon ships an embedded web console at http://localhost:9100/ui/ — no separate install needed.

# Start Moon with the admin port enabled
./target/release/moon --port 6379 --admin-port 9100 --shards 4

Open http://localhost:9100/ui/ in your browser. The console has 7 views:

View What it does
Dashboard Real-time QPS, latency P50/P99, memory, clients, keyspace — driven by SSE at 1 Hz
Browser Namespace tree, virtual-scrolled key list, type-specific editors for all 6 data types
Console Monaco editor with RESP + Cypher syntax, 233-command autocomplete, multi-tab, history
Vectors 3D UMAP projection of vector indexes, HNSW layer overlay, KNN search with distance rings
Graph Force-directed 3D layout of graph data, Cypher query editor, node/edge inspector
Memory Keyspace treemap, slowlog table, command stats
Help Getting Started guide with seed examples

Seed data to explore all views

Use the built-in seed script to populate KV, vector, and graph data:

# Install Python deps (one-time)
pip install redis numpy

# Seed 2000 KV keys, 50K vectors (384d), 10K graph nodes + 30K edges
python3 scripts/seed-console-fixtures.py \
  --resp-port 6379 \
  --admin-port 9100 \
  --kv-count 2000 \
  --vector-count 50000 \
  --graph-nodes 10000 \
  --graph-edges 30000

Or seed manually via the Console view (Cmd+Enter to execute):

# KV data with namespaces
SET user:1:name "Alice"
SET user:1:email "alice@example.com"
HSET session:1 user_id 1 created_at 1712345678 ttl 3600
LPUSH queue:emails "welcome-alice" "verify-alice"
SADD tags:user:1 "admin" "early-adopter" "beta"
ZADD leaderboard 9500 "alice" 8200 "bob" 7100 "charlie"

# Vector index (auto-indexed on HSET)
FT.CREATE embeddings ON HASH PREFIX 1 doc: SCHEMA v VECTOR HNSW 6 DIM 384 TYPE FLOAT32 DISTANCE_METRIC COSINE

# Graph data
GRAPH.CREATE social
GRAPH.ADDNODE social alice Person '{"name":"Alice","age":30}'
GRAPH.ADDNODE social bob Person '{"name":"Bob","age":25}'
GRAPH.ADDEDGE social alice bob FOLLOWS '{"since":"2024"}'
GRAPH.QUERY social "MATCH (a:Person)-[:FOLLOWS]->(b) RETURN a.name, b.name"

Auth and CORS (production)

# Enable Bearer auth + CORS allowlist
./target/release/moon \
  --port 6379 \
  --admin-port 9100 \
  --console-auth-required \
  --console-auth-secret "your-secret-key" \
  --console-cors-origin "https://your-domain.com" \
  --console-rate-limit 100 \
  --console-rate-burst 200

Features at a glance

Category Highlights
Data types Strings, lists, hashes, sets, sorted sets, streams, HyperLogLog, bitmaps, vectors
Persistence Forkless RDB, per-shard AOF (always/everysec/no), WAL v2 framing, tiered disk offload
Networking RESP2/RESP3, HELLO negotiation, TLS 1.3 (rustls + aws-lc-rs), mTLS, pipelining, client-side caching
Clustering 16,384 hash slots, gossip, MOVED/ASK, live slot migration, PSYNC2 replication (v0.1.x: --shards 1 master only — see clustering guide), majority-vote failover
Scripting & security Lua 5.4 (EVAL/EVALSHA), ACL users/keys/channels/commands, protected mode
Vector search FT.CREATE/FT.SEARCH/FT.AGGREGATE, HNSW + TurboQuant 4-bit, auto-indexing on HSET, hybrid dense+sparse+BM25
Full-text search BM25 inverted index, typo tolerance (Levenshtein), TAG and NUMERIC fields, HIGHLIGHT/SUMMARIZE, three-way RRF fusion
Graph engine 14 GRAPH.* commands, Cypher subset (MATCH/WHERE/RETURN/CREATE/DELETE/SET/MERGE), hybrid graph+vector queries, CSR segments, SIMD cosine
Transactions MULTI/EXEC (Redis compat) + TXN.BEGIN/TXN.COMMIT/TXN.ABORT (cross-store ACID with undo-log rollback)
Workspaces WS CREATE/WS AUTH/WS LIST — multi-tenant namespace isolation with transparent key prefixing
Message queues MQ CREATE/MQ PUSH/MQ POP/MQ ACK — durable queues with dead-letter, triggers, WAL recovery
Temporal TEMPORAL.SNAPSHOT_AT/TEMPORAL.INVALIDATE — bi-temporal MVCC for KV and graph point-in-time queries
Web console Dashboard, Browser, Console, Vector Explorer, Graph Explorer, Memory view — embedded in binary, served at /ui/
Observability INFO, SLOWLOG, COMMAND DOCS, OBJECT, DEBUG, MEMORY, Prometheus metrics, structured tracing logs

Full command list: docs/commands.mdx. Configuration flags: docs/configuration.mdx. Architecture deep-dive: docs/architecture.mdx. Guides: transactions, workspaces, message queues, temporal queries, full-text search.

Development

# Unit tests (2,797+ tests)
cargo test --lib

# Full CI matrix (native macOS + Linux via OrbStack)
cargo fmt --check && cargo clippy -- -D warnings && cargo test --release

# Data-consistency tests vs Redis as ground truth (132 tests, 1/4/12 shards)
./scripts/test-consistency.sh

# Throughput comparison vs Redis
./scripts/bench-production.sh

# Flamegraph a hot path
cargo flamegraph --bin moon -- --port 6399 --shards 1

Contribution guide and coding rules (unsafe policy, hot-path allocation rules, lock discipline) are in CLAUDE.md and UNSAFE_POLICY.md.

Roadmap

Moon is pre-1.0 and experimental. Current focus:

  • Correctness parity with Redis 8.x across the full command surface
  • AI-native primitives: session dedup, hybrid vector+sparse search, agentic caching
  • Multi-node clustering with gossip, slot migration, and PSYNC2 replication
  • GPU-accelerated vector search (CUDA, feature-gated)
  • Production hardening and SLO validation (see docs/PRODUCTION-CONTRACT.md)

Completed in v0.1.0–v0.1.8:

  • Tiered disk offload (RAM → NVMe) with 100% crash recovery
  • In-process vector search (HNSW + TurboQuant 4/8-bit) with FT.* API
  • BM25 full-text search with three-way hybrid fusion (BM25 + dense + sparse)
  • Property graph engine with Cypher subset (14 GRAPH.* commands)
  • Cross-store ACID transactions (TXN.BEGIN/COMMIT/ABORT) across KV, vector, and graph
  • Workspace partitioning for multi-tenant namespace isolation
  • Durable message queues with dead-letter and debounced triggers
  • Bi-temporal MVCC for point-in-time KV and graph queries
  • Web console (7 views, embedded in binary)
  • macOS support (aarch64 + x86_64, both runtimes)
  • Thread-per-core dispatch optimization (5.11M GET/s on x86_64)

Production readiness is not a v0.1 goal. Storage formats, APIs, and config flags may change between releases.

Production Readiness

Moon's v1.0 promises — SLOs, durability modes, supported platforms, and a machine-checkable GA exit-criteria checklist — live in docs/PRODUCTION-CONTRACT.md. The contract is the single source of truth every v0.1.3 hardening phase tests against.

Credits

Moon stands on the shoulders of systems research and an open-source ecosystem. Headline credits:

Full list with per-dependency rationale, research paper summaries, and benchmarking methodology: docs/references.mdx.

License

Apache License 2.0

About

moon - memory boosting for Spaceship

Resources

License

Contributing

Security policy

Stars

Watchers

Forks

Packages

 
 
 

Contributors