Skip to content
View robustmq's full-sized avatar

Block or report robustmq

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Maximum 250 characters. Please don't include any personal information such as legal names or email addresses. Markdown supported. This note will be visible to only you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
robustmq/README.md

RobustMQ Logo

Ask DeepWiki zread Latest Release License GitHub issues GitHub stars Coverage Build Status Rust Version

Communication infrastructure for the AI era — one binary, one broker, one storage layer, any protocol

What is RobustMQWhy RobustMQFeaturesRoadmapArchitectureQuick StartDocumentationContributingCommunity


⚠️ Development Status RobustMQ is in early development and not yet production-ready. MQTT core is stable and continuing to mature. Kafka, NATS, and AMQP are under active development. Production readiness is targeted for 0.4.0.

🌟 What is RobustMQ

RobustMQ is a unified messaging engine built with Rust. One binary, one broker, no external dependencies — deployable from edge devices to cloud clusters. It natively supports MQTT, Kafka, NATS, and AMQP on a shared storage layer: one message written once, consumed by any protocol.

RobustMQ Architecture

Six core scenarios on one system:

Scenario How
AI Agent communication $AI.API.* subject space over NATS: native Agent registration, discovery, invocation, and load balancing
IoT device ingestion Devices publish via MQTT; AI platforms and data pipelines consume via Kafka — same data, no bridging
Streaming data pipelines Standard Kafka protocol, existing Kafka SDKs connect with zero migration cost
Edge-to-cloud sync Single binary, near-zero memory, offline buffering with automatic cloud sync on reconnect
Ultra-low-latency dispatch NATS pure in-memory routing — no disk writes, millisecond to sub-millisecond latency
Traditional messaging Native AMQP support — existing RabbitMQ applications migrate with minimal changes
MQTT publish  →  RobustMQ unified storage  →  Kafka consume
                                           →  NATS subscribe
                                           →  AMQP consume

🤔 Why RobustMQ

Today's messaging infrastructure is a collection of protocol silos. IoT uses MQTT brokers, data pipelines use Kafka, enterprise systems use RabbitMQ, and AI Agent communication has no native solution. Multiple systems mean duplicate data copies, overlapping operations, and bridging layers that add latency and failure points.

Existing systems carry heavy architectural baggage. Kafka's file-system-based design hits a hard ceiling at tens of thousands of topics. RabbitMQ's Erlang runtime limits throughput headroom. None of these systems were designed for the AI era — retrofitting them is patching old foundations.

RobustMQ is designed from scratch to solve this structurally: unified storage + native multi-protocol support. Not bridging, not routing — one copy of data, each protocol reading it through its own semantic lens. One system replaces multiple brokers. No data duplication, no operational overlap.

✨ Features

  • 🦀 Rust-native: No GC, stable and predictable memory footprint, no periodic spikes — consistent from edge devices to cloud clusters
  • 🗄️ Unified storage layer: All protocols share one storage engine — data written once, consumed by any protocol, no duplication
  • 🔌 Native multi-protocol: MQTT 3.1/3.1.1/5.0, Kafka, NATS, AMQP — natively implemented, full protocol semantics, not emulated
  • 🏢 Native multi-tenancy: Unified across all protocols — full data isolation and independent permission management per tenant
  • 🌐 Edge-to-cloud: Single binary, zero dependencies, offline buffering with auto-sync — same runtime from edge gateways to cloud clusters
  • 🤖 AI Agent communication: NATS-based $AI.API.* extension — native Agent registration, discovery, invocation, and orchestration
  • Ultra-low-latency dispatch: NATS pure in-memory routing — no disk writes, millisecond to sub-millisecond latency
  • 💾 Multi-mode storage: Memory / RocksDB / File, per-topic configuration, automatic cold data tiering to S3
  • 🔄 Shared subscription: Break the "concurrency = partition count" limit — consumers scale elastically at any time
  • 🛠️ Minimal operations: Single binary, zero external dependencies, built-in Raft consensus, ready out of the box

🗺️ Roadmap

The approach: slow is smooth, smooth is fast. Each phase done properly before moving on.

Phase 1 — MQTT (current)
  MQTT core production-ready, continuously refined to be the best MQTT Broker available
  Architecture and infrastructure hardened in parallel

Phase 2 — NATS + AI Agent (in progress)
  NATS protocol compatibility + $AI.API.* extension
  Native Agent registration, discovery, invocation, and load balancing

Phase 3 — Kafka (in progress)
  Full Kafka protocol compatibility
  Complete the IoT-to-streaming data path, edge-to-cloud data flow

Phase 4 — AMQP (planned)
  Full AMQP protocol compatibility
  Traditional enterprise messaging migration path
Feature Status
MQTT 3.x / 5.0 core ✅ Available
Session persistence and recovery ✅ Available
Shared subscription ✅ Available
Authentication and ACL ✅ Available
Grafana + Prometheus monitoring ✅ Available
Web management console ✅ Available
Kafka protocol 🚧 In development
NATS protocol 🔬 Demo validated, in development
AMQP protocol 🔬 Demo validated, in development
$AI.API.* Agent communication 🔬 Demo validated, in development

🏗️ Architecture

RobustMQ has three components with fixed, clean boundaries:

  • Meta Service — metadata management, Raft-based consensus
  • Broker — protocol parsing and routing (MQTT / Kafka / NATS / AMQP)
  • Storage Engine — unified data storage with pluggable backends

Adding a new protocol means implementing only the Broker parsing layer. Adding a new storage backend means implementing only the Storage Engine interface. The core architecture does not change.

🚀 Quick Start

One-Line Installation

curl -fsSL https://raw.githubusercontent.com/robustmq/robustmq/main/scripts/install.sh | bash
broker-server start

Multi-Protocol in Action

# Publish via MQTT
mqttx pub -h localhost -p 1883 -t "robustmq.multi.protocol" -m "Hello RobustMQ!"

# Consume the same message via Kafka
kafka-console-consumer.sh --bootstrap-server localhost:9092 \
  --topic robustmq.multi.protocol --from-beginning

# Consume the same message via NATS
nats sub "robustmq.multi.protocol"

Web Dashboard

Access http://localhost:8080 for cluster monitoring and management.

Web UI Web UI Cluster

Try Online Demo

📚 Full installation and usage guide: Documentation

🔧 Development

git clone https://github.com/robustmq/robustmq.git
cd robustmq
cargo run --package cmd --bin broker-server

make build           # Basic build
make build-full      # With frontend

📚 Build Guide

📚 Documentation

🤝 Contributing

We welcome contributions. See our Contribution Guide and Good First Issues.

🌐 Community

🇨🇳 Chinese Community

  • 微信群: Join our WeChat group for Chinese-speaking users

    WeChat Group QR Code
  • 开发者微信: If the group QR code has expired, follow our official WeChat account

    WeChat Official Account QR Code

License

RobustMQ is licensed under the Apache License 2.0. See LICENSING.md for details.


Built with ❤️ by the RobustMQ team and contributors.

Popular repositories Loading

  1. robustmq robustmq Public

    Communication infrastructure for the AI era — one binary, one broker, one storage layer, any protocol

    Rust 1.5k 216

  2. robustmq-geek robustmq-geek Public

    Rust 57 8

  3. robustmq-copilot robustmq-copilot Public

    The one-stop RobustMQ operation and maintenance management platform

    TypeScript 6 6

  4. robustmq-proto robustmq-proto Public

    Rust 4 11

  5. rust-properties rust-properties Public

    1 1

  6. raft-rs raft-rs Public

    Forked from tikv/raft-rs

    Raft distributed consensus algorithm implemented in Rust.

    Rust