docs: add bus architecture guide + integration tests

chazmaniandinkle · claude · chazmaniandinkle · commit e64c7f9ebf85 · 2026-04-07T20:06:52.000-04:00
ARCHITECTURE.md (216 lines):
- Core types (CognitiveEvent, CognitiveIntent, EncodedOutput)
- Bus operations (perceive, act, hud) with flow diagrams
- Current modalities (voice, text)
- How to add a new modality (4-step guide)
- Integration points (MCP + HTTP API)

tests/test_bus_wiring.py (6 tests, 5 pass, 1 skip):
- Bus singleton exists and has VoiceModule
- health() and hud() return expected structure
- diagnostics() MCP tool includes bus state
- http_api import check (skipped: missing pysbd dep)

Co-Authored-By: Claude Opus 4.6 (1M context) &lt;noreply@anthropic.com&gt;
diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md
@@ -0,0 +1,216 @@
+# Mod3 Architecture: The Modality Bus
+
+The modality bus is the sensorimotor boundary between cognitive agents and physical signals. Agents think in cognitive events ("someone spoke", "say this"); the bus translates between those events and raw bytes (audio, text, future: vision, spatial).
+
+```
+                        ModalityBus
+    ┌──────────────────────────────────────────────┐
+    │                                              │
+    │  ┌─────────┐  ┌─────────┐  ┌─────────┐      │
+    │  │  Voice   │  │  Text   │  │ Vision* │ ...  │
+    │  │ Module   │  │ Module  │  │ Module  │      │
+    │  └────┬─────┘  └────┬────┘  └────┬────┘      │
+    │       │             │            │            │
+    │  ┌────┴─────────────┴────────────┴────┐      │
+    │  │         Event Log + Listeners       │      │
+    │  └────┬─────────────┬────────────┬────┘      │
+    │       │             │            │            │
+    │  ┌────┴────┐  ┌─────┴─────┐  ┌──┴───┐       │
+    │  │ Channel │  │  Channel  │  │ ...  │       │
+    │  │ discord │  │  http-api │  │      │       │
+    │  └─────────┘  └───────────┘  └──────┘       │
+    └──────────────────────────────────────────────┘
+
+    * Vision/Spatial are defined in ModalityType but not yet implemented.
+```
+
+## Core Types (modality.py)
+
+### Cognitive Primitives
+
+The agent never touches raw bytes. It sees these:
+
+```python
+@dataclass
+class CognitiveEvent:          # Input percept
+    modality: ModalityType     # VOICE, TEXT, VISION, SPATIAL
+    content: str               # The meaning (transcribed text, caption, etc.)
+    source_channel: str        # Which channel it arrived on
+    confidence: float          # Decoder certainty (0.0 - 1.0)
+    timestamp: float
+    metadata: dict[str, Any]
+
+@dataclass
+class CognitiveIntent:         # Output intent (not yet encoded)
+    modality: ModalityType | None  # None = let the bus decide
+    content: str               # What to communicate
+    target_channel: str        # Specific channel, or "" for bus routing
+    priority: int              # Higher = more urgent
+    metadata: dict[str, Any]   # voice, speed, emotion, etc.
+
+@dataclass
+class EncodedOutput:           # Raw signal ready for delivery
+    modality: ModalityType
+    data: bytes                # WAV, PNG, JSON, etc.
+    format: str                # "wav", "png", "text", etc.
+    duration_sec: float
+    metadata: dict[str, Any]
+```
+
+### Abstract Base Classes
+
+Every modality module implements three components:
+
+```python
+class Gate(ABC):
+    def check(self, raw: bytes, **kwargs) -> GateResult: ...
+
+class Decoder(ABC):
+    def decode(self, raw: bytes, **kwargs) -> CognitiveEvent: ...
+
+class Encoder(ABC):
+    def encode(self, intent: CognitiveIntent) -> EncodedOutput: ...
+
+class ModalityModule(ABC):
+    modality_type -> ModalityType   # Which modality this handles
+    gate -> Gate | None             # Input filter (None = pass all)
+    decoder -> Decoder | None       # raw -> CognitiveEvent
+    encoder -> Encoder | None       # CognitiveIntent -> EncodedOutput
+    state -> ModuleState            # Live HUD state
+    health() -> dict                # Diagnostics
+```
+
+`Gate` is optional. Text has no gate (all text passes). Voice uses VAD to reject silence.
+
+## The Bus (bus.py)
+
+`ModalityBus` manages module registration, signal routing, and state tracking.
+
+### perceive() -- Input Path
+
+```
+raw bytes ──→ Gate.check() ──→ Decoder.decode() ──→ CognitiveEvent
+                  │                   │
+              (rejected?)        (empty content?)
+                  ↓                   ↓
+               None               None (filtered)
+```
+
+```python
+bus.perceive(raw: bytes, modality: str | ModalityType, channel: str = "", **kwargs)
+    -> CognitiveEvent | None
+```
+
+1. Resolve the modality module from the registry
+2. If the module has a gate, run `gate.check(raw)`. Emit a `modality.gate` bus event. Return `None` if rejected.
+3. Run `decoder.decode(raw)`. If content is empty (e.g., hallucination filtered), emit `modality.filtered` and return `None`.
+4. Stamp `source_channel`, emit `modality.input`, return the event.
+
+### act() -- Output Path
+
+```
+CognitiveIntent ──→ resolve modality ──→ Encoder.encode() ──→ EncodedOutput
+                                                                    │
+                                                          channel.deliver()
+```
+
+```python
+bus.act(intent: CognitiveIntent, channel: str = "", blocking: bool = False)
+    -> QueuedJob | EncodedOutput
+```
+
+1. Resolve output modality: explicit on intent, or inferred from channel capabilities (prefers voice over text), or defaults to text.
+2. Encode via the module's encoder. Emits `modality.encode_start` and `modality.output` bus events.
+3. If the target channel has a `deliver` callback, call it with the encoded output.
+4. If `blocking=True`, returns `EncodedOutput` directly. Otherwise queues via `OutputQueueManager` and returns a `QueuedJob`.
+
+### hud() -- Agent Awareness
+
+```python
+bus.hud() -> dict
+```
+
+Returns a live snapshot of all modules and channels: current status, active jobs, queue depths, recent events. Designed to be injected into the agent's context window so it knows what the body is doing.
+
+### Channels
+
+Channels declare which modalities they support. The bus auto-routes output based on channel capabilities.
+
+```python
+bus.register_channel("discord-voice", [ModalityType.VOICE, ModalityType.TEXT],
+                     deliver=send_to_discord)
+```
+
+### Bus Events
+
+Every boundary crossing is recorded as a `BusEvent` (type, modality, channel, timestamp, data). Listeners can subscribe via `bus.on_event(callback)` for ledger integration. The bus keeps the last 500 events in memory.
+
+## Current Modalities
+
+### Voice (modules/voice.py)
+
+| Component | Class | Implementation |
+|-----------|-------|----------------|
+| Gate | `VoiceGate` | Silero VAD via `vad.detect_speech()`. Threshold-configurable (default 0.5). Rejects audio with no detected speech. |
+| Decoder | `WhisperDecoder` | `mlx_whisper` STT on Apple Silicon. Lazy-loads `mlx-community/whisper-turbo`. Applies `vad.is_hallucination()` filter to reject phantom transcripts. |
+| Decoder (legacy) | `PlaceholderDecoder` | Accepts pre-transcribed text. Used by the MCP server for the `speak` tool path where text is already known. |
+| Encoder | `VoiceEncoder` | Wraps `engine.synthesize()` (Kokoro, Voxtral, Chatterbox, Spark). Default voice: `bm_lewis` at 1.25x speed. Returns WAV bytes. |
+
+### Text (modules/text.py)
+
+| Component | Class | Implementation |
+|-----------|-------|----------------|
+| Gate | None | All text passes through. |
+| Decoder | `TextDecoder` | Identity transform: `bytes.decode("utf-8")` -> `CognitiveEvent`. |
+| Encoder | `TextEncoder` | Identity transform: `intent.content.encode("utf-8")` -> `EncodedOutput`. |
+
+Text exists so it is a first-class modality on the bus, not a special case.
+
+## Integration Points
+
+### MCP Server (server.py)
+
+The MCP server creates the bus singleton at module level:
+
+```python
+_bus = _create_bus()  # ModalityBus with VoiceModule(decoder=PlaceholderDecoder())
+```
+
+MCP tools (`speak`, `diagnostics`, `vad_check`) use `_bus` for voice state tracking, health reports, and VAD. The `speak` tool resolves voices through the bus's voice module, sets encoder state, and uses the engine directly for synthesis (the adaptive player handles local playback).
+
+The `diagnostics` tool returns `_bus.health()` and `_bus.hud()`.
+
+### HTTP API (http_api.py)
+
+The HTTP API imports the bus singleton from the MCP server:
+
+```python
+from server import _bus as _shared_bus  # Shared instance when co-hosted
+_bus = _shared_bus                       # Falls back to fresh ModalityBus if import fails
+```
+
+It ensures both Text and Voice modules are registered, then exposes the bus directly:
+
+| Endpoint | Bus Method |
+|----------|------------|
+| `GET /v1/bus/hud` | `_bus.hud()` |
+| `GET /v1/bus/health` | `_bus.health()` |
+| `POST /v1/bus/perceive` | `_bus.perceive(raw, modality, channel)` |
+| `POST /v1/bus/act` | `_bus.act(intent, channel, blocking=True)` |
+| `GET /health` | includes `_bus.health()` and `_bus.hud()` |
+
+When running with `--all`, both MCP and HTTP share the same bus instance and model cache.
+
+## Adding a New Modality
+
+1. **Create `modules/your_modality.py`** -- implement `Gate`, `Decoder`, `Encoder` (all optional), and a `ModalityModule` subclass that wires them together. See `modules/text.py` for the minimal case or `modules/voice.py` for the full pattern.
+
+2. **Add the modality type** to `ModalityType` in `modality.py` if needed. `VISION` and `SPATIAL` are already defined.
+
+3. **Register with the bus** where it is created (`server.py` and/or `http_api.py`):
+   ```python
+   bus.register(VisionModule())
+   bus.register_channel("webcam-feed", [ModalityType.VISION])
+   ```
+
+4. **No routing changes needed.** The bus auto-routes `act()` based on channel capabilities. The HTTP API's `/v1/bus/perceive` and `/v1/bus/act` already accept any registered modality via the `modality` parameter.
diff --git a/tests/test_bus_wiring.py b/tests/test_bus_wiring.py
@@ -0,0 +1,166 @@
+"""Integration tests for Mod3 bus wiring.
+
+Verify that the ModalityBus singleton in server.py is correctly
+instantiated and wired through to http_api.py, with a VoiceModule
+registered and all key APIs returning expected structures.
+
+Run: python3 -m pytest tests/test_bus_wiring.py -v
+"""
+
+import json
+import os
+import sys
+
+import pytest
+
+# Ensure the project root is on the path so imports resolve
+sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
+
+
+# ---------------------------------------------------------------------------
+# Helpers
+# ---------------------------------------------------------------------------
+
+
+def _skip_if_import_fails(module_name: str):
+    """Return a pytest skip decorator if the given module cannot be imported."""
+    try:
+        __import__(module_name)
+    except (ImportError, ModuleNotFoundError) as e:
+        pytest.skip(f"{module_name} unavailable: {e}")
+
+
+# ---------------------------------------------------------------------------
+# Tests
+# ---------------------------------------------------------------------------
+
+
+def test_bus_singleton_exists():
+    """server._bus exists and is a ModalityBus instance."""
+    from bus import ModalityBus
+    from server import _bus
+
+    assert _bus is not None, "_bus should not be None"
+    assert isinstance(_bus, ModalityBus), f"_bus should be a ModalityBus, got {type(_bus).__name__}"
+
+
+def test_bus_has_voice_module():
+    """The server bus has a VoiceModule registered under ModalityType.VOICE."""
+    from modality import ModalityType
+    from modules.voice import VoiceModule
+    from server import _bus
+
+    modules = getattr(_bus, "_modules", {})
+    assert ModalityType.VOICE in modules, "Bus should have a VOICE module registered"
+    voice_module = modules[ModalityType.VOICE]
+    assert isinstance(voice_module, VoiceModule), (
+        f"VOICE module should be VoiceModule, got {type(voice_module).__name__}"
+    )
+    # The server uses PlaceholderDecoder (no heavy model deps)
+    assert voice_module.gate is not None, "VoiceModule should have a gate"
+    assert voice_module.decoder is not None, "VoiceModule should have a decoder"
+    assert voice_module.encoder is not None, "VoiceModule should have an encoder"
+
+
+def test_bus_health_returns_dict():
+    """_bus.health() returns a dict with modules, channels, queues, event_count."""
+    from server import _bus
+
+    health = _bus.health()
+    assert isinstance(health, dict), f"health() should return a dict, got {type(health).__name__}"
+
+    expected_keys = {"modules", "channels", "queues", "event_count"}
+    assert expected_keys.issubset(health.keys()), (
+        f"health() missing keys: {expected_keys - health.keys()}"
+    )
+
+    # modules should contain at least 'voice'
+    assert "voice" in health["modules"], "health() modules should include 'voice'"
+
+    voice_health = health["modules"]["voice"]
+    assert "has_gate" in voice_health, "voice health should report has_gate"
+    assert "has_decoder" in voice_health, "voice health should report has_decoder"
+    assert "has_encoder" in voice_health, "voice health should report has_encoder"
+    assert voice_health["has_gate"] is True
+    assert voice_health["has_decoder"] is True
+    assert voice_health["has_encoder"] is True
+
+
+def test_bus_hud_returns_dict():
+    """_bus.hud() returns a dict with modules, channels, queues, recent_events."""
+    from server import _bus
+
+    hud = _bus.hud()
+    assert isinstance(hud, dict), f"hud() should return a dict, got {type(hud).__name__}"
+
+    expected_keys = {"modules", "channels", "queues", "recent_events"}
+    assert expected_keys.issubset(hud.keys()), (
+        f"hud() missing keys: {expected_keys - hud.keys()}"
+    )
+
+    # modules should contain 'voice' with status info
+    assert "voice" in hud["modules"], "hud() modules should include 'voice'"
+    voice_hud = hud["modules"]["voice"]
+    assert "status" in voice_hud, "voice HUD entry should have 'status'"
+    assert voice_hud["status"] == "idle", f"voice status should be 'idle', got {voice_hud['status']}"
+
+    # timestamp should be present and numeric
+    assert "timestamp" in hud, "hud() should include a timestamp"
+    assert isinstance(hud["timestamp"], (int, float)), "timestamp should be numeric"
+
+    # recent_events should be a list
+    assert isinstance(hud["recent_events"], list), "recent_events should be a list"
+
+    # channels and queues should be dicts
+    assert isinstance(hud["channels"], dict), "channels should be a dict"
+    assert isinstance(hud["queues"], dict), "queues should be a dict"
+
+
+def test_diagnostics_includes_bus():
+    """The diagnostics() MCP tool response includes a 'bus' key with health and hud."""
+    from server import diagnostics
+
+    raw = diagnostics()
+    data = json.loads(raw)
+
+    assert "bus" in data, "diagnostics() should include a 'bus' key"
+    bus_data = data["bus"]
+
+    assert "health" in bus_data, "bus section should include 'health'"
+    assert "hud" in bus_data, "bus section should include 'hud'"
+
+    # Verify nested structure is populated
+    assert "modules" in bus_data["health"], "bus.health should have 'modules'"
+    assert "modules" in bus_data["hud"], "bus.hud should have 'modules'"
+    assert "voice" in bus_data["health"]["modules"], "bus health modules should include 'voice'"
+    assert "voice" in bus_data["hud"]["modules"], "bus hud modules should include 'voice'"
+
+
+def test_http_api_imports_bus():
+    """http_api.py can import the bus from server without circular import errors."""
+    # This import itself is the test: http_api does
+    #   from server import _bus as _shared_bus
+    # If there's a circular import, this will raise ImportError.
+    # http_api also imports engine which may not be available, so we
+    # handle that gracefully.
+    try:
+        from http_api import _bus as http_bus
+    except ImportError as e:
+        # If engine or another heavy dep is missing, that's OK for this test
+        # as long as it's not a circular import error.
+        if "circular" in str(e).lower():
+            pytest.fail(f"Circular import detected: {e}")
+        pytest.skip(f"http_api import failed (likely missing dep): {e}")
+    except Exception as e:
+        # Some deps (engine, sounddevice, etc.) may fail on import.
+        # The key assertion is that it doesn't fail due to circular imports
+        # between server.py and http_api.py.
+        if "circular" in str(e).lower():
+            pytest.fail(f"Circular import detected: {e}")
+        pytest.skip(f"http_api import failed (non-circular): {e}")
+
+    from bus import ModalityBus
+
+    assert isinstance(http_bus, ModalityBus), (
+        f"http_api._bus should be a ModalityBus, got {type(http_bus).__name__}"
+    )
