MirrorNeuron System Tests

mn-system-tests coordinates cross-repository validation for the core, SDK, CLI, API, Web UI, installers, current otterdesk-blueprints catalog, live integration flows, and deterministic benchmark fixtures.

Setup

Install the test dependencies from this repository:

.venv/bin/python -m pip install -r requirements.txt

Inspect the runner:

.venv/bin/python test_all.py --help

Suite Tiers

Offline gate, safe for local development and CI:

.venv/bin/python -m pytest contracts benchmarks installer -q
.venv/bin/python test_all.py --fast --skip-core --skip-node

Injected cross-component contracts:

.venv/bin/python -m pytest contracts -q
.venv/bin/python test_all.py --contracts

These tests exercise API, CLI, and SDK code paths with FakeRuntimeClient instead of Redis, Elixir, Docker, or gRPC.

Interface performance benchmarks:

.venv/bin/python test_all.py --performance

This writes results/performance.txt and results/performance.json with API, SDK/gRPC-boundary, CLI, LLM stream parser, and opt-in live probe timings. The report records hardware, software, package, and git metadata with the numbers.

Live integration and e2e tests:

.venv/bin/python test_all.py --integration --live
.venv/bin/python test_all.py --e2e --live

Live tests are gated by RUN_MN_SYSTEM_TESTS=1; the runner sets this when --live is present and can start isolated local Core/API services when needed.

Cluster and stress checks:

MN_BENCHMARK_WORKER_COUNT=100 .venv/bin/python -m pytest integration -k parallel_worker
MN_CLUSTER_REMOTE_HOST=192.168.4.173 .venv/bin/python test_all.py --redis-ha

Two-box Docker overlay checks remain skipped unless MN_CLUSTER_REMOTE_HOST is set and Docker/SSH are available.

Runner Notes

• --changed selects suites related to dirty sibling repositories.
• --blueprints runs the current otterdesk-blueprints/tests catalog checks.
• --benchmarks runs deterministic offline benchmark/report tests.
• --performance runs key interface performance benchmarks and records results/performance.txt.
• --contracts runs fast API/CLI/SDK boundary tests with injected dependencies.
• --redis-ha runs Redis Sentinel smoke tests and accepts two-box host options.
• --results-dir PATH writes runner logs and summaries outside the default results/ directory.

Every test_all.py run writes results/system-tests.txt, results/system-tests.json, runner-driven pytest-*.json summaries, and per-step logs under results/logs/.

Details

• Regression Matrix
• Benchmark README
• Results README
• Testing Guide
• MirrorNeuron Component Guide

Notes

• Installer checks expect this workspace layout.
• Live tests should use isolated MN_GRPC_PORT, MN_API_BASE_URL, and MN_REDIS_NAMESPACE values when run manually.
• Performance benchmarks default to injected offline probes. Set RUN_MN_PERF_LIVE=1 with MN_API_URL or MN_API_BASE_URL, MN_GRPC_TARGET, MN_LLM_BASE_URL plus MN_LLM_MODEL, or MN_WEB_UI_URL to add live endpoint measurements.
• Add a repository-level license before distributing benchmark fixtures or generated reports outside the project.