TonsOfFun · Oct 21, 2026 · Oct 22, 2026 · Oct 22, 2026 · Oct 22, 2026 · Oct 22, 2026
diff --git a/AGENTS.md b/AGENTS.md
@@ -24,6 +24,8 @@ Keep each private customer's material within its own organization context.
 | Agent concerns/mixins | `lib/active_agent/concerns/` |
 | Agent-as-tool delegation | `lib/active_agent/delegation/` |
 | Rails generators | `lib/generators/active_agent/` |
+| Scenario evaluations (parser, runner, scorer, report, publisher) | `lib/active_agent/evals/` |
+| Command line (`activeagent` executable) | `exe/activeagent`, `lib/active_agent/cli/` |
 | Dashboard engine (`actionagent` gem) | `actionagent/` |
 | Dashboard React source | `actionagent/frontend/` |
 | Test suite | `test/` |
@@ -283,13 +285,23 @@ bin/test test/path/to/test.rb
 # Run tests for a specific provider
 bin/test test/integration/open_ai/
 
+# The command line: standalone (no Rails loaded), then against the engine's dummy app
+bin/test test/cli/cli_test.rb actionagent/test/cli_contract_test.rb
+
 # Run both trees — the framework's test/ and the engine's actionagent/test/
 bundle exec rake test
 ```
 
 The engine's tests live in `actionagent/test/`. Both `bin/test` and the
 Rakefile's `test` task collect the framework and engine suites by default.
 
+A full `bin/test` with no `.env.test` needs the placeholder provider keys CI
+sets (`ANTHROPIC_API_KEY=ANTHROPIC_API_KEY OPEN_AI_API_KEY=OPEN_AI_API_KEY
+OPEN_ROUTER_API_KEY=OPEN_ROUTER_API_KEY`); without them every provider test
+whose client refuses to initialize without a key errors, a few hundred of
+them, before any request is stubbed or replayed. The suite's own exit code
+is the signal: piping it through `tail` hides it.
+
 ### Test Fixtures
 
 - VCR cassettes in `test/fixtures/vcr_cassettes/`
@@ -375,6 +387,65 @@ cycle. (Both gemspecs declare `railties`; that one is not part of the split.)
   (authentication, `current_user_resolver`, `multi_tenant`,
   `table_name_prefix`, `execution_enabled`, sandbox backends, quotas); all
   are optional and unset means single-user self-hosted behaviour.
+- The MCP facade (`Api::MCPController`, `POST <mount>/mcp`, a dashboard API
+  key as the Bearer token) also serves the dashboard's own tools
+  (`Api::MCPDashboardTools`: `evaluations_*`, `evaluation_runs_*`,
+  `scenarios_merge`, `explorations_submit`, `traces_search|get|export`,
+  `catalogs_list|import|preview|export`, `catalog_set_run`,
+  `agents_versions`, `input_requests_*`). A new one is added in four
+  places — `NAMES`, `DEFINITIONS` (its JSON schema),
+  `dispatch_dashboard_tool`, and `DASHBOARD_TOOLS` in
+  `actionagent/test/mcp_dashboard_tools_test.rb` — plus the tools table in
+  `docs/framework/dashboard.md`. A tool reads under the key's owner exactly
+  as the JSON API reads for a signed-in owner (`owned`, `owner_agents`,
+  `owned_traces`), raises `ToolError` for anything the client can correct,
+  and every result is scrubbed of the owner's credentials before it leaves.
+- Every serialized evaluation run (`EvaluationSerializer.run_summary`)
+  carries `run_id`, the handle `GET <mount>/api/evaluation_reports/:run_id`
+  takes: a published run's own, a dashboard run's
+  `dashboard-run-<id>-<created_at>` (`EvaluationReportExport.run_id_for`).
+
+## The command line: the gem's third surface
+
+`exe/activeagent` (`lib/active_agent/cli.rb` and `lib/active_agent/cli/`) is
+the `activeagent` executable: it works with a dashboard over HTTP from a
+terminal or a CI job, and ships in the `activeagent` gem. Read
+`docs/framework/cli.md` first. The shape to keep:
+
+- It loads neither Rails nor the framework: stdlib `OptionParser` and
+  `Net::HTTP` only, plus `lib/active_agent/evals/publisher.rb`, which is
+  standalone too. `test/cli/cli_test.rb` requires the CLI alone, so a
+  `require "active_support"` (or anything else from the framework) under
+  `lib/active_agent/cli/` fails that test.
+- Every command answers through an API-key-authenticated endpoint, never a
+  session route: reads and runs through the facade's dashboard tools
+  (`Client#tool`, the dashboard API key); imports and the report export
+  through the endpoints the framework's own reporters post to
+  (`<mount>/api/evaluation_reports`, `<mount>/api/traces`; `Client#get` and
+  `Client#post`, the ingest key). A dashboard that offers neither exits
+  `EXIT_UNAVAILABLE` (5), never scrapes the UI's session API.
+- A dashboard feature ships with its CLI command, its API-key endpoint and
+  its UI together (#619); the rake tasks in `actionagent/lib/tasks/` stay
+  for in-app and offline use. When a command needs a read the facade lacks,
+  add a dashboard tool (above) rather than a controller of its own.
+- Adding a command: a private `<group>_<name>` method in the group's module
+  (`cli/eval_commands.rb`, `cli/resource_commands.rb`,
+  `cli/trace_commands.rb`, `cli/session_commands.rb`), named in the group's
+  list in `CLI#dispatch`. `parse` handles the options and `--help`, `emit`
+  prints `--json` or yields the table form, `write_output` writes a
+  document to `-o FILE` or standard output, and a failure is
+  `CLI::Error.new(message, exit_code:)`. Server messages reach the terminal
+  only through `Client#sanitize`, which filters both keys out; never print
+  a response body raw.
+- Tests: `test/cli/cli_test.rb` stubs the dashboard with WebMock and runs
+  the CLI in process (`CLI.start(argv, out:, err:, env:, credentials_path:)`
+  returns the exit code); `actionagent/test/cli_contract_test.rb` hands the
+  same requests to the dummy app (`stub_request(...).to_rack(Rails.application)`),
+  so a change to the facade or the collector that breaks the CLI fails
+  there, not on a user's machine.
+- Documentation: `docs/framework/cli.md` is the page (sidebar: Framework →
+  Command Line), with the facade's tools table in
+  `docs/framework/dashboard.md` and the run export in `docs/evals/publication.md`.
 
 ## Common Gotchas
 
@@ -410,6 +481,11 @@ bin/test
 # Lint
 bin/rubocop
 
+# The command line, from this checkout (`activeagent` once the gem is installed)
+exe/activeagent --help
+exe/activeagent login --url http://localhost:3000/activeagents --key aa_...
+exe/activeagent evals run "Support orders" --min-pass-rate 0.8
+
 # Cross-repo: this checkout against a local solid_agent (strict = no skips)
 SOLID_AGENT_PATH=../solid_agent \
   BUNDLE_GEMFILE=gemfiles/solid_agent_main.gemfile \

diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -7,6 +7,53 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+The `activeagent` command line: the gem's third surface beside the dashboard
+and its API, working with a dashboard over HTTP from a terminal or a CI job,
+and the facade tools it needed. A run, a scenario catalog and a trace each
+move between installs as one file. The framework and the dashboard engine
+move to 1.9.4 together.
+
+Upgrading: no migration. The CLI needs `actionagent` 1.9.4 on the dashboard
+for `catalogs export`, `catalogs import --preview`, `agents versions` and
+`traces export` (the facade's new tools), and 1.9.3 for `evals report` (the
+report export); an older dashboard answers those commands with exit code 5.
+
+### Added
+
+- **`activeagent` command line** (`activeagent`, `actionagent`; refs #619).
+  The gem ships an `activeagent` executable (stdlib `OptionParser` and
+  `Net::HTTP`, no new dependencies) that works with a dashboard over HTTP:
+  `login`/`logout` (named profiles in `~/.config/activeagent/credentials.yml`,
+  `0600`; `ACTIVEAGENT_URL`, `ACTIVEAGENT_API_KEY` and
+  `ACTIVEAGENT_INGEST_KEY` override them for CI), `status`,
+  `evals list|runs|run|report|import`, `catalogs list|import|export`
+  (`import --preview` shows what a document would change, `--mode merge`
+  keeps what it leaves out), `agents list|versions` and
+  `traces list|get|export|import`, with `--json` output and distinct exit
+  codes. `evals run --min-pass-rate 0.8` waits for the run and exits 4
+  below the threshold, for CI gating. Reads and runs go through the MCP
+  facade's dashboard tools with a dashboard API key; the imports and
+  `evals report` use the endpoints the framework's own reporters post to,
+  with the ingest key. A run, a catalog and a trace each move between
+  installs as one file: `evals report RUN --format json` writes the
+  version-1 envelope `evals import` posts (`--format html|md` the report
+  page or its Markdown, for a pull request comment), `catalogs export KEY`
+  the canonical document (YAML, or `--format json`) that `catalogs import`
+  reads, and `traces export TRACE` the payload `traces import` sends to
+  trace ingest. See `docs/framework/cli.md`.
+
+- **Four more dashboard tools on the MCP facade** (`actionagent`), the
+  API-key routes the CLI needed: `catalogs_export` (a catalog's canonical
+  document as YAML or JSON, whole or one product or set), `catalogs_preview`
+  (what `catalogs_import` would change, writing nothing), `agents_versions`
+  (an agent's versions with their diffs, as `GET /api/agents/:id/versions`)
+  and `traces_export` (one trace, whole and untruncated, as the payload
+  `POST <mount>/api/traces` takes, so a trace moves between installs).
+  `catalogs_import` takes `mode: replace|merge` as the API does. Every run
+  summary (`GET /api/evaluations`, the facade's `evaluations_get`,
+  `evaluations_run` and `evaluation_runs_get`) now carries `run_id`, the
+  handle `GET <mount>/api/evaluation_reports/:run_id` takes.
+
 ### Changed
 
 - **One badge for every status and message role in the dashboard**

diff --git a/README.md b/README.md
@@ -181,7 +181,12 @@ telemetry:
 The generator mounts the engine at `/activeagents` — open it and every
 generation appears as a trace. See
 [docs/framework/dashboard.md](docs/framework/dashboard.md) for
-authentication, remote ingestion, and multi-tenant mode. The hosted
+authentication, remote ingestion, and multi-tenant mode. The gem's
+`activeagent` command line works with that dashboard from a terminal or a
+CI job — `activeagent evals run "Support orders" --min-pass-rate 0.8` fails
+a build below the threshold, and runs, scenario catalogs and traces move
+between installs as JSON files; see
+[docs/framework/cli.md](docs/framework/cli.md). The hosted
 platform at [activeagents.ai](https://activeagents.ai) runs this same
 engine multi-tenant, adding what a hosted product has to have — accounts,
 plans, billing, quotas and managed sandboxes; every workspace starts with
@@ -198,6 +203,7 @@ a free low-volume trial.
 - **Agent-as-Tool Delegation**: Hand work to sub-agents with declared schemas, cost/latency budgets, and swappable backends
 - **Context Management**: Maintain conversation history across interactions
 - **Structured Output**: Define JSON schemas for predictable responses
+- **Command Line**: `activeagent` runs evaluations and gates CI on a pass rate from a terminal, and moves runs, scenario catalogs and traces between dashboards as JSON
 
 ## Examples
 

diff --git a/actionagent/README.md b/actionagent/README.md
@@ -112,14 +112,22 @@ and `?download=1` an attachment. A
 signed-in session reads it at
 `<mount>/api/evaluations/:id/runs/:run_id/report.json`, the HTML page at
 `report`, and GitHub-flavored Markdown at `report.md`, each downloadable with
-`?download=1`. From the command line:
+`?download=1`. Inside the host app:
 
 ```bash
 bin/rails action_agent:evaluations:export RUN=<run_id> FORMAT=json|html|md [FILE=path]
 bin/rails action_agent:evaluations:export EVALUATION=<id> RUN=<id> FORMAT=md
 bin/rails action_agent:evaluations:import FILE=run.json [ACCOUNT_ID=<id>]
 ```
 
+From any machine with the `activeagent` gem, over HTTP with the ingest key
+(see [Command Line](https://docs.activeagents.ai/framework/cli)):
+
+```bash
+activeagent evals report <run_id> --format json -o run.json   # or html, md
+activeagent evals import run.json
+```
+
 See [Publishing an externally executed evaluation](https://docs.activeagents.ai/evals/publication)
 for the envelope, the run_id a dashboard run exports under, and every response.
 

diff --git a/actionagent/app/controllers/action_agent/api/agents_controller.rb b/actionagent/app/controllers/action_agent/api/agents_controller.rb
@@ -591,23 +591,7 @@ def mcp_server_params
       end
 
       def version_json(version, include_diff: false)
-        json = {
-          id: version.id,
-          version_number: version.version_number,
-          change_summary: version.change_summary,
-          created_by: version.created_by,
-          created_at: version.created_at,
-          is_latest: version.latest?,
-          release: version.release?,
-          release_digest: version.release_digest,
-          revision: version.revision
-        }
-
-        if include_diff && version.previous
-          json[:diff] = version.diff(version.previous)
-        end
-
-        json
+        AgentVersionSerializer.call(version, include_diff: include_diff)
       end
 
       def default_appearance_for(preset)

diff --git a/actionagent/app/controllers/action_agent/api/mcp_controller.rb b/actionagent/app/controllers/action_agent/api/mcp_controller.rb
@@ -195,8 +195,11 @@ def initialize_instructions
           "failures). A run_<slug> call that pauses to ask for input returns its input request ids: " \
           "input_requests_list shows what paused runs are waiting on, and input_requests_answer answers a text or " \
           "choice request. Scenarios kept as a YAML catalog of products and sets (an .activeagents/evals file) are " \
-          "imported with catalogs_import, listed with catalogs_list, and one set is run as an evaluation named " \
-          "catalog/product/set with catalog_set_run (pass project_id to run against a project's running sandbox)."
+          "imported with catalogs_import (catalogs_preview shows what an import would change first), listed with " \
+          "catalogs_list, written back out with catalogs_export, and one set is run as an evaluation named " \
+          "catalog/product/set with catalog_set_run (pass project_id to run against a project's running sandbox). " \
+          "agents_versions lists an agent's versions with what each changed, and traces_export reads one trace " \
+          "whole, as the payload the trace ingest endpoint takes."
       end
 
       MESSAGE_INPUT_SCHEMA = {