Add netsim scenarios

.gitmodules

@@ -1,3 +1,6 @@
 [submodule ".ai/system"]
 	path = .ai/system
 	url = git@github.com:cryptopunkscc/astral-docs.git
+[submodule "netsim/tasks/_lib/astral-py"]
+	path = netsim/tasks/_lib/astral-py
+	url = ssh://git@git.satforge.dev/satforge/astral-py.git

docs/running-as-a-service.md

@@ -0,0 +1,91 @@
+# Running astrald as a service
+
+`astrald` is a long-running daemon. Run it under systemd on Linux.
+
+## Build
+
+```shell
+CGO_ENABLED=0 go build -o /usr/local/bin/astrald      ./cmd/astrald
+CGO_ENABLED=0 go build -o /usr/local/bin/astral-query ./cmd/astral-query
+```
+
+Go >= 1.25.0 is required. astrald uses pure-Go SQLite, so `CGO_ENABLED=0` builds a
+static binary. The `./` prefix is required; `go build .` at the repo root builds
+an empty stub.
+
+## Root directory
+
+astrald stores config, identity, and data under a root directory derived from
+`$HOME`. A systemd service has no `$HOME`. Pass `-root <dir>` to set the root
+explicitly, or set `Environment=HOME=<dir>`. The first start generates the node
+identity — a `secp256k1` key at `<root>/config/node_key` — with no interaction.
+
+## Unit
+
+`/etc/systemd/system/astrald.service`:
+
+```ini
+[Unit]
+Description=astral daemon
+
+[Service]
+ExecStart=/usr/local/bin/astrald -root /var/lib/astrald
+Environment=HOME=/root
+Restart=on-failure
+KillSignal=SIGINT
+
+[Install]
+WantedBy=multi-user.target
+```
+
+`Type=simple` is the systemd default and is omitted. astrald traps `SIGINT`, not
+`SIGTERM`; `KillSignal=SIGINT` makes `systemctl stop` shut it down gracefully.
+
+```shell
+systemctl enable --now astrald
+```
+
+This unit runs astrald as root — the simplest setup. To run it as your own user
+instead, install it as a user service: place the unit at
+`~/.config/systemd/user/astrald.service`, drop `Environment=HOME=` and the `-root`
+flag (config and data then default to `~/.config/astrald` and
+`~/.local/share/astrald`), and run `systemctl --user enable --now astrald`.
+`loginctl enable-linger $USER` keeps it running without an active login session.
+
+## Health check
+
+```shell
+astral-query localnode:.spec
+```
+
+The local API listens on `tcp:127.0.0.1:8625` with anonymous access. `.spec` is a
+built-in, always-available op. Exit code 0 means the node is up.
+
+## Ports
+
+Default transports bind all interfaces.
+
+| Port | Proto | Purpose |
+|---|---|---|
+| 1791 | TCP | node links |
+| 1792 | UDP | KCP transport |
+| 1791 | UDP | UTP transport |
+| 8822 | UDP | `ether` LAN discovery |
+| 8625 | TCP 127.0.0.1 | local apphost API |
+| 8624 | TCP 0.0.0.0 | apphost HTTP API |
+
+## Imaging and snapshots
+
+Which step you take depends on the capture type:
+
+- **Disk image (cold):** stop astrald first for a clean on-disk state; keep the
+  unit enabled so it autostarts on boot.
+- **Live RAM snapshot (e.g. netsim):** leave astrald running so it resumes
+  already-running on restore.
+
+```shell
+systemctl enable astrald
+systemctl stop astrald   # disk image only — skip for a live RAM snapshot
+```
+
+The identity at `<root>/config/node_key` persists across either capture.

netsim/.gitignore

@@ -0,0 +1,2 @@
+__pycache__/
+*.pyc

netsim/README.md

@@ -0,0 +1,155 @@
+# netsim scenarios for astrald
+
+Test scaffolding that drives `netsim` to build and run `astrald` on a simulated
+LAN. It contains no astrald Go source and modifies none.
+
+`netsim` boots Ubuntu 26.04 cloud-image VMs on `10.77.0.0/24` with per-VM NAT. A
+*task* is a host-side script that configures the VMs. A *story* runs a list of
+tasks in one simulation and saves a named *stage*. `lab.story` builds the
+`astrald-lab` stage: two nodes running astrald, with a Qwen Code operator on
+`node1`.
+
+## Layout
+
+```
+netsim/
+  tasks/                               # each task: run.sh (+ verify.sh / verify.py) + README.md
+    install-astrald/                   # build + run astrald as a service on each node
+    configure-astral-agent/            # install the astral-agent skill into the qwen operator
+    bootstrap-user-software-key/       # make node1 a User node, new key            -> one-node
+    import-user-software-key/          # make node1 a User node, existing mnemonic  -> one-node
+    adopt-node/                        # adopt node2 into swarm + register node aliases -> two-nodes
+    object-store/                      # node1 stores an object (--target localnode|node2) -> two-nodes-data[-peer]
+    read-remote-object/                # node1's agent reads node2's object over astral (used by read-remote-peer)
+    expel-node/                        # node1 (User) permanently bans node2 from the swarm -> two-nodes-expel
+    _lib/                              # shared verify library (astralapi.py) + astral-py submodule
+  scenarios/                           # one dir per scenario: <name>.story + README.md (plain-words)
+    lab/                               # null           -> astrald-lab           (fixture)
+    bootstrap-user-software-key/       # astrald-lab    -> one-node              (fixture)
+    import-user-software-key/          # astrald-lab    -> one-node  (alt.)
+    adopt-node/                        # one-node       -> two-nodes             (fixture)
+    object-store/                      # two-nodes      -> two-nodes-data        (store on node1)
+    object-store-peer/                 # two-nodes      -> two-nodes-data-peer   (store on node2)
+    read-remote-peer/                  # two-nodes      -> two-nodes-peer-read   (store on node2, then read it)
+    expel-node/                        # two-nodes      -> two-nodes-expel
+    tor-link/                          # two-nodes      -> two-nodes-tor         (re-link over Tor)
+    nat-punch/                         # two-nodes      -> two-nodes-nat         (NAT hole-punch)
+  link.sh                          # register tasks with netsim (idempotent; re-run anytime)
+  README.md
+```
+
+## Registering tasks
+
+`netsim` discovers tasks only under `~/.local/share/netsim/tasks/`. `link.sh`
+symlinks every task under `tasks/` — each folder containing a `run.sh` — there.
+It is idempotent; re-run it after adding a task. The symlinks leave netsim's
+shipped builtins intact.
+
+```sh
+./netsim/link.sh
+netsim tasks        # confirm: install-astrald is listed as a user task
+```
+
+## Verifier library
+
+The `verify.py` oracles share `tasks/_lib/astralapi.py`, which reaches each
+VM's apphost through the **astral-py** client vendored as a submodule at
+`tasks/_lib/astral-py`. Initialize it once per worktree (`workon.sh` does not
+`--recurse`); a missing submodule fails with a loud `ImportError`:
+
+```sh
+git submodule update --init netsim/tasks/_lib/astral-py
+```
+
+The verifiers fall back to the Go `astral-query` CLI for any op the client can't
+serve, but the submodule must be present for `verify.py` to import.
+
+## Lab
+
+`lab.story` builds the full lab in one simulation: two nodes running astrald and
+a Qwen Code operator on `node1`, equipped with the `astral-agent` skill.
+
+```
+# lab.story — the astrald lab, built in one netsim simulation.
+# Result: a single stage with two nodes running astrald and a Qwen Code
+# operator on node1, equipped with the astral-agent skill.
+add-vm --hostname node1
+add-vm --hostname node2
+install-astrald
+install-qwen-code --vm node1 --create-user
+configure-astral-agent --vm node1
+```
+
+A story is a plain-text file with one `task [args...]` per line, shell-style
+quoting, and `#` for full-line or trailing comments. `netsim story` boots one
+simulation, runs the listed tasks in order in the same VMs, and saves a single
+stage at the end. It stops at the first failing task. Order is significant:
+
+* `add-vm --hostname node1` and `add-vm --hostname node2` use the `add-vm`
+  builtin; they create the two plain Ubuntu VMs on the LAN.
+* `install-astrald` is the [custom task](tasks/install-astrald/README.md); with no
+  `--vm` it installs astrald on every running VM, so on both nodes. It runs
+  `run.sh` then `verify.sh` and fails the story unless astrald builds, starts, and
+  answers `astral-query localnode:.spec` on every node. The service is left
+  enabled and running, so the stage snapshots a live node that resumes
+  already-running on restore.
+* `install-qwen-code --vm node1 --create-user` uses the `install-qwen-code`
+  builtin; it installs the Qwen Code CLI on `node1` and points it at the
+  inference endpoint. The builtin installs for user `tester`, which does not
+  exist on a fresh cloud image, so `--create-user` is required. `node2` stays a
+  plain astrald peer.
+* `configure-astral-agent --vm node1` is a [custom task](tasks/configure-astral-agent/README.md);
+  it installs the `astral-agent` skill into the Qwen Code operator so it can drive
+  astrald from the skill's knowledge. The host must have `SATFORGE_SKILLS_DEPLOY_KEY`
+  set (a deploy key for the private skills repo) — see its README.
+
+Both VMs must exist and run before `install-astrald`, astrald must be present
+before the Qwen Code operator is layered on `node1`, and the operator must exist
+before its skill is configured.
+
+Register the custom tasks once (see [Registering tasks](#registering-tasks)),
+then build the lab:
+
+```sh
+./netsim/link.sh
+export SATFORGE_SKILLS_DEPLOY_KEY=~/.ssh/satforge_skills_deploy   # see tasks/configure-astral-agent
+netsim story --stage null --save astrald-lab netsim/scenarios/lab/lab.story
+```
+
+The result is the stage `astrald-lab`: `node1` and `node2` running astrald, with a
+Qwen Code operator on `node1` equipped with the `astral-agent` skill. Re-enter it
+with `netsim shell --stage astrald-lab`.
+
+## Swarm pipeline
+
+Each post-lab flow is its own scenario under `scenarios/<name>/` — a `<name>.story`
+plus a plain-words `README.md` — layered on the previous stage (its `start`/`save`
+stages are in the story header and README). Intermediate stages stay reusable, so
+you can replay one flow without rebuilding the chain:
+
+```
+astrald-lab ─[bootstrap-user-software-key]→ one-node ─[adopt-node]→ two-nodes ─[object-store]→ two-nodes-data
+```
+
+```sh
+netsim story --stage astrald-lab    --save one-node             netsim/scenarios/bootstrap-user-software-key/bootstrap-user-software-key.story
+netsim story --stage one-node       --save two-nodes            netsim/scenarios/adopt-node/adopt-node.story
+netsim story --stage two-nodes      --save two-nodes-data       netsim/scenarios/object-store/object-store.story
+netsim story --stage two-nodes      --save two-nodes-peer-read  netsim/scenarios/read-remote-peer/read-remote-peer.story
+netsim story --stage two-nodes      --save two-nodes-expel      netsim/scenarios/expel-node/expel-node.story
+```
+
+`expel-node` is a separate branch off `two-nodes`: the User on node1 permanently
+bans node2, so the swarm roster shrinks (node2 drops out of `user.swarm_status`,
+lands in `user.list_expelled`, and the link is torn down). It produces its own
+`two-nodes-expel` stage rather than feeding the data-object chain.
+
+Each story drives the Qwen operator through its `astral-agent` skill, then runs an
+independent `verify.sh`/`verify.py` check — so a story is a pass/fail integration
+test for that flow.
+
+## Scope
+
+The lab stands up two astrald nodes, links them into one User Swarm, stores an
+object on a node, and reads it from a peer across the swarm. Nodes discover each
+other on the shared L2 LAN via UDP 8822 (`ether`/`nearby`).

netsim/link.sh

@@ -0,0 +1,24 @@
+#!/bin/sh
+# link.sh — register every task under tasks/ as a netsim user task.
+# netsim only discovers tasks in ~/.local/share/netsim/tasks/, so symlink each
+# task dir (each folder under tasks/ with a run.sh) there. Idempotent; re-run anytime.
+set -eu
+
+# CDPATH= is an intentional one-shot env prefix for cd, not an assignment
+# shellcheck disable=SC1007
+repo=$(CDPATH= cd -- "$(dirname -- "$0")" && pwd)
+dest="${NETSIM_HOME:-$HOME/.local/share/netsim}/tasks"
+mkdir -p "$dest"
+
+found=0
+# a "task" = any folder under tasks/ that contains a run.sh
+for rs in "$repo"/tasks/*/run.sh; do
+    [ -f "$rs" ] || continue
+    d=$(dirname "$rs")
+    ln -sfn "$d" "$dest/$(basename "$d")"
+    echo "linked $(basename "$d")"
+    found=$((found + 1))
+done
+
+[ "$found" -gt 0 ] || { echo "no tasks (folders with run.sh) found in $repo/tasks" >&2; exit 1; }
+echo "done: $found task(s) registered — run 'netsim tasks' to confirm"

netsim/scenarios/adopt-node/README.md

@@ -0,0 +1,10 @@
+# adopt-node
+
+Joins two nodes into the same user's network so they trust each other.
+
+- **Kind:** fixture · **Family:** foundation
+- **Chain:** `one-node` → `two-nodes`
+- **Steps:** adopt-node
+- **Run:** `netsim story --stage one-node --save two-nodes netsim/scenarios/adopt-node/adopt-node.story`
+
+One node brings a second node into its personal network as a sibling, then verifies both share the same user contract and see each other as linked peers. This is the stable two-node baseline the multi-node scenarios start from.

netsim/scenarios/adopt-node/adopt-node.story

@@ -0,0 +1,4 @@
+# adopt-node.story — adopt node2 into node1's User swarm (symmetric roster).
+# start: one-node   save: two-nodes
+#   netsim story --stage one-node --save two-nodes netsim/scenarios/adopt-node/adopt-node.story
+adopt-node

netsim/scenarios/bootstrap-user-software-key/README.md

@@ -0,0 +1,10 @@
+# bootstrap-user-software-key
+
+Turns a node into a user-controlled node by creating a fresh user identity.
+
+- **Kind:** fixture · **Family:** identity
+- **Chain:** `astrald-lab` → `one-node`
+- **Steps:** bootstrap-user-software-key
+- **Run:** `netsim story --stage astrald-lab --save one-node netsim/scenarios/bootstrap-user-software-key/bootstrap-user-software-key.story`
+
+Creates a user account on the node and activates it with a contract, then confirms the node recognizes the user and accepts user commands. The result is a working user-controlled node that later scenarios build on.

netsim/scenarios/bootstrap-user-software-key/bootstrap-user-software-key.story

@@ -0,0 +1,4 @@
+# bootstrap-user-software-key.story — node1 becomes a User-controlled node.
+# start: astrald-lab   save: one-node
+#   netsim story --stage astrald-lab --save one-node netsim/scenarios/bootstrap-user-software-key/bootstrap-user-software-key.story
+bootstrap-user-software-key

netsim/scenarios/expel-node/README.md

@@ -0,0 +1,10 @@
+# expel-node
+
+Tests permanently banning another node from a shared swarm.
+
+- **Kind:** scenario · **Family:** network
+- **Chain:** `two-nodes` → `two-nodes-expel`
+- **Steps:** expel-node
+- **Run:** `netsim story --stage two-nodes --save two-nodes-expel netsim/scenarios/expel-node/expel-node.story`
+
+One node expels another: it goes on a blocklist and is dropped from the active members, and the test confirms it is blocked and no longer on the roster. This is how a swarm enforces membership and keeps out unwanted nodes.

netsim/scenarios/expel-node/expel-node.story

@@ -0,0 +1,4 @@
+# expel-node.story — node1 (the User) permanently bans node2 from its swarm.
+# start: two-nodes   save: two-nodes-expel
+#   netsim story --stage two-nodes --save two-nodes-expel netsim/scenarios/expel-node/expel-node.story
+expel-node

netsim/scenarios/import-user-software-key/README.md

@@ -0,0 +1,10 @@
+# import-user-software-key
+
+Tests importing an existing user identity into a node from a recovery phrase.
+
+- **Kind:** scenario · **Family:** identity
+- **Chain:** `astrald-lab` → `one-node`
+- **Steps:** import-user-software-key
+- **Run:** `netsim story --stage astrald-lab --save one-node netsim/scenarios/import-user-software-key/import-user-software-key.story`
+
+Takes an existing recovery phrase and uses it to rebuild the keys that make the node a user node with an active contract, then confirms the node identifies as that user. This is the alternative to bootstrap-user-software-key (both yield the one-node state) for the recover-an-identity path.

netsim/scenarios/import-user-software-key/import-user-software-key.story

@@ -0,0 +1,6 @@
+# import-user-software-key.story — make node1 a User node from an EXISTING mnemonic
+# (embedded in the task's prompt.md; alternative to bootstrap-user-software-key).
+# Optional env ASTRAL_USER_ID makes verify assert the derived id.
+# start: astrald-lab   save: one-node
+#   netsim story --stage astrald-lab --save one-node netsim/scenarios/import-user-software-key/import-user-software-key.story
+import-user-software-key

netsim/scenarios/lab/README.md

@@ -0,0 +1,10 @@
+# lab
+
+Builds the shared test lab: two astrald nodes plus an AI operator.
+
+- **Kind:** fixture · **Family:** foundation
+- **Chain:** `null` → `astrald-lab`
+- **Steps:** add-vm · install-astrald · install-qwen-code · configure-astral-agent
+- **Run:** `netsim story --stage null --save astrald-lab netsim/scenarios/lab/lab.story`
+
+Creates two virtual machines and installs astrald on each so they can work together over a network. It also sets up Qwen Code, an AI assistant, on the first machine with a skill for talking to astrald. This baseline is the foundation every other scenario starts from.

netsim/scenarios/lab/lab.story

@@ -0,0 +1,9 @@
+# lab.story — the astrald lab, built in one netsim simulation.
+# start: null   save: astrald-lab
+# Result: a single stage with two nodes running astrald and a Qwen Code
+# operator on node1, equipped with the astral-agent skill.
+add-vm --hostname node1
+add-vm --hostname node2
+install-astrald
+install-qwen-code --vm node1 --create-user
+configure-astral-agent --vm node1

netsim/scenarios/nat-punch/README.md

@@ -0,0 +1,12 @@
+# nat-punch
+
+Two NAT'd peers hole-punch to a direct connection, coordinating over Tor.
+
+- **Kind:** scenario · **Family:** network
+- **Chain:** `two-nodes` → `two-nodes-nat`
+- **Steps:** add-vm · install-astrald · enable-tor · enter-nat · configure-nat-tor · add-reflector · punch-nat
+- **Run:** `netsim story --stage two-nodes --save two-nodes-nat netsim/scenarios/nat-punch/nat-punch.story`
+
+Both nodes are put behind their own NAT so they have no direct path to each other. A public reflector node tells each one its outside address, and they use a Tor link to coordinate a simultaneous connection attempt. On success the pair ends up talking over a direct kcp link instead of relaying through Tor.
+
+> **Status:** works end to end (direct kcp link verified on both peers). The simulated NAT is currently a permissive *full-cone* NAT; a stricter, filtered NAT that a punch must genuinely defeat is under evaluation.