[Feat] Add visual Sokoban mode

CHANGELOG.md

@@ -14,6 +14,10 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); ver
 - Public231 decision-model benchmark with raw HTTP evidence, pinned upstream scoring,
   complete-denominator quality metrics and per-round service latency.
 
+- Sokoban `--visual`: the same levels, loop and scoring with the board rendered by Valen's own renderer — one
+  PNG per decision, the theme and tile size each level carries, and Valen's instruction and direction texts in
+  the level's language (`en`/`zh`). The ASCII board never rides with the picture and stays out of the episode
+  records; the `visual` extra (Pillow) is needed, text-board runs stay dependency-free.
 - Optional local Cua-S1 4B inference through the `cua-four-b` extra and `CUA_S1_VARIANT=4b`, supporting text and
   multimodal input. Desktop `--pixel-target` offers named screenshot points; clicks remain bound to the observed
   window and capture.

CONTRIBUTING.md

@@ -250,13 +250,14 @@ Everything outside `openjiuwen` is an extra. An agent whose extra is missing say
 | `alfworld` | alfworld, textworld | `s1a run alfworld`; also `ALFWORLD_DATA` and Python 3.11, see `evals/README.md` |
 | `alfworld-visual` | the `alfworld` extra, ai2thor 2.1.0, torch | `evals/replay/thor_replay.py`, the AI2-THOR scene behind an ALFWorld trial in the replay page; the 400 MB Unity build downloads on first use |
 | `report` | pillow, playwright | `python -m evals.replay`, the showcase pages and GIFs; `--gif` also needs `uv run playwright install chromium` |
+| `visual` | pillow | `s1a run sokoban --visual`: the board renderer; text-board Sokoban stays dependency-free |
 | `laya` | laya (torch, transformers) | `--model laya` on every agent and on `decide` and `probe`: Laya in process, no Jev key; the checkpoint downloads into the Hugging Face cache (`HF_HOME`) on first use |
 | `cua` | cua-s1 (torch), huggingface-hub | `--model cua` on tool and browser agents and on `decide` and `probe`: Cua-S1 Nano in process; the 3 MB checkpoint downloads into the Hugging Face cache (`HF_HOME`) on first use |
 | `omnijev` | torch, torchvision, transformers, peft, accelerate, safetensors | `--model omnijev` on the browser agents: OmniJev in process, no Jev key; the model code is a clone of the OmniJev repository named by `OMNIJEV_REPO`, the checkpoint and base model by `OMNIJEV_CHECKPOINT` and `OMNIJEV_BASE` (`docs/configuration.md`) |
 | `dev` | pytest, pytest-asyncio, ruff, ty, jsonschema, pyyaml, referencing | the test suite, `scripts/smoke.sh` and the lint and type checks; the last three check the served-Laya fixtures against `docs/api/` |
 | `cua-four-b` | the `cua` extra, transformers 5, peft, torchvision, pillow | optional local Cua-S1 4B; set `CUA_S1_VARIANT=4b`, and `CUA_S1_MODALITY=text` or `multimodal` |
 
-`uv sync --all-extras` installs all eight. The CLI runs from a checkout; a wheel install (`uv tool install`,
+`uv sync --all-extras` installs all nine. The CLI runs from a checkout; a wheel install (`uv tool install`,
 `pip install`) is unsupported, because the data folders (`evals/2048`, `evals/millionaire`, `evals/labelled`) sit
 next to the package; the command refuses to start outside a checkout with one line on stderr. Runs, logs and
 results go under the checkout (`runs/`, `evals/results`), or under `S1A_HOME` when that variable names another

NOTICE

@@ -12,3 +12,8 @@ This product includes software developed by third parties:
   evals/2048/style/fonts are Clear Sans by Intel Corporation (Apache-2.0)
 - evals/millionaire/questions.json, fetched at first run: Open Trivia Database
   (https://opentdb.com), CC BY-SA 4.0
+- s1a/agents/_sokoban.py: adapted Valen Sokoban simulator (Liuziyu77/Valen), Apache-2.0.
+  s1a/agents/_sokoban_render.py and the visual-mode instruction texts: adapted Valen Sokoban
+  renderer and dataset texts (Liuziyu77/Valen), Apache-2.0.
+  Sokoban levels and test references: Valen-Team/Valen-Eval-Game, Apache-2.0.
+  Pinned sources and modifications: THIRD_PARTY_LICENSES.md

README.md

@@ -148,7 +148,9 @@ The gates and the templates: [docs/skills.md](docs/skills.md#build-a-system-1-ag
 - `desktop`: computer use, any Windows or macOS app window through [Cua Driver](https://cua.ai/docs/cua-driver).
 - `ticket_router`: 30 labelled support tickets to five queues.
 - `alfworld`: household tasks in text, with the AI2-THOR scene in the replays.
-- `game2048`, `millionaire`, `blackjack`: games with a score per episode.
+- `game2048`, `millionaire`, `blackjack`, `sokoban`: games with a score per episode.
+  Sokoban plays Valen’s selected evaluation levels as text boards or, with `--visual`, as Valen’s rendered
+  boards; see [the protocol](evals/sokoban/README.md).
 - `injection_guard`: a rail that answers one question at a hook of a running agent and fails closed.
 
 These use cases are [application candidates](CONTRIBUTING.md#application-candidates) for the required

THIRD_PARTY_LICENSES.md

@@ -39,3 +39,22 @@ Fonts under `evals/2048/style/fonts` are Clear Sans by Intel Corporation, Apache
 
 `evals/millionaire/questions.json`, fetched on the first Millionaire run, holds questions from the
 [Open Trivia Database](https://opentdb.com), Creative Commons Attribution-ShareAlike 4.0 International.
+
+## Valen Sokoban
+
+`s1a/agents/_sokoban.py` is adapted from
+[Liuziyu77/Valen](https://github.com/Liuziyu77/Valen/blob/c96c4f736c6928d2cb1166bc105a24cebdfcdc40/evaluation/sokoban/env.py),
+commit `c96c4f736c6928d2cb1166bc105a24cebdfcdc40`, Apache-2.0.
+Changes: removed unused hash, inverse-action and clone helpers; applied repository formatting.
+
+`s1a/agents/_sokoban_render.py`, and the visual-mode rules and direction names in
+`s1a/agents/sokoban.py`, are adapted from the same commit's
+[render.py](https://github.com/Liuziyu77/Valen/blob/c96c4f736c6928d2cb1166bc105a24cebdfcdc40/evaluation/sokoban/render.py)
+and [dataset.py](https://github.com/Liuziyu77/Valen/blob/c96c4f736c6928d2cb1166bc105a24cebdfcdc40/evaluation/sokoban/dataset.py),
+Apache-2.0. The renderer's drawing logic is unchanged, formatted; the rules and direction texts are verbatim.
+
+`s1a/agents/_data/sokoban_levels.jsonl` and `tests/fixtures/sokoban_reference.jsonl` are unchanged copies of
+`eval_sokoban/levels.jsonl` and `eval_sokoban/reference.jsonl` from
+[Valen-Team/Valen-Eval-Game](https://huggingface.co/datasets/Valen-Team/Valen-Eval-Game/tree/4f25038cd30d9a96aa377eea176a4d2e1680a35c),
+revision `4f25038cd30d9a96aa377eea176a4d2e1680a35c`, declared Apache-2.0.
+The full Apache-2.0 license text is this repository's `LICENSE`.

docs/agents.md

@@ -14,6 +14,7 @@ hook of a running agent. The injection guard rail fails closed: a decision error
 | `game2048` | tool | score and largest tile at a move cap | `s1a run game2048 --model jev --rethink on --episodes 10` |
 | `millionaire` | tool | winnings on a 15-question quiz ladder | `s1a run millionaire --model jev --rethink off --episodes 5` |
 | `blackjack` | tool | payoff per hand (RLCard) | `s1a run blackjack --model jev --rethink off --episodes 100` |
+| `sokoban` | tool | solved fraction on 100 selected Valen levels: text boards over `jev`; `--visual` renders for an image-reading model | `s1a run sokoban --model jev --rethink off --episodes 100` |
 | `injection_guard` | rail | precision and recall on a labelled injection set | `s1a run injection_guard` |
 
 Every tool agent takes `--model jev|clm|laya|laya-served|cua|llm|random|rule`, `--rethink on|off`, `--episodes N`, `--seed S`,
@@ -87,3 +88,19 @@ shipped set is 30 tickets in `s1a/agents/_data/ticket_router_eval.jsonl`, each w
 order status; the label is read by the scorer alone. `--dataset` points the agent at a
 JSONL of your own with the same fields, and `--batch-size` caps the tickets per episode. The rules text the model
 reads is `RULES` in `s1a/agents/ticket_router.py`.
+
+## Sokoban
+
+`uv run s1a run sokoban --model jev --rethink off --episodes 100` plays the bundled 100 Valen levels as text
+boards. No extra is needed. `--seed` is the zero-based level offset; the selected range must fit within 100
+levels. The score is 1 for solved and 0 otherwise. Use `--model random` for an offline smoke run; there is no
+rule baseline. Rethink must be off. See [protocol and provenance](../evals/sokoban/README.md).
+
+`--visual` plays the same levels over Valen's own rendering — one PNG per decision, theme and tile size from
+the level, rules and direction labels in the level's language — with the ASCII board left out. Text-only
+backends (`jev`, `clm`, `laya`, `laya-served`, text-modality `cua`, `llm`) are refused before the first
+decision instead of running blind. `cua`'s multimodal 4B takes the picture, but admission is capability,
+not a result: no visual Sokoban run is recorded for it, so the `jev` command above stays the reference
+text-mode run. (`omnijev` is image-capable too, but it is a browser-front choice; `s1a run` cannot
+select it.) `--model random --visual` smokes the rendering offline (`uv run --extra visual`);
+the served Valen backend this observation was made for ships as its own change.

docs/decision-models.md

@@ -39,12 +39,13 @@ decision.choice("pick").key, decision.latency_ms, decision.usage.input_tokens
 ```
 
 `decide_many` is the one place validation happens. It refuses, with `MODEL_SERVICE_CONFIG_ERROR`, an empty
-question set, an image on a text-only model and a question type the backend lacks. It checks every answer:
-a choice must name an offered key, its probabilities must cover exactly the offered keys, lie in [0, 1], sum to 1
-within 0.02 and peak at the key; a noul probability lies in [0, 1] and its confidence defaults to `max(p, 1 - p)`.
-An unusable answer is re-asked with the same request up to `attempts` times (never on a deterministic backend, never
-after a transport error) and then raises `MODEL_CALL_FAILED`. `decide`, `choose` and `ask` are one-question
-shorthands; `warm()` and `close()` open and release the backend.
+question set and a question type the backend lacks; a text-only backend instead reads the observation without its
+images and warns once per instance. It checks every answer: a choice must name an offered key, its probabilities
+must cover exactly the offered keys, lie in [0, 1], sum to 1 within 0.02 and peak at the key; a noul probability
+lies in [0, 1] and its confidence defaults to `max(p, 1 - p)`. An unusable answer is re-asked with the same request
+up to `attempts` times (never on a deterministic backend, never after a transport error) and then raises
+`MODEL_CALL_FAILED`. `decide`, `choose` and `ask` are one-question shorthands; `warm()` and `close()` open
+and release the backend.
 
 ## Backends
 

evals/README.md

@@ -6,6 +6,7 @@ folder per episode). `uv run python -m evals.table evals/results` aggregates the
 
 | eval | measures | run |
 |---|---|---|
+| Sokoban (selected Valen levels) | solved fraction; the command below is the text board — `--visual` is the upstream image observation and takes an image-capable backend (multimodal `cua`; the image-capable `omnijev` is browser-front only, not selectable by `s1a run`); no visual run recorded | `s1a run sokoban --model jev --rethink off --episodes 100` |
 | Blackjack (RLCard) | payoff per hand | `s1a run blackjack --model jev --rethink off --episodes 100` |
 | 2048 (the MIT game, self-hosted) | score and largest tile at a move cap | `s1a run game2048 --model jev --rethink on --episodes 10` |
 | Millionaire (self-hosted quiz, Open Trivia DB) | winnings; the 50:50 lifeline is a candidate the model may pick; six ladders: `--seed` plus `--episodes` stays at or below 6 | `s1a run millionaire --model jev --rethink off --episodes 5` |

evals/sokoban/README.md

@@ -0,0 +1,66 @@
+# Sokoban (text boards and, with --visual, Valen's rendered boards)
+
+```bash
+uv run s1a run sokoban --model random --rethink off --episodes 1 --max-steps 10
+uv run s1a run sokoban --model jev --rethink off --episodes 100
+```
+
+No additional dependency, browser, dataset download or GPU is needed for the environment.
+The model receives an ASCII board and remaining step budget, with the legend in its rules.
+The four directions remain available even when blocked; blocked moves consume a step.
+There is no solver, legal-action mask, undo, rule baseline or rethink assistance.
+
+`--seed` selects the zero-based level offset, with consecutive levels per episode; the
+range must stay within the bundled 100 levels. The episode ends on success or after at
+most 200 actions (`--max-steps` can lower this); `--timeout` also caps wall time.
+Score is 1 for a solved board and 0 otherwise, so `mean_score` is the solved fraction
+among non-error episodes. Always report errors separately. Episode metadata records the
+level ID, text observation mode, pushes and whether the environment step limit was reached.
+The standard episode records include the final board, steps, elapsed time and decisions.
+
+These are Valen's **selected** 100 evaluation levels (35 easy, 65 medium), not an unbiased
+sample. They retained successful 2B RLCD cases before filling the set. Text-mode results use
+symbolic boards and must not be compared as equivalent to Valen's image-only evaluation.
+No learned-model performance has been measured by this import.
+
+## Visual mode
+
+```bash
+uv run --extra visual s1a run sokoban --model random --rethink off --episodes 1 --max-steps 10 --visual
+```
+
+The `visual` extra (Pillow) rides on the command; `uv sync --extra visual` alone would prune
+the other extras, so add it to your usual sync set instead (`uv sync --extra dev --extra report --extra visual`).
+
+`--visual` keeps the levels, episode loop, budgets and scoring untouched and swaps the
+observation: each decision is shown Valen's own rendering of the board, one PNG per move,
+drawn with the `theme` and `tile_size` the level carries, alongside the remaining step
+budget. The ASCII board never accompanies the picture — the preview checkpoint read images —
+and the pictures stay out of the episode records (`observation_mode: visual` marks the mode).
+The rules and the four direction labels follow each level's `language` (`en`/`zh`) and are
+Valen's own instruction texts, verbatim.
+
+A text-only backend — `jev`, `clm`, `laya`, `laya-served`, `cua` in its text modalities, and `llm` (the chat
+model reads tool text only) — is refused before the first decision rather than run blind over a board it
+cannot see; **the jev command at the top of this page is the text-mode run**. The guard admits what can
+actually see the board — `cua`'s multimodal 4B — but admission is capability, not a Sokoban
+result: no visual run is recorded. (`omnijev` is image-capable too, but it is a browser-front choice;
+`s1a run` cannot select it.) `--model random` is the other exception, the offline smoke for
+the rendering; the served Valen worker behind `/v1/systemone` is the backend this observation was made for
+and ships as its own change. Its
+protocol takes one PNG/JPEG data URL as `state.image`, capped at an 8 MiB body, a 4 MiB image
+and 2048 pixels per side — a bundled level at tile 44 renders far below all of these.
+
+This is the observation Valen's preview checkpoint was trained and validated on; the text
+mode above remains the off-distribution control. Measured results belong to the runs recorded
+under `evals/results/`, not to this import.
+
+The simulator and the renderer both come from Valen, not vllm-jev's demo assets. The level
+definitions are bundled without changes; evaluator-only reference solutions live under
+`tests/fixtures` and are never loaded by the agent. Tests replay reference solutions over
+both observation modes to validate transitions, level compatibility and rendering; this is
+not model accuracy.
+
+See [third-party provenance](../../THIRD_PARTY_LICENSES.md#valen-sokoban) for pinned
+code/data revisions and [the upstream dataset card](https://huggingface.co/datasets/Valen-Team/Valen-Eval-Game)
+for the original protocol and selection details.

pyproject.toml

@@ -41,6 +41,7 @@ alfworld-visual = [
     "torchvision>=0.15",
 ]
 report = ["pillow>=10", "playwright>=1.45"]  # evals.replay: pages, GIFs; never imported by a runner
+visual = ["pillow>=10"]  # s1a run sokoban --visual: the board renderer; text mode stays dependency-free
 laya = ["laya>=0.3.9"]  # the in-process decision model behind --model laya; pulls torch and transformers
 cua = [  # Cua-S1 Nano behind --model cua; pinned to the Cua PR that ships the checkpoints (trycua/cua#4023), pulls torch
     "cua-s1 @ git+https://github.com/trycua/cua.git@aea61b6eb97e2d8c0f6f71eb804e5769fe910af4#subdirectory=libs/cua-s1/python",