Add critic_features: per-step critic bundle + rule-based scorer

README/WHATS_NEW_zh-CN.md

@@ -1,5 +1,11 @@
 # 本次更新 — AutoControl
 
+## 本次更新 (2026-06-24) — 逐步评审特征 + 规则式步骤评分
+
+把为代理步骤评分所需的证据打包,并内建规则式评分器。完整参考:[`docs/source/Zh/doc/new_features/v177_features_doc.rst`](../docs/source/Zh/doc/new_features/v177_features_doc.rst)。
+
+- **`build_critic_record` / `score_step_rule_based` / `to_judge_prompt`**(`AC_build_critic_record`、`AC_score_step`):`trajectory_eval` 对整条轨迹评分而无逐步证据;`agent_trace` 发出 span 而非质量;`agent_replay` 保存步骤却不评分。本功能把 `action_effect` + `observation_delta` + `postcondition` 组合成单一逐步记录,接着 `score_step_rule_based` 给出确定性的 `{outcome, process_score, reasons}`(不需模型),`to_judge_prompt` 把它渲染给可选的 LLM-as-judge。纯标准库聚合器;不导入 `PySide6`。
+
 ## 本次更新 (2026-06-24) — 标题与正文分类 + 文档大纲
 
 以高度区分标题与正文,并建立文档大纲。完整参考:[`docs/source/Zh/doc/new_features/v176_features_doc.rst`](../docs/source/Zh/doc/new_features/v176_features_doc.rst)。

README/WHATS_NEW_zh-TW.md

@@ -1,5 +1,11 @@
 # 本次更新 — AutoControl
 
+## 本次更新 (2026-06-24) — 逐步評審特徵 + 規則式步驟評分
+
+把為代理步驟評分所需的證據打包,並內建規則式評分器。完整參考:[`docs/source/Zh/doc/new_features/v177_features_doc.rst`](../docs/source/Zh/doc/new_features/v177_features_doc.rst)。
+
+- **`build_critic_record` / `score_step_rule_based` / `to_judge_prompt`**(`AC_build_critic_record`、`AC_score_step`):`trajectory_eval` 對整條軌跡評分而無逐步證據;`agent_trace` 發出 span 而非品質;`agent_replay` 保存步驟卻不評分。本功能把 `action_effect` + `observation_delta` + `postcondition` 組合成單一逐步記錄,接著 `score_step_rule_based` 給出確定性的 `{outcome, process_score, reasons}`(不需模型),`to_judge_prompt` 把它渲染給可選的 LLM-as-judge。純標準函式庫聚合器;不匯入 `PySide6`。
+
 ## 本次更新 (2026-06-24) — 標題與內文分類 + 文件大綱
 
 以高度區分標題與內文,並建立文件大綱。完整參考:[`docs/source/Zh/doc/new_features/v176_features_doc.rst`](../docs/source/Zh/doc/new_features/v176_features_doc.rst)。

WHATS_NEW.md

@@ -1,5 +1,11 @@
 # What's New — AutoControl
 
+## What's new (2026-06-24) — Per-Step Critic Features + Rule-Based Step Scorer
+
+Bundle the evidence to score an agent step, with a built-in rule-based scorer. Full reference: [`docs/source/Eng/doc/new_features/v177_features_doc.rst`](docs/source/Eng/doc/new_features/v177_features_doc.rst).
+
+- **`build_critic_record` / `score_step_rule_based` / `to_judge_prompt`** (`AC_build_critic_record`, `AC_score_step`): `trajectory_eval` scores a whole trajectory with no per-step evidence; `agent_trace` emits spans not quality; `agent_replay` stores steps but doesn't score. This composes `action_effect` + `observation_delta` + `postcondition` into one per-step record, then `score_step_rule_based` gives a deterministic `{outcome, process_score, reasons}` (no model needed) and `to_judge_prompt` renders it for an optional LLM-as-judge. Pure-stdlib aggregator; no `PySide6`.
+
 ## What's new (2026-06-24) — Heading vs Body Classification + Document Outline
 
 Tell headings from body text by height and build a document outline. Full reference: [`docs/source/Eng/doc/new_features/v176_features_doc.rst`](docs/source/Eng/doc/new_features/v176_features_doc.rst).

docs/source/Eng/doc/new_features/v177_features_doc.rst

@@ -0,0 +1,46 @@
+Per-Step Critic Features + Rule-Based Step Scorer
+=================================================
+
+Scoring an agent's step needs the evidence in one place — what the action was, what changed,
+whether it landed on target, whether the declared postcondition held. ``trajectory_eval``
+scores a *finished whole trajectory* against a static rubric and has no per-step evidence;
+``agent_trace`` emits OTel spans (tokens / latency), not decision quality; ``agent_replay``
+persists ``{obs, action, result}`` but does no scoring. ``critic_features`` is the missing
+per-step layer: it composes ``action_effect`` (did it do anything, where),
+``observation_delta`` (how much changed) and ``postcondition`` (did the expected outcome hold)
+into one compact record, and ships a deterministic rule-based scorer so the feature works fully
+headless — leaving the optional LLM-as-judge to the integrator.
+
+Pure-stdlib; composes existing pure modules; deterministic and unit-testable with no device.
+Imports no ``PySide6``.
+
+Headless API
+------------
+
+.. code-block:: python
+
+    from je_auto_control import (build_critic_record, score_step_rule_based,
+                                 to_judge_prompt)
+
+    record = build_critic_record({"type": "click", "x": 480, "y": 260},
+                                 before_elements, after_elements,
+                                 postcondition={"appears": {"role": "dialog"}})
+    score = score_step_rule_based(record)
+    # {"outcome": True, "process_score": 1.0, "reasons": [...]}
+
+    prompt = to_judge_prompt(record)   # compact text for an LLM-as-judge
+
+``build_critic_record`` returns ``{action, effect, delta_counts}`` plus a ``postcondition``
+report when a spec is given. ``score_step_rule_based`` returns ``{outcome, process_score,
+reasons}`` — ``outcome`` is a binary success (the action did something *and* any postcondition
+held), ``process_score`` is a 0..1 quality from the effect class (halved if the postcondition
+failed). ``to_judge_prompt`` renders the record for an external judge.
+
+Executor commands
+-----------------
+
+``AC_build_critic_record`` (``action`` / ``before`` / ``after`` / ``postcondition`` /
+``radius`` → the record) and ``AC_score_step`` (``record`` → ``{outcome, process_score,
+reasons}``). They are exposed as the MCP tools ``ac_build_critic_record`` / ``ac_score_step``
+(read-only) and as the Script Builder commands **Build Critic Record** / **Score Step
+(rule-based)** under **Native UI**.

docs/source/Eng/eng_index.rst

@@ -199,6 +199,7 @@ Comprehensive guides for all AutoControl features.
    doc/new_features/v174_features_doc
    doc/new_features/v175_features_doc
    doc/new_features/v176_features_doc
+   doc/new_features/v177_features_doc
    doc/ocr_backends/ocr_backends_doc
    doc/observability/observability_doc
    doc/operations_layer/operations_layer_doc

docs/source/Zh/doc/new_features/v177_features_doc.rst

@@ -0,0 +1,41 @@
+逐步評審特徵 + 規則式步驟評分
+==============================
+
+為代理的步驟評分需要把證據集中一處——動作是什麼、變了什麼、是否落在目標、宣告的後置條件
+是否成立。``trajectory_eval`` 對*已完成的整條軌跡*依靜態準則評分,沒有逐步證據;
+``agent_trace`` 發出 OTel span(權杖 / 延遲),而非決策品質;``agent_replay`` 保存
+``{obs, action, result}`` 卻不評分。``critic_features`` 正是缺少的逐步層:它把 ``action_effect``
+(有無效果、落在何處)、``observation_delta``(變了多少)與 ``postcondition``(預期結果是否成立)
+組合成單一精簡記錄,並附上確定性的規則式評分器,使此功能可完整無頭運作——把可選的
+LLM-as-judge 留給整合者。
+
+純標準函式庫;組合既有純模組;確定性、可在無裝置下單元測試。不匯入 ``PySide6``。
+
+無頭 API
+--------
+
+.. code-block:: python
+
+    from je_auto_control import (build_critic_record, score_step_rule_based,
+                                 to_judge_prompt)
+
+    record = build_critic_record({"type": "click", "x": 480, "y": 260},
+                                 before_elements, after_elements,
+                                 postcondition={"appears": {"role": "dialog"}})
+    score = score_step_rule_based(record)
+    # {"outcome": True, "process_score": 1.0, "reasons": [...]}
+
+    prompt = to_judge_prompt(record)   # 給 LLM-as-judge 的精簡文字
+
+``build_critic_record`` 回傳 ``{action, effect, delta_counts}``,並在給定規格時附上
+``postcondition`` 報告。``score_step_rule_based`` 回傳 ``{outcome, process_score, reasons}``
+——``outcome`` 為二元成功(動作有效果*且*任何後置條件成立),``process_score`` 為依效果類別的
+0..1 品質(後置條件失敗時減半)。``to_judge_prompt`` 把記錄渲染給外部評審。
+
+執行器指令
+----------
+
+``AC_build_critic_record``(``action`` / ``before`` / ``after`` / ``postcondition`` /
+``radius`` → 該記錄)與 ``AC_score_step``(``record`` → ``{outcome, process_score, reasons}``)。
+兩者以 MCP 工具 ``ac_build_critic_record`` / ``ac_score_step``(唯讀)及 Script Builder 指令
+**Build Critic Record** / **Score Step (rule-based)**(位於 **Native UI** 分類下)形式提供。

docs/source/Zh/zh_index.rst

@@ -199,6 +199,7 @@ AutoControl 所有功能的完整使用指南。
    doc/new_features/v174_features_doc
    doc/new_features/v175_features_doc
    doc/new_features/v176_features_doc
+   doc/new_features/v177_features_doc
    doc/ocr_backends/ocr_backends_doc
    doc/observability/observability_doc
    doc/operations_layer/operations_layer_doc

je_auto_control/__init__.py

@@ -351,6 +351,10 @@
 from je_auto_control.utils.settle_detector import (
     SettleState, SettleTracker, is_settled, settle_point,
 )
+# Per-step critic feature bundle + rule-based step scorer
+from je_auto_control.utils.critic_features import (
+    build_critic_record, score_step_rule_based, to_judge_prompt,
+)
 # Locate on-screen regions by colour (mask + connected components)
 from je_auto_control.utils.color_region import (
     find_color_region, find_color_regions,
@@ -1318,6 +1322,9 @@ def start_autocontrol_gui(*args, **kwargs):
     "SettleTracker",
     "settle_point",
     "is_settled",
+    "build_critic_record",
+    "score_step_rule_based",
+    "to_judge_prompt",
     "find_color_region",
     "find_color_regions",
     "ssim_compare",

je_auto_control/gui/script_builder/command_schema.py

@@ -3303,6 +3303,29 @@ def _add_set_of_marks_specs(specs: List[CommandSpec]) -> None:
         ),
         description="Index where a churn series first settles (offline settle check).",
     ))
+    specs.append(CommandSpec(
+        "AC_build_critic_record", "Native UI", "Build Critic Record",
+        fields=(
+            FieldSpec("action", FieldType.STRING,
+                      placeholder='{"type":"click","x":50,"y":50}'),
+            FieldSpec("before", FieldType.STRING,
+                      placeholder='[{"role":"button","x":0,"y":0}]'),
+            FieldSpec("after", FieldType.STRING,
+                      placeholder='[{"role":"dialog","x":40,"y":40}]'),
+            FieldSpec("postcondition", FieldType.STRING, optional=True,
+                      placeholder='{"appears":{"role":"dialog"}}'),
+            FieldSpec("radius", FieldType.INT, optional=True, default=64),
+        ),
+        description="Per-step critic evidence (effect + delta + postcondition).",
+    ))
+    specs.append(CommandSpec(
+        "AC_score_step", "Native UI", "Score Step (rule-based)",
+        fields=(
+            FieldSpec("record", FieldType.STRING,
+                      placeholder='{"effect":{"effect":"changed_near_target"}}'),
+        ),
+        description="Rule-based outcome + process score of a critic record.",
+    ))
     specs.append(CommandSpec(
         "AC_consensus_element", "Native UI", "Grounding Consensus Element",
         fields=(

je_auto_control/utils/critic_features/__init__.py

@@ -0,0 +1,6 @@
+"""Per-step critic feature bundle + a rule-based step scorer."""
+from je_auto_control.utils.critic_features.critic_features import (
+    build_critic_record, score_step_rule_based, to_judge_prompt,
+)
+
+__all__ = ["build_critic_record", "score_step_rule_based", "to_judge_prompt"]

je_auto_control/utils/critic_features/critic_features.py

@@ -0,0 +1,79 @@
+"""Per-step critic feature bundle + a rule-based step scorer.
+
+Scoring an agent's step needs the evidence in one place — what the action was, what changed,
+whether it landed on target, whether the declared postcondition held. ``trajectory_eval``
+scores a *finished whole trajectory* against a static rubric and has no per-step evidence;
+``agent_trace`` emits OTel spans (tokens / latency), not decision quality; ``agent_replay``
+persists ``{obs, action, result}`` but does no scoring. ``critic_features`` is the missing
+per-step layer: it composes ``action_effect`` (did it do anything, where), ``observation_delta``
+(how much changed) and ``postcondition`` (did the expected outcome hold) into one compact
+record, and ships a deterministic rule-based scorer so the feature works fully headless —
+leaving the optional LLM-as-judge to the integrator (``to_judge_prompt``).
+
+Pure-stdlib; composes existing pure modules; deterministic and unit-testable with no device.
+Imports no ``PySide6``.
+"""
+from typing import Any, Dict, Optional, Sequence
+
+Element = Dict[str, Any]
+
+_EFFECT_SCORE = {"changed_near_target": 1.0, "changed": 0.6,
+                 "changed_elsewhere": 0.3, "no_op": 0.0}
+
+
+def build_critic_record(action: Any, before: Sequence[Element],
+                        after: Sequence[Element], *,
+                        postcondition: Optional[Dict[str, Any]] = None,
+                        radius: int = 64) -> Dict[str, Any]:
+    """Compose a per-step critic record from the before/after observation + action.
+
+    Returns ``{action, effect, delta_counts}`` and, when a ``postcondition`` spec is
+    given, the ``postcondition`` report — the evidence bundle a step critic scores.
+    """
+    from je_auto_control.utils.action_effect import classify_effect
+    from je_auto_control.utils.observation_delta import delta_index
+    verdict = classify_effect(before, after, action, radius=int(radius)).to_dict()
+    delta = delta_index(before, after)
+    record: Dict[str, Any] = {
+        "action": action, "effect": verdict,
+        "delta_counts": {"added": len(delta["added"]),
+                         "removed": len(delta["removed"]),
+                         "changed": len(delta["changed"]),
+                         "stable": len(delta["stable"])}}
+    if postcondition is not None:
+        from je_auto_control.utils.postcondition import check_postcondition
+        record["postcondition"] = check_postcondition(
+            after, postcondition, before=before).to_dict()
+    return record
+
+
+def score_step_rule_based(record: Dict[str, Any]) -> Dict[str, Any]:
+    """Score a critic record deterministically → ``{outcome, process_score, reasons}``.
+
+    ``outcome`` is a binary success (the action did something *and* any postcondition held);
+    ``process_score`` is a 0..1 quality from the effect class, halved if the postcondition
+    failed.
+    """
+    effect = record["effect"]["effect"]
+    process = _EFFECT_SCORE.get(effect, 0.0)
+    report = record.get("postcondition")
+    postcondition_ok = report["ok"] if report else True
+    reasons = [f"effect={effect}"]
+    if report is not None:
+        reasons.append(f"postcondition={'ok' if postcondition_ok else 'failed'}")
+    return {"outcome": effect != "no_op" and postcondition_ok,
+            "process_score": round(process * (1.0 if postcondition_ok else 0.5), 4),
+            "reasons": reasons}
+
+
+def to_judge_prompt(record: Dict[str, Any]) -> str:
+    """Render a critic record as a compact text block for an LLM-as-judge."""
+    counts = record["delta_counts"]
+    lines = [f"Action: {record['action']}",
+             f"Effect: {record['effect']['effect']} ({record['effect']['reason']})",
+             f"Changed: +{counts['added']} -{counts['removed']} ~{counts['changed']}"]
+    report = record.get("postcondition")
+    if report is not None:
+        lines.append(f"Postcondition ok: {report['ok']} "
+                     f"(failed: {report['failed']})")
+    return "\n".join(lines)

je_auto_control/utils/executor/action_executor.py

@@ -4265,6 +4265,32 @@ def _settle_point(churns: Any, quiet_samples: Any = 3,
     return {"settled": index is not None, "index": index}
 
 
+def _build_critic_record(action: Any, before: Any, after: Any,
+                         postcondition: Any = None, radius: Any = 64) -> Dict[str, Any]:
+    """Adapter: per-step critic feature bundle (effect + delta + postcondition)."""
+    import json
+    from je_auto_control.utils.critic_features import build_critic_record
+    if isinstance(action, str):
+        action = json.loads(action)
+    if isinstance(before, str):
+        before = json.loads(before)
+    if isinstance(after, str):
+        after = json.loads(after)
+    if isinstance(postcondition, str):
+        postcondition = json.loads(postcondition) if postcondition.strip() else None
+    return build_critic_record(action, before, after, postcondition=postcondition,
+                               radius=int(radius))
+
+
+def _score_step(record: Any) -> Dict[str, Any]:
+    """Adapter: rule-based score of a critic record."""
+    import json
+    from je_auto_control.utils.critic_features import score_step_rule_based
+    if isinstance(record, str):
+        record = json.loads(record)
+    return score_step_rule_based(record)
+
+
 def _validate_action(action: Any, screen: Any = None,
                      targets: Any = None) -> Dict[str, Any]:
     """Adapter: validate a coordinate action (bounds + optional snap-to-target)."""
@@ -6157,6 +6183,8 @@ def __init__(self):
             "AC_consensus_point": _consensus_point,
             "AC_consensus_element": _consensus_element,
             "AC_settle_point": _settle_point,
+            "AC_build_critic_record": _build_critic_record,
+            "AC_score_step": _score_step,
             "AC_validate_action": _validate_action,
             "AC_replay_trace": _replay_trace,
             "AC_match_elements": _match_elements,

je_auto_control/utils/mcp_server/tools/_factories.py

@@ -3454,6 +3454,33 @@ def observation_tools() -> List[MCPTool]:
             handler=h.settle_point,
             annotations=READ_ONLY,
         ),
+        MCPTool(
+            name="ac_build_critic_record",
+            description=("Build a per-step critic record from 'action' + 'before' / "
+                         "'after' element lists (+ optional 'postcondition' spec): "
+                         "composes effect / delta-counts / postcondition into "
+                         "{action, effect, delta_counts, postcondition?} — the "
+                         "evidence a step critic scores."),
+            input_schema=schema({
+                "action": {"type": "object"},
+                "before": {"type": "array", "items": {"type": "object"}},
+                "after": {"type": "array", "items": {"type": "object"}},
+                "postcondition": {"type": "object"},
+                "radius": {"type": "integer"}},
+                required=["action", "before", "after"]),
+            handler=h.build_critic_record,
+            annotations=READ_ONLY,
+        ),
+        MCPTool(
+            name="ac_score_step",
+            description=("Rule-based score of a critic 'record' (from "
+                         "ac_build_critic_record): {outcome (binary success), "
+                         "process_score (0..1), reasons}. Deterministic, no model."),
+            input_schema=schema({"record": {"type": "object"}},
+                                required=["record"]),
+            handler=h.score_step,
+            annotations=READ_ONLY,
+        ),
     ]
 
 

je_auto_control/utils/mcp_server/tools/_handlers.py

@@ -2493,6 +2493,16 @@ def settle_point(churns, quiet_samples=3, max_churn=1.0):
     return _settle_point(churns, quiet_samples, max_churn)
 
 
+def build_critic_record(action, before, after, postcondition=None, radius=64):
+    from je_auto_control.utils.executor.action_executor import _build_critic_record
+    return _build_critic_record(action, before, after, postcondition, radius)
+
+
+def score_step(record):
+    from je_auto_control.utils.executor.action_executor import _score_step
+    return _score_step(record)
+
+
 def validate_action(action, screen=None, targets=None):
     from je_auto_control.utils.executor.action_executor import _validate_action
     return _validate_action(action, screen, targets)