Add tabular dataset diff by key

README.md

@@ -13,6 +13,7 @@
 
 ## Table of Contents
 
+- [What's new (2026-06-21) — Dataset Diff (Row-Set Change Report)](#whats-new-2026-06-21--dataset-diff-row-set-change-report)
 - [What's new (2026-06-21) — Distribution Drift Detection](#whats-new-2026-06-21--distribution-drift-detection)
 - [What's new (2026-06-21) — Layered Configuration Resolver](#whats-new-2026-06-21--layered-configuration-resolver)
 - [What's new (2026-06-21) — Server-Sent Events (SSE) Client Parser](#whats-new-2026-06-21--server-sent-events-sse-client-parser)
@@ -135,6 +136,12 @@
 
 ---
 
+## What's new (2026-06-21) — Dataset Diff (Row-Set Change Report)
+
+Diff two tabular extracts by key. Full reference: [`docs/source/Eng/doc/new_features/v83_features_doc.rst`](docs/source/Eng/doc/new_features/v83_features_doc.rst).
+
+- **`diff_rows` / `cell_changes` / `summarize_diff`** (`AC_diff_rows`, `AC_cell_changes`): the framework diffed screens/snapshots but had nothing to diff two **tabular** row-sets by key. This keys both sides and reports `{added, removed, changed, unchanged}` (changed carries `{key, old, new}`), expands per-cell `{key, column, old, new}` changes, and counts each bucket. Supports composite keys; last-write-wins on duplicates. Pure-stdlib, deterministic.
+
 ## What's new (2026-06-21) — Distribution Drift Detection
 
 Check whether today's data is shaped like the baseline. Full reference: [`docs/source/Eng/doc/new_features/v82_features_doc.rst`](docs/source/Eng/doc/new_features/v82_features_doc.rst).

README/README_zh-CN.md

@@ -12,6 +12,7 @@
 
 ## 目录
 
+- [本次更新 (2026-06-21) — 数据集差异(数据行变更报告)](#本次更新-2026-06-21--数据集差异数据行变更报告)
 - [本次更新 (2026-06-21) — 分布漂移检测](#本次更新-2026-06-21--分布漂移检测)
 - [本次更新 (2026-06-21) — 分层配置解析器](#本次更新-2026-06-21--分层配置解析器)
 - [本次更新 (2026-06-21) — Server-Sent Events (SSE) 客户端解析器](#本次更新-2026-06-21--server-sent-events-sse-客户端解析器)
@@ -134,6 +135,12 @@
 
 ---
 
+## 本次更新 (2026-06-21) — 数据集差异(数据行变更报告)
+
+按键比对两份表格式提取。完整参考:[`docs/source/Zh/doc/new_features/v83_features_doc.rst`](../docs/source/Zh/doc/new_features/v83_features_doc.rst)。
+
+- **`diff_rows` / `cell_changes` / `summarize_diff`**(`AC_diff_rows`、`AC_cell_changes`):框架能比对画面/快照,但没有任何东西能按键比对两个**表格式**数据行集合。本功能为两侧建键索引并报告 `{added, removed, changed, unchanged}`(changed 带 `{key, old, new}`),展开逐列 `{key, column, old, new}` 变更,并统计每个分类。支持复合键;重复键以最后一行为准。纯标准库、确定。
+
 ## 本次更新 (2026-06-21) — 分布漂移检测
 
 检查今天的数据形状是否与基准一致。完整参考:[`docs/source/Zh/doc/new_features/v82_features_doc.rst`](../docs/source/Zh/doc/new_features/v82_features_doc.rst)。

README/README_zh-TW.md

@@ -12,6 +12,7 @@
 
 ## 目錄
 
+- [本次更新 (2026-06-21) — 資料集差異(資料列變更報告)](#本次更新-2026-06-21--資料集差異資料列變更報告)
 - [本次更新 (2026-06-21) — 分布漂移偵測](#本次更新-2026-06-21--分布漂移偵測)
 - [本次更新 (2026-06-21) — 分層設定解析器](#本次更新-2026-06-21--分層設定解析器)
 - [本次更新 (2026-06-21) — Server-Sent Events (SSE) 用戶端解析器](#本次更新-2026-06-21--server-sent-events-sse-用戶端解析器)
@@ -134,6 +135,12 @@
 
 ---
 
+## 本次更新 (2026-06-21) — 資料集差異(資料列變更報告)
+
+依鍵比對兩份表格式萃取。完整參考:[`docs/source/Zh/doc/new_features/v83_features_doc.rst`](../docs/source/Zh/doc/new_features/v83_features_doc.rst)。
+
+- **`diff_rows` / `cell_changes` / `summarize_diff`**(`AC_diff_rows`、`AC_cell_changes`):框架能比對畫面/快照,但沒有任何東西能依鍵比對兩個**表格式**資料列集合。本功能為兩側建鍵索引並回報 `{added, removed, changed, unchanged}`(changed 帶 `{key, old, new}`),展開逐欄 `{key, column, old, new}` 變更,並統計每個分類。支援複合鍵;重複鍵以最後一列為準。純標準函式庫、具決定性。
+
 ## 本次更新 (2026-06-21) — 分布漂移偵測
 
 檢查今天的資料形狀是否與基準一致。完整參考:[`docs/source/Zh/doc/new_features/v82_features_doc.rst`](../docs/source/Zh/doc/new_features/v82_features_doc.rst)。

docs/source/Eng/doc/new_features/v83_features_doc.rst

@@ -0,0 +1,42 @@
+Dataset Diff (Row-Set Change Report)
+====================================
+
+The framework diffs *screens/snapshots* (``screen_state.diff_snapshots``,
+``diff_screenshots``) but had nothing to diff two **tabular** row-sets by key —
+the standard "what changed between yesterday's and today's extract" report.
+This keys both sides, then reports added / removed / changed / unchanged rows
+and per-cell changes.
+
+Pure standard library; imports no ``PySide6``. Every function is pure (rows in,
+dict/list out), so it is fully deterministic in CI.
+
+Headless API
+------------
+
+.. code-block:: python
+
+    from je_auto_control import diff_rows, cell_changes, summarize_diff, load_rows
+
+    old = load_rows("yesterday.csv")
+    new = load_rows("today.csv")
+
+    diff = diff_rows(old, new, "id")          # or ["region", "id"]
+    summarize_diff(diff)                       # {added, removed, changed, unchanged}
+
+    for change in cell_changes(old, new, "id"):
+        print(change["key"], change["column"], change["old"], "->", change["new"])
+
+``diff_rows`` keys both row-sets and returns ``{added, removed, changed,
+unchanged}``: ``added`` / ``removed`` / ``unchanged`` are row lists, while
+``changed`` holds ``{key, old, new}`` entries (the key is a scalar for a single
+column or a list for a composite key). On duplicate keys the last row wins.
+``cell_changes`` expands the changed rows into ``{key, column, old, new}``
+records. ``summarize_diff`` counts each bucket.
+
+Executor commands
+-----------------
+
+``AC_diff_rows`` returns ``{diff, summary}`` for ``old_rows`` / ``new_rows`` and
+a ``key`` (column name or JSON list). ``AC_cell_changes`` returns
+``{changes}``. Both are exposed as MCP tools (``ac_diff_rows`` /
+``ac_cell_changes``) and as Script Builder commands under **Data**.

docs/source/Eng/eng_index.rst

@@ -105,6 +105,7 @@ Comprehensive guides for all AutoControl features.
    doc/new_features/v80_features_doc
    doc/new_features/v81_features_doc
    doc/new_features/v82_features_doc
+   doc/new_features/v83_features_doc
    doc/ocr_backends/ocr_backends_doc
    doc/observability/observability_doc
    doc/operations_layer/operations_layer_doc

docs/source/Zh/doc/new_features/v83_features_doc.rst

@@ -0,0 +1,37 @@
+資料集差異(資料列變更報告)
+========================
+
+框架能比對*畫面/快照*(``screen_state.diff_snapshots``、``diff_screenshots``),但沒有任何東西能依
+鍵比對兩個**表格式**資料列集合 —— 也就是經典的「今天的萃取相較昨天變了什麼」報告。本功能為兩側建立
+鍵索引,再回報新增 / 刪除 / 變更 / 未變更的資料列以及逐欄變更。
+
+純標準函式庫;不匯入 ``PySide6``。每個函式皆為純函式(輸入列、輸出 dict/list),因此在 CI 中完全
+具決定性。
+
+無頭 API
+--------
+
+.. code-block:: python
+
+    from je_auto_control import diff_rows, cell_changes, summarize_diff, load_rows
+
+    old = load_rows("yesterday.csv")
+    new = load_rows("today.csv")
+
+    diff = diff_rows(old, new, "id")          # 或 ["region", "id"]
+    summarize_diff(diff)                       # {added, removed, changed, unchanged}
+
+    for change in cell_changes(old, new, "id"):
+        print(change["key"], change["column"], change["old"], "->", change["new"])
+
+``diff_rows`` 為兩個資料列集合建立鍵索引,回傳 ``{added, removed, changed, unchanged}``:``added`` /
+``removed`` / ``unchanged`` 是資料列清單,而 ``changed`` 收錄 ``{key, old, new}``(單欄鍵為純量,複合鍵
+為 list)。鍵重複時以最後一列為準。``cell_changes`` 把變更的列展開成 ``{key, column, old, new}`` 記錄。
+``summarize_diff`` 統計每個分類的數量。
+
+執行器命令
+----------
+
+``AC_diff_rows`` 對 ``old_rows`` / ``new_rows`` 與 ``key``(欄名或 JSON 清單)回傳 ``{diff, summary}``。
+``AC_cell_changes`` 回傳 ``{changes}``。兩者皆以 MCP 工具(``ac_diff_rows`` / ``ac_cell_changes``)以及
+Script Builder 中 **Data** 分類下的命令提供。

docs/source/Zh/zh_index.rst

@@ -105,6 +105,7 @@ AutoControl 所有功能的完整使用指南。
    doc/new_features/v80_features_doc
    doc/new_features/v81_features_doc
    doc/new_features/v82_features_doc
+   doc/new_features/v83_features_doc
    doc/ocr_backends/ocr_backends_doc
    doc/observability/observability_doc
    doc/operations_layer/operations_layer_doc

je_auto_control/__init__.py

@@ -163,6 +163,10 @@
 from je_auto_control.utils.data_drift import (
     categorical_drift, detect_drift, ks_two_sample, psi,
 )
+# Tabular row-set diff (CDC-style added/removed/changed by key)
+from je_auto_control.utils.dataset_diff import (
+    cell_changes, diff_rows, summarize_diff,
+)
 # i18n / l10n testing: pseudo-localize, overflow + catalog checks
 from je_auto_control.utils.i18n_test import (
     check_catalog, check_overflow, pseudo_localize, pseudo_localize_catalog,
@@ -828,6 +832,7 @@ def start_autocontrol_gui(*args, **kwargs):
     "extract_fields", "mask_rows", "validate_rows",
     "infer_schema", "profile_rows",
     "categorical_drift", "detect_drift", "ks_two_sample", "psi",
+    "cell_changes", "diff_rows", "summarize_diff",
     "check_catalog", "check_overflow", "pseudo_localize",
     "pseudo_localize_catalog",
     "Checkpoint", "CheckpointStore", "run_resumable",

je_auto_control/gui/script_builder/command_schema.py

@@ -1720,6 +1720,29 @@ def _add_resilience_specs(specs: List[CommandSpec]) -> None:
         ),
         description="Categorical drift: chi-square + total-variation distance.",
     ))
+    specs.append(CommandSpec(
+        "AC_diff_rows", "Data", "Dataset Diff: Rows by Key",
+        fields=(
+            FieldSpec("old_rows", FieldType.STRING,
+                      placeholder='[{"id": 1, "name": "a"}]'),
+            FieldSpec("new_rows", FieldType.STRING,
+                      placeholder='[{"id": 1, "name": "b"}]'),
+            FieldSpec("key", FieldType.STRING,
+                      placeholder='id  (or ["id", "region"])'),
+        ),
+        description="Diff two row-sets by key: added/removed/changed/unchanged.",
+    ))
+    specs.append(CommandSpec(
+        "AC_cell_changes", "Data", "Dataset Diff: Cell Changes",
+        fields=(
+            FieldSpec("old_rows", FieldType.STRING,
+                      placeholder='[{"id": 1, "name": "a"}]'),
+            FieldSpec("new_rows", FieldType.STRING,
+                      placeholder='[{"id": 1, "name": "b"}]'),
+            FieldSpec("key", FieldType.STRING, placeholder="id"),
+        ),
+        description="Per-cell {key, column, old, new} changes between row-sets.",
+    ))
     specs.append(CommandSpec(
         "AC_rate_limit", "Flow", "Rate Limit (Token Bucket)",
         fields=(

je_auto_control/utils/dataset_diff/__init__.py

@@ -0,0 +1,6 @@
+"""Tabular row-set diffing (CDC-style) for AutoControl data checks."""
+from je_auto_control.utils.dataset_diff.dataset_diff import (
+    cell_changes, diff_rows, summarize_diff,
+)
+
+__all__ = ["cell_changes", "diff_rows", "summarize_diff"]

je_auto_control/utils/dataset_diff/dataset_diff.py

@@ -0,0 +1,83 @@
+"""Diff two tabular row-sets by primary key (CDC-style change report).
+
+The framework diffs *screens/snapshots* (``screen_state.diff_snapshots``,
+``diff_screenshots``) but had nothing to diff two **tabular** row-sets by key —
+the standard "what changed between yesterday's and today's extract" report.
+This keys both sides, then reports added / removed / changed / unchanged rows
+and per-cell changes.
+
+Pure standard library; imports no ``PySide6``. Every function is pure (rows in,
+dict/list out) so it is fully deterministic in CI.
+"""
+from typing import Any, Dict, List, Sequence, Tuple, Union
+
+Key = Union[str, Sequence[str]]
+
+
+def _key_columns(key: Key) -> List[str]:
+    return [key] if isinstance(key, str) else list(key)
+
+
+def _key_of(row: Dict[str, Any], columns: Sequence[str]) -> Tuple[Any, ...]:
+    return tuple(row.get(column) for column in columns)
+
+
+def _key_view(key_tuple: Tuple[Any, ...]) -> Any:
+    return key_tuple[0] if len(key_tuple) == 1 else list(key_tuple)
+
+
+def _index(rows: Sequence[Dict[str, Any]],
+           columns: Sequence[str]) -> Dict[Tuple[Any, ...], Dict[str, Any]]:
+    return {_key_of(row, columns): dict(row) for row in rows}
+
+
+def diff_rows(old_rows: Sequence[Dict[str, Any]],
+              new_rows: Sequence[Dict[str, Any]],
+              key: Key) -> Dict[str, List[Any]]:
+    """Diff ``old_rows`` against ``new_rows`` keyed by ``key``.
+
+    Returns ``{added, removed, changed, unchanged}`` where ``added`` /
+    ``removed`` / ``unchanged`` are row lists and ``changed`` holds
+    ``{key, old, new}`` entries. On duplicate keys, the last row wins.
+    """
+    columns = _key_columns(key)
+    old_index = _index(old_rows, columns)
+    new_index = _index(new_rows, columns)
+    added = [row for key_tuple, row in new_index.items()
+             if key_tuple not in old_index]
+    removed = [row for key_tuple, row in old_index.items()
+               if key_tuple not in new_index]
+    changed: List[Dict[str, Any]] = []
+    unchanged: List[Dict[str, Any]] = []
+    for key_tuple, new_row in new_index.items():
+        old_row = old_index.get(key_tuple)
+        if old_row is None:
+            continue
+        if old_row == new_row:
+            unchanged.append(new_row)
+        else:
+            changed.append({"key": _key_view(key_tuple),
+                            "old": old_row, "new": new_row})
+    return {"added": added, "removed": removed,
+            "changed": changed, "unchanged": unchanged}
+
+
+def cell_changes(old_rows: Sequence[Dict[str, Any]],
+                 new_rows: Sequence[Dict[str, Any]],
+                 key: Key) -> List[Dict[str, Any]]:
+    """Return per-cell changes ``{key, column, old, new}`` for changed rows."""
+    changes: List[Dict[str, Any]] = []
+    for entry in diff_rows(old_rows, new_rows, key)["changed"]:
+        old_row, new_row = entry["old"], entry["new"]
+        for column in sorted(set(old_row) | set(new_row), key=str):
+            if old_row.get(column) != new_row.get(column):
+                changes.append({"key": entry["key"], "column": column,
+                                "old": old_row.get(column),
+                                "new": new_row.get(column)})
+    return changes
+
+
+def summarize_diff(diff: Dict[str, List[Any]]) -> Dict[str, int]:
+    """Count each bucket of a :func:`diff_rows` result."""
+    return {part: len(diff.get(part, []))
+            for part in ("added", "removed", "changed", "unchanged")}

je_auto_control/utils/executor/action_executor.py

@@ -3099,6 +3099,32 @@ def _categorical_drift(reference: Any, current: Any) -> Dict[str, Any]:
     return categorical_drift(reference, current)
 
 
+def _coerce_diff_inputs(old_rows: Any, new_rows: Any, key: Any):
+    import json
+    if isinstance(old_rows, str):
+        old_rows = json.loads(old_rows)
+    if isinstance(new_rows, str):
+        new_rows = json.loads(new_rows)
+    if isinstance(key, str) and key.strip().startswith("["):
+        key = json.loads(key)
+    return old_rows, new_rows, key
+
+
+def _diff_rows(old_rows: Any, new_rows: Any, key: Any) -> Dict[str, Any]:
+    """Adapter: diff two row-sets by key into {diff, summary}."""
+    from je_auto_control.utils.dataset_diff import diff_rows, summarize_diff
+    old_rows, new_rows, key = _coerce_diff_inputs(old_rows, new_rows, key)
+    diff = diff_rows(old_rows, new_rows, key)
+    return {"diff": diff, "summary": summarize_diff(diff)}
+
+
+def _cell_changes(old_rows: Any, new_rows: Any, key: Any) -> Dict[str, Any]:
+    """Adapter: per-cell changes between two row-sets keyed by key."""
+    from je_auto_control.utils.dataset_diff import cell_changes
+    old_rows, new_rows, key = _coerce_diff_inputs(old_rows, new_rows, key)
+    return {"changes": cell_changes(old_rows, new_rows, key)}
+
+
 def _percentiles(samples: Any, qs: Any = None) -> Dict[str, Any]:
     """Adapter: exact percentiles of a numeric sample list (or JSON string)."""
     import json
@@ -4187,6 +4213,8 @@ def __init__(self):
             "AC_explain_config": _explain_config,
             "AC_detect_drift": _detect_drift,
             "AC_categorical_drift": _categorical_drift,
+            "AC_diff_rows": _diff_rows,
+            "AC_cell_changes": _cell_changes,
             "AC_unified_diff": _unified_diff,
             "AC_apply_unified": _apply_unified,
             "AC_three_way_merge": _three_way_merge,

je_auto_control/utils/mcp_server/tools/_factories.py

@@ -3343,6 +3343,37 @@ def rate_limit_tools() -> List[MCPTool]:
     ]
 
 
+def dataset_diff_tools() -> List[MCPTool]:
+    rows_schema = {"type": "array", "items": {"type": "object"}}
+    key_schema = {"type": ["string", "array"]}
+    return [
+        MCPTool(
+            name="ac_diff_rows",
+            description=("Diff 'old_rows' against 'new_rows' keyed by 'key' "
+                         "(column name or list). Returns {diff: {added, removed, "
+                         "changed, unchanged}, summary}."),
+            input_schema=schema(
+                {"old_rows": rows_schema, "new_rows": rows_schema,
+                 "key": key_schema},
+                ["old_rows", "new_rows", "key"]),
+            handler=h.diff_rows,
+            annotations=READ_ONLY,
+        ),
+        MCPTool(
+            name="ac_cell_changes",
+            description=("Per-cell changes between 'old_rows' and 'new_rows' "
+                         "keyed by 'key'. Returns {changes: [{key, column, old, "
+                         "new}]}."),
+            input_schema=schema(
+                {"old_rows": rows_schema, "new_rows": rows_schema,
+                 "key": key_schema},
+                ["old_rows", "new_rows", "key"]),
+            handler=h.cell_changes,
+            annotations=READ_ONLY,
+        ),
+    ]
+
+
 def data_drift_tools() -> List[MCPTool]:
     seq_schema = {"type": "array"}
     return [
@@ -5041,6 +5072,7 @@ def media_assert_tools() -> List[MCPTool]:
     slo_tools, percentiles_tools, bulkhead_tools, http_cassette_tools,
     trace_context_tools, data_profile_tools, http_problem_tools, dotenv_tools,
     sse_client_tools, layered_config_tools, data_drift_tools,
+    dataset_diff_tools,
     saga_tools, decision_table_tools, locator_repair_tools,
     pii_text_tools, sarif_tools,
     screen_record_tools,