Merge pull request #255 from Integration-Automation/feat/process-mining-batch

JE-Chen · web-flow · commit c98fe41fcd4b · 2026-06-20T13:56:31.000+08:00
Add task/process mining for automation-candidate discovery
diff --git a/README.md b/README.md
@@ -13,6 +13,7 @@
 
 ## Table of Contents
 
+- [What's new (2026-06-20) — Task / Process Mining (Automation-Candidate Discovery)](#whats-new-2026-06-20--task--process-mining-automation-candidate-discovery)
 - [What's new (2026-06-20) — Stuck-Loop Guard (Agent Loop Progress Detection)](#whats-new-2026-06-20--stuck-loop-guard-agent-loop-progress-detection)
 - [What's new (2026-06-20) — Coordinate-Space Mapping (Model Grid ⇄ Physical Pixels)](#whats-new-2026-06-20--coordinate-space-mapping-model-grid--physical-pixels)
 - [What's new (2026-06-20) — Voice-Command Router](#whats-new-2026-06-20--voice-command-router)
@@ -99,6 +100,12 @@
 
 ---
 
+## What's new (2026-06-20) — Task / Process Mining (Automation-Candidate Discovery)
+
+Discover what to automate from recorded action logs. Full reference: [`docs/source/Eng/doc/new_features/v47_features_doc.rst`](docs/source/Eng/doc/new_features/v47_features_doc.rst).
+
+- **`mine_action_log` / `find_repeated_sequences` / `directly_follows` / `rank_automation_candidates`** (`AC_mine_actions`, `ac_mine_actions`): mines a recorded action log for frequent, repeatable command n-grams, builds a directly-follows graph, and ranks automation candidates by `count × length` — the RPA "task mining" pillar AutoControl recorded data for but never analysed. Pure-stdlib; operates on the existing action-list shape; a candidate that recurs and spans several steps is a strong "extract into a skill" signal.
+
 ## What's new (2026-06-20) — Stuck-Loop Guard (Agent Loop Progress Detection)
 
 Catch agents stuck in no-progress loops. Full reference: [`docs/source/Eng/doc/new_features/v46_features_doc.rst`](docs/source/Eng/doc/new_features/v46_features_doc.rst).
diff --git a/README/README_zh-CN.md b/README/README_zh-CN.md
@@ -12,6 +12,7 @@
 
 ## 目录
 
+- [本次更新 (2026-06-20) — 任务 / 流程挖掘(自动化候选发现)](#本次更新-2026-06-20--任务--流程挖掘自动化候选发现)
 - [本次更新 (2026-06-20) — 卡循环守卫(Agent Loop 进度检测)](#本次更新-2026-06-20--卡循环守卫agent-loop-进度检测)
 - [本次更新 (2026-06-20) — 坐标空间映射(模型网格 ⇄ 物理像素)](#本次更新-2026-06-20--坐标空间映射模型网格--物理像素)
 - [本次更新 (2026-06-20) — 语音指令路由器](#本次更新-2026-06-20--语音指令路由器)
@@ -98,6 +99,12 @@
 
 ---
 
+## 本次更新 (2026-06-20) — 任务 / 流程挖掘(自动化候选发现)
+
+从录制的动作日志发现该自动化什么。完整参考:[`docs/source/Zh/doc/new_features/v47_features_doc.rst`](../docs/source/Zh/doc/new_features/v47_features_doc.rst)。
+
+- **`mine_action_log` / `find_repeated_sequences` / `directly_follows` / `rank_automation_candidates`**(`AC_mine_actions`、`ac_mine_actions`):挖掘录制的动作日志中频繁、可重复的指令 n-gram,建立 directly-follows 图,并依 `count × length` 为自动化候选排名 —— 这是 AutoControl 一直在录数据却从未分析的 RPA「任务挖掘」支柱。纯标准库;作用于既有动作列表结构;一个经常重现且横跨多步的候选,是「抽成 skill」的强烈信号。
+
 ## 本次更新 (2026-06-20) — 卡循环守卫(Agent Loop 进度检测)
 
 捕捉卡在无进展循环的 agent。完整参考:[`docs/source/Zh/doc/new_features/v46_features_doc.rst`](../docs/source/Zh/doc/new_features/v46_features_doc.rst)。
diff --git a/README/README_zh-TW.md b/README/README_zh-TW.md
@@ -12,6 +12,7 @@
 
 ## 目錄
 
+- [本次更新 (2026-06-20) — 任務 / 流程探勘(自動化候選發現)](#本次更新-2026-06-20--任務--流程探勘自動化候選發現)
 - [本次更新 (2026-06-20) — 卡迴圈守衛(Agent Loop 進度偵測)](#本次更新-2026-06-20--卡迴圈守衛agent-loop-進度偵測)
 - [本次更新 (2026-06-20) — 座標空間對映(模型網格 ⇄ 實體像素)](#本次更新-2026-06-20--座標空間對映模型網格--實體像素)
 - [本次更新 (2026-06-20) — 語音指令路由器](#本次更新-2026-06-20--語音指令路由器)
@@ -98,6 +99,12 @@
 
 ---
 
+## 本次更新 (2026-06-20) — 任務 / 流程探勘(自動化候選發現)
+
+從錄製的動作日誌發現該自動化什麼。完整參考:[`docs/source/Zh/doc/new_features/v47_features_doc.rst`](../docs/source/Zh/doc/new_features/v47_features_doc.rst)。
+
+- **`mine_action_log` / `find_repeated_sequences` / `directly_follows` / `rank_automation_candidates`**(`AC_mine_actions`、`ac_mine_actions`):探勘錄製的動作日誌中頻繁、可重複的指令 n-gram,建立 directly-follows 圖,並依 `count × length` 為自動化候選排名 —— 這是 AutoControl 一直在錄資料卻從未分析的 RPA「任務探勘」支柱。純標準函式庫;作用於既有動作清單結構;一個經常重現且橫跨多步的候選,是「抽成 skill」的強烈訊號。
+
 ## 本次更新 (2026-06-20) — 卡迴圈守衛(Agent Loop 進度偵測)
 
 捕捉卡在無進展迴圈的 agent。完整參考:[`docs/source/Zh/doc/new_features/v46_features_doc.rst`](../docs/source/Zh/doc/new_features/v46_features_doc.rst)。
diff --git a/docs/source/Eng/doc/new_features/v47_features_doc.rst b/docs/source/Eng/doc/new_features/v47_features_doc.rst
@@ -0,0 +1,40 @@
+Task / Process Mining (Automation-Candidate Discovery)
+======================================================
+
+Enterprise RPA suites *discover* what to automate by mining recorded desktop
+actions for frequent, repeatable sub-sequences. AutoControl records rich action
+logs but never analysed them; ``mine_action_log`` turns a log into a ranked list
+of automation candidates — it counts repeated command n-grams, builds a
+directly-follows graph, and scores candidates by how often **and** how long each
+repeated run is.
+
+It operates on the project's action-list shape (each step is a ``["AC_name",
+{...}]`` pair or a ``{"command": "AC_name", ...}`` mapping). Pure standard
+library (``collections``); imports no ``PySide6``.
+
+Headless API
+------------
+
+.. code-block:: python
+
+    from je_auto_control import mine_action_log, directly_follows
+
+    report = mine_action_log(recorded_actions, min_len=2, max_len=5, min_count=3)
+    report.total_actions
+    for cand in report.candidates[:5]:        # best first
+        print(cand.pattern.actions, cand.pattern.count, cand.score)
+
+    directly_follows(recorded_actions)        # {(a, b): edge_count} flow graph
+
+``find_repeated_sequences`` returns the raw n-gram :class:`SequencePattern` list;
+``rank_automation_candidates`` scores them (``count × length`` — more and longer
+repeats rank higher). A candidate that recurs often and spans several steps is a
+strong "extract this into a reusable skill" signal.
+
+Executor command
+----------------
+
+``AC_mine_actions`` takes ``actions`` (a list, or a JSON-string list from the
+visual builder) plus ``min_len`` / ``max_len`` / ``min_count`` and returns
+``{total_actions, patterns, candidates}``. The same operation is exposed as the
+MCP tool ``ac_mine_actions`` and as a Script Builder command under **Report**.
diff --git a/docs/source/Eng/eng_index.rst b/docs/source/Eng/eng_index.rst
@@ -69,6 +69,7 @@ Comprehensive guides for all AutoControl features.
    doc/new_features/v44_features_doc
    doc/new_features/v45_features_doc
    doc/new_features/v46_features_doc
+   doc/new_features/v47_features_doc
    doc/ocr_backends/ocr_backends_doc
    doc/observability/observability_doc
    doc/operations_layer/operations_layer_doc
diff --git a/docs/source/Zh/doc/new_features/v47_features_doc.rst b/docs/source/Zh/doc/new_features/v47_features_doc.rst
@@ -0,0 +1,36 @@
+任務 / 流程探勘(自動化候選發現)
+================================
+
+企業 RPA 套件透過探勘錄製的桌面動作中頻繁、可重複的子序列來*發現*該自動化什麼。
+AutoControl 一直錄製豐富的動作日誌卻從未分析;``mine_action_log`` 將日誌轉成一份排序後
+的自動化候選清單 —— 它計數重複的指令 n-gram、建立 directly-follows 圖,並依每段重複執
+行的**次數與長度**為候選評分。
+
+它作用於本專案的動作清單結構(每步為 ``["AC_name", {...}]`` 對或 ``{"command":
+"AC_name", ...}`` 對映)。純標準函式庫(``collections``);不匯入 ``PySide6``。
+
+無頭 API
+--------
+
+.. code-block:: python
+
+    from je_auto_control import mine_action_log, directly_follows
+
+    report = mine_action_log(recorded_actions, min_len=2, max_len=5, min_count=3)
+    report.total_actions
+    for cand in report.candidates[:5]:        # 由高至低
+        print(cand.pattern.actions, cand.pattern.count, cand.score)
+
+    directly_follows(recorded_actions)        # {(a, b): 邊計數} 流程圖
+
+``find_repeated_sequences`` 回傳原始的 n-gram :class:`SequencePattern` 清單;
+``rank_automation_candidates`` 為其評分(``count × length`` —— 越多、越長的重複排名越
+高)。一個經常重現且橫跨多步的候選,是「把它抽成可重用 skill」的強烈訊號。
+
+執行器指令
+----------
+
+``AC_mine_actions`` 接受 ``actions``(清單,或視覺化建構器傳入的 JSON 字串清單)以及
+``min_len`` / ``max_len`` / ``min_count``,並回傳 ``{total_actions, patterns,
+candidates}``。相同操作亦提供為 MCP 工具 ``ac_mine_actions``,以及 Script Builder 中
+**Report** 分類下的指令。
diff --git a/docs/source/Zh/zh_index.rst b/docs/source/Zh/zh_index.rst
@@ -69,6 +69,7 @@ AutoControl 所有功能的完整使用指南。
    doc/new_features/v44_features_doc
    doc/new_features/v45_features_doc
    doc/new_features/v46_features_doc
+   doc/new_features/v47_features_doc
    doc/ocr_backends/ocr_backends_doc
    doc/observability/observability_doc
    doc/operations_layer/operations_layer_doc
diff --git a/je_auto_control/__init__.py b/je_auto_control/__init__.py
@@ -259,6 +259,11 @@
 from je_auto_control.utils.loop_guard import (
     LoopGuard, LoopVerdict, default_loop_guard, digest_result,
 )
+# Task/process mining: automation-candidate discovery from action logs
+from je_auto_control.utils.process_mining import (
+    Candidate, MiningReport, SequencePattern, directly_follows,
+    find_repeated_sequences, mine_action_log, rank_automation_candidates,
+)
 # Background popup/interrupt watchdog (unattended automation)
 from je_auto_control.utils.watchdog import (
     PopupWatchdog, WatchdogRule, default_popup_watchdog,
@@ -715,6 +720,9 @@ def start_autocontrol_gui(*args, **kwargs):
     "VoiceCommand", "VoiceRouter", "default_voice_router",
     "CoordinateSpace", "downscale_png", "normalized_space", "xga_space",
     "LoopGuard", "LoopVerdict", "default_loop_guard", "digest_result",
+    "Candidate", "MiningReport", "SequencePattern", "directly_follows",
+    "find_repeated_sequences", "mine_action_log",
+    "rank_automation_candidates",
     # MCP server
     "AuditLogger", "HttpMCPServer", "MCPContent", "MCPPrompt",
     "MCPPromptArgument", "MCPResource", "MCPServer", "MCPTool",
diff --git a/je_auto_control/gui/script_builder/command_schema.py b/je_auto_control/gui/script_builder/command_schema.py
@@ -1056,6 +1056,17 @@ def _add_misc_specs(specs: List[CommandSpec]) -> None:
         fields=(),
         description="Clear the default loop guard's history.",
     ))
+    specs.append(CommandSpec(
+        "AC_mine_actions", "Report", "Mine Action Log",
+        fields=(
+            FieldSpec("actions", FieldType.STRING,
+                      placeholder='[["AC_click_mouse", {}], ...]'),
+            FieldSpec("min_len", FieldType.INT, optional=True, default=2),
+            FieldSpec("max_len", FieldType.INT, optional=True, default=5),
+            FieldSpec("min_count", FieldType.INT, optional=True, default=3),
+        ),
+        description="Find repeated sequences + rank automation candidates.",
+    ))
     specs.append(CommandSpec(
         "AC_generate_sop", "Report", "Generate SOP Document",
         fields=(
diff --git a/je_auto_control/utils/executor/action_executor.py b/je_auto_control/utils/executor/action_executor.py
@@ -3229,6 +3229,22 @@ def _loop_guard_reset() -> Dict[str, Any]:
     return {"reset": True}
 
 
+def _mine_actions(actions: Any, min_len: int = 2, max_len: int = 5,
+                  min_count: int = 3) -> Dict[str, Any]:
+    """Adapter: mine an action log for repeated, automatable sequences."""
+    from je_auto_control.utils.process_mining import mine_action_log
+    report = mine_action_log(_coerce_list(actions), min_len=min_len,
+                             max_len=max_len, min_count=min_count)
+    return {
+        "total_actions": report.total_actions,
+        "patterns": [{"actions": list(p.actions), "count": p.count}
+                     for p in report.patterns],
+        "candidates": [{"actions": list(c.pattern.actions),
+                        "count": c.pattern.count, "score": c.score}
+                       for c in report.candidates],
+    }
+
+
 class Executor:
     """
     Executor
@@ -3503,6 +3519,7 @@ def __init__(self):
             "AC_to_model": _to_model,
             "AC_loop_guard_observe": _loop_guard_observe,
             "AC_loop_guard_reset": _loop_guard_reset,
+            "AC_mine_actions": _mine_actions,
             "AC_a11y_record_start": _a11y_record_start,
             "AC_a11y_record_stop": _a11y_record_stop,
             "AC_a11y_record_events": _a11y_record_events,
diff --git a/je_auto_control/utils/mcp_server/tools/_factories.py b/je_auto_control/utils/mcp_server/tools/_factories.py
@@ -3101,6 +3101,26 @@ def loop_guard_tools() -> List[MCPTool]:
     ]
 
 
+def process_mining_tools() -> List[MCPTool]:
+    return [
+        MCPTool(
+            name="ac_mine_actions",
+            description=("Mine a recorded 'actions' log for repeated command "
+                         "sub-sequences (n-grams of length min_len..max_len "
+                         "seen >= min_count) and rank automation candidates by "
+                         "count*length. Returns {total_actions, patterns, "
+                         "candidates}."),
+            input_schema=schema(
+                {"actions": {"type": "array"},
+                 "min_len": {"type": "integer"},
+                 "max_len": {"type": "integer"},
+                 "min_count": {"type": "integer"}}, ["actions"]),
+            handler=h.mine_actions,
+            annotations=READ_ONLY,
+        ),
+    ]
+
+
 def unattended_tools() -> List[MCPTool]:
     return [
         MCPTool(
@@ -4163,6 +4183,7 @@ def media_assert_tools() -> List[MCPTool]:
     trajectory_eval_tools, compliance_tools, agent_trace_tools,
     video_report_tools, fuzzy_tools, artifact_store_tools, image_dedup_tools,
     locale_tools, voice_tools, coordinate_space_tools, loop_guard_tools,
+    process_mining_tools,
     screen_record_tools,
     process_and_shell_tools, remote_desktop_tools, gamepad_tools,
     usb_passthrough_tools, assertion_tools, data_source_tools,
diff --git a/je_auto_control/utils/mcp_server/tools/_handlers.py b/je_auto_control/utils/mcp_server/tools/_handlers.py
@@ -1498,6 +1498,20 @@ def loop_guard_reset():
     return {"reset": True}
 
 
+def mine_actions(actions, min_len=2, max_len=5, min_count=3):
+    from je_auto_control.utils.process_mining import mine_action_log
+    report = mine_action_log(actions, min_len=min_len, max_len=max_len,
+                             min_count=min_count)
+    return {
+        "total_actions": report.total_actions,
+        "patterns": [{"actions": list(p.actions), "count": p.count}
+                     for p in report.patterns],
+        "candidates": [{"actions": list(c.pattern.actions),
+                        "count": c.pattern.count, "score": c.score}
+                       for c in report.candidates],
+    }
+
+
 def vlm_locate(description: str,
                screen_region: Optional[List[int]] = None,
                model: Optional[str] = None) -> Optional[List[int]]:
diff --git a/je_auto_control/utils/process_mining/__init__.py b/je_auto_control/utils/process_mining/__init__.py
@@ -0,0 +1,11 @@
+"""Task/process mining: discover automation candidates from action logs."""
+from je_auto_control.utils.process_mining.process_mining import (
+    Candidate, MiningReport, SequencePattern, directly_follows,
+    find_repeated_sequences, mine_action_log, rank_automation_candidates,
+)
+
+__all__ = [
+    "Candidate", "MiningReport", "SequencePattern", "directly_follows",
+    "find_repeated_sequences", "mine_action_log",
+    "rank_automation_candidates",
+]
diff --git a/je_auto_control/utils/process_mining/process_mining.py b/je_auto_control/utils/process_mining/process_mining.py
@@ -0,0 +1,99 @@
+"""Mine a recorded action log for repetitive, automatable sequences.
+
+Enterprise RPA suites *discover* what to automate by mining recorded desktop
+actions for frequent, repeatable sub-sequences. AutoControl records rich action
+logs but never analysed them; this turns a log into a ranked list of automation
+candidates: it counts repeated command n-grams, builds a directly-follows graph,
+and scores candidates by how often and how long the repeated run is.
+
+Operates on the project's action-list shape — each step is either a
+``["AC_name", {...}]`` pair or a ``{"command": "AC_name", ...}`` mapping. Pure
+standard library (``collections``); imports no ``PySide6``.
+"""
+from collections import Counter
+from dataclasses import dataclass
+from typing import Any, Dict, List, Sequence, Tuple
+
+
+@dataclass(frozen=True)
+class SequencePattern:
+    """A repeated command sub-sequence and how often it occurs."""
+
+    actions: Tuple[str, ...]
+    count: int
+
+
+@dataclass(frozen=True)
+class Candidate:
+    """An automation candidate: a pattern with a priority score."""
+
+    pattern: SequencePattern
+    score: int
+
+
+@dataclass(frozen=True)
+class MiningReport:
+    """The result of mining an action log."""
+
+    total_actions: int
+    patterns: List[SequencePattern]
+    candidates: List[Candidate]
+
+
+def _command_name(action: Any) -> str:
+    if isinstance(action, dict):
+        return str(action.get("command", action.get("name", "")))
+    if isinstance(action, (list, tuple)) and action:
+        return str(action[0])
+    return str(action)
+
+
+def _command_names(actions: Sequence[Any]) -> List[str]:
+    return [_command_name(action) for action in actions]
+
+
+def find_repeated_sequences(actions: Sequence[Any], *, min_len: int = 2,
+                            max_len: int = 5, min_count: int = 3
+                            ) -> List[SequencePattern]:
+    """Return command n-grams (``min_len``..``max_len``) seen >= ``min_count``."""
+    names = _command_names(actions)
+    patterns: List[SequencePattern] = []
+    for length in range(min_len, max_len + 1):
+        if length > len(names):
+            break
+        counter: Counter = Counter(
+            tuple(names[i:i + length])
+            for i in range(len(names) - length + 1))
+        patterns.extend(
+            SequencePattern(gram, count)
+            for gram, count in counter.items() if count >= min_count)
+    patterns.sort(key=lambda p: (p.count * len(p.actions)), reverse=True)
+    return patterns
+
+
+def directly_follows(actions: Sequence[Any]) -> Dict[Tuple[str, str], int]:
+    """Return the directly-follows edge counts of the command flow."""
+    names = _command_names(actions)
+    edges: Counter = Counter(
+        (names[i], names[i + 1]) for i in range(len(names) - 1))
+    return dict(edges)
+
+
+def rank_automation_candidates(report: MiningReport) -> List[Candidate]:
+    """Score patterns by ``count * length`` (more/longer repeats rank higher)."""
+    candidates = [
+        Candidate(pattern, pattern.count * len(pattern.actions))
+        for pattern in report.patterns
+    ]
+    candidates.sort(key=lambda c: c.score, reverse=True)
+    return candidates
+
+
+def mine_action_log(actions: Sequence[Any], *, min_len: int = 2,
+                    max_len: int = 5, min_count: int = 3) -> MiningReport:
+    """Mine ``actions`` into a report of patterns and ranked candidates."""
+    patterns = find_repeated_sequences(
+        actions, min_len=min_len, max_len=max_len, min_count=min_count)
+    report = MiningReport(len(actions), patterns, [])
+    return MiningReport(len(actions), patterns,
+                        rank_automation_candidates(report))
diff --git a/test/unit_test/headless/test_process_mining_batch.py b/test/unit_test/headless/test_process_mining_batch.py
