Add rich clipboard (CF_HTML build/parse/get/set)

JE-Chen · JE-Chen · commit 612da2d09cdf · 2026-06-23T08:44:24.000+08:00
diff --git a/README/WHATS_NEW_zh-CN.md b/README/WHATS_NEW_zh-CN.md
@@ -1,5 +1,11 @@
 # 本次更新 — AutoControl
 
+## 本次更新 (2026-06-23) — 丰富剪贴板(HTML / CF_HTML)
+
+把*格式化*的 HTML 复制粘贴到 Word / Outlook。完整参考:[`docs/source/Zh/doc/new_features/v144_features_doc.rst`](../docs/source/Zh/doc/new_features/v144_features_doc.rst)。
+
+- **`build_cf_html` / `parse_cf_html` / `set_clipboard_html` / `get_clipboard_html`**(`AC_set_clipboard_html`、`AC_get_clipboard_html`):基础剪贴板只处理纯文字 + 图像——富文字粘贴需要 `CF_HTML`,其字节偏移标头(`StartHTML`/`EndHTML`/`StartFragment`/`EndFragment`)极易出错。`build_cf_html`/`parse_cf_html` 以纯 Python 计算与还原它(往返测试、多字节 UTF-8 正确);`set/get_clipboard_html` 将其包装于 Win32 剪贴板(含纯文字后备)。字节偏移运算可无头测试;只有 I/O 为 Windows。
+
 ## 本次更新 (2026-06-23) — 可串接 / 可过滤的候选定位器
 
 用链式调用细化已定位的元素:`.within(panel).filter(has_text="Delete").nth(1)`。完整参考:[`docs/source/Zh/doc/new_features/v143_features_doc.rst`](../docs/source/Zh/doc/new_features/v143_features_doc.rst)。
diff --git a/README/WHATS_NEW_zh-TW.md b/README/WHATS_NEW_zh-TW.md
@@ -1,5 +1,11 @@
 # 本次更新 — AutoControl
 
+## 本次更新 (2026-06-23) — 豐富剪貼簿(HTML / CF_HTML)
+
+把*格式化*的 HTML 複製貼上到 Word / Outlook。完整參考:[`docs/source/Zh/doc/new_features/v144_features_doc.rst`](../docs/source/Zh/doc/new_features/v144_features_doc.rst)。
+
+- **`build_cf_html` / `parse_cf_html` / `set_clipboard_html` / `get_clipboard_html`**(`AC_set_clipboard_html`、`AC_get_clipboard_html`):基礎剪貼簿只處理純文字 + 影像——富文字貼上需要 `CF_HTML`,其位元組偏移標頭(`StartHTML`/`EndHTML`/`StartFragment`/`EndFragment`)極易出錯。`build_cf_html`/`parse_cf_html` 以純 Python 計算與還原它(往返測試、多位元組 UTF-8 正確);`set/get_clipboard_html` 將其包裝於 Win32 剪貼簿(含純文字後備)。位元組偏移運算可無頭測試;只有 I/O 為 Windows。
+
 ## 本次更新 (2026-06-23) — 可串接 / 可過濾的候選定位器
 
 用鏈式呼叫細化已定位的元素:`.within(panel).filter(has_text="Delete").nth(1)`。完整參考:[`docs/source/Zh/doc/new_features/v143_features_doc.rst`](../docs/source/Zh/doc/new_features/v143_features_doc.rst)。
diff --git a/WHATS_NEW.md b/WHATS_NEW.md
@@ -1,5 +1,11 @@
 # What's New — AutoControl
 
+## What's new (2026-06-23) — Rich Clipboard (HTML / CF_HTML)
+
+Copy and paste *formatted* HTML into Word / Outlook. Full reference: [`docs/source/Eng/doc/new_features/v144_features_doc.rst`](docs/source/Eng/doc/new_features/v144_features_doc.rst).
+
+- **`build_cf_html` / `parse_cf_html` / `set_clipboard_html` / `get_clipboard_html`** (`AC_set_clipboard_html`, `AC_get_clipboard_html`): the base clipboard handles plain text + image only — rich paste needs `CF_HTML`, whose byte-offset header (`StartHTML`/`EndHTML`/`StartFragment`/`EndFragment`) is famously error-prone. `build_cf_html`/`parse_cf_html` compute and recover it in pure Python (round-trip tested, correct across multi-byte UTF-8); `set/get_clipboard_html` wrap them over the Win32 clipboard (with a plain-text fallback). Byte-offset math is headless-testable; only the I/O is Windows.
+
 ## What's new (2026-06-23) — Composable / Filtered Candidate Locators
 
 Refine located elements with a chain: `.within(panel).filter(has_text="Delete").nth(1)`. Full reference: [`docs/source/Eng/doc/new_features/v143_features_doc.rst`](docs/source/Eng/doc/new_features/v143_features_doc.rst).
diff --git a/docs/source/Eng/doc/new_features/v144_features_doc.rst b/docs/source/Eng/doc/new_features/v144_features_doc.rst
@@ -0,0 +1,44 @@
+Rich Clipboard — HTML (CF_HTML)
+===============================
+
+The base ``clipboard`` module handles plain text (``CF_UNICODETEXT``) and image
+(``CF_DIB``) only. Pasting *formatted* content into Word / Outlook / a rich editor
+needs the ``CF_HTML`` format, whose ``Version / StartHTML / EndHTML / StartFragment /
+EndFragment`` **byte-offset** header is notoriously error-prone to build by hand.
+``build_cf_html`` / ``parse_cf_html`` compute and recover that header in pure Python
+(a fully unit-tested round-trip, correct across multi-byte UTF-8), and
+``set_clipboard_html`` / ``get_clipboard_html`` wrap them over the Win32 clipboard.
+
+The byte-offset math is platform-independent and headless-testable; only the actual
+clipboard I/O is Windows (raising ``RuntimeError`` elsewhere, like the base module).
+Imports no ``PySide6``.
+
+Headless API
+------------
+
+.. code-block:: python
+
+    from je_auto_control import (build_cf_html, parse_cf_html,
+                                 set_clipboard_html, get_clipboard_html)
+
+    set_clipboard_html("<b>Bold</b> and <i>italic</i>",
+                       fragment_plaintext="Bold and italic")   # Windows
+    html = get_clipboard_html()                                 # Windows
+
+    # The pure pieces work anywhere (e.g. to pre-build a payload):
+    payload = build_cf_html("<p>hello</p>")     # bytes, valid CF_HTML
+    assert parse_cf_html(payload) == "<p>hello</p>"
+
+``build_cf_html`` returns valid ``CF_HTML`` UTF-8 bytes whose offsets point exactly at
+the fragment; ``parse_cf_html`` recovers the fragment from bytes or text (preferring
+the comment markers, falling back to the byte offsets). ``set_clipboard_html`` also
+seeds plain text via ``fragment_plaintext`` so apps that ignore HTML still paste
+something.
+
+Executor commands
+-----------------
+
+``AC_set_clipboard_html`` (``html`` / ``fragment_plaintext`` → ``{set, length}``) and
+``AC_get_clipboard_html`` (→ ``{found, html}``). They are exposed as the MCP tools
+``ac_set_clipboard_html`` / ``ac_get_clipboard_html`` and as Script Builder commands
+under **Data**.
diff --git a/docs/source/Eng/eng_index.rst b/docs/source/Eng/eng_index.rst
@@ -166,6 +166,7 @@ Comprehensive guides for all AutoControl features.
    doc/new_features/v141_features_doc
    doc/new_features/v142_features_doc
    doc/new_features/v143_features_doc
+   doc/new_features/v144_features_doc
    doc/ocr_backends/ocr_backends_doc
    doc/observability/observability_doc
    doc/operations_layer/operations_layer_doc
diff --git a/docs/source/Zh/doc/new_features/v144_features_doc.rst b/docs/source/Zh/doc/new_features/v144_features_doc.rst
@@ -0,0 +1,38 @@
+豐富剪貼簿——HTML(CF_HTML)
+=============================
+
+基礎 ``clipboard`` 模組只處理純文字(``CF_UNICODETEXT``)與影像(``CF_DIB``)。將*格式化*內容貼進 Word /
+Outlook / 富文字編輯器需要 ``CF_HTML`` 格式,其 ``Version / StartHTML / EndHTML / StartFragment /
+EndFragment`` **位元組偏移**標頭以手寫極易出錯。``build_cf_html`` / ``parse_cf_html`` 以純 Python 計算與還原
+該標頭(完整單元測試的往返,且在多位元組 UTF-8 下正確),而 ``set_clipboard_html`` / ``get_clipboard_html``
+將其包裝於 Win32 剪貼簿之上。
+
+位元組偏移運算與平台無關且可無頭測試;只有實際的剪貼簿 I/O 為 Windows(在其他平台丟出 ``RuntimeError``,
+與基礎模組一致)。不匯入 ``PySide6``。
+
+無頭 API
+--------
+
+.. code-block:: python
+
+    from je_auto_control import (build_cf_html, parse_cf_html,
+                                 set_clipboard_html, get_clipboard_html)
+
+    set_clipboard_html("<b>Bold</b> and <i>italic</i>",
+                       fragment_plaintext="Bold and italic")   # Windows
+    html = get_clipboard_html()                                 # Windows
+
+    # 純函式在任何平台皆可用(例如預先建立 payload):
+    payload = build_cf_html("<p>hello</p>")     # bytes,有效 CF_HTML
+    assert parse_cf_html(payload) == "<p>hello</p>"
+
+``build_cf_html`` 回傳偏移精確指向片段的有效 ``CF_HTML`` UTF-8 位元組;``parse_cf_html`` 從 bytes 或文字還原片段
+(優先用註解標記,退而用位元組偏移)。``set_clipboard_html`` 也會以 ``fragment_plaintext`` 設定純文字,讓忽略
+HTML 的程式仍能貼上內容。
+
+執行器命令
+----------
+
+``AC_set_clipboard_html``(``html`` / ``fragment_plaintext`` → ``{set, length}``)與 ``AC_get_clipboard_html``
+(→ ``{found, html}``)。它們以 MCP 工具 ``ac_set_clipboard_html`` / ``ac_get_clipboard_html`` 以及 Script
+Builder 中 **Data** 分類下的命令提供。
diff --git a/docs/source/Zh/zh_index.rst b/docs/source/Zh/zh_index.rst
@@ -166,6 +166,7 @@ AutoControl 所有功能的完整使用指南。
    doc/new_features/v141_features_doc
    doc/new_features/v142_features_doc
    doc/new_features/v143_features_doc
+   doc/new_features/v144_features_doc
    doc/ocr_backends/ocr_backends_doc
    doc/observability/observability_doc
    doc/operations_layer/operations_layer_doc
diff --git a/je_auto_control/__init__.py b/je_auto_control/__init__.py
@@ -335,6 +335,10 @@
 )
 # Composable / filtered candidate locators (chained-locator idiom)
 from je_auto_control.utils.locator_chain import Candidates, from_boxes
+# Rich clipboard formats — HTML (CF_HTML) build / parse / get / set
+from je_auto_control.utils.rich_clipboard import (
+    build_cf_html, get_clipboard_html, parse_cf_html, set_clipboard_html,
+)
 # CI workflow annotations (GitHub Actions)
 from je_auto_control.utils.ci_annotations import (
     emit_annotations, format_annotation,
@@ -1187,6 +1191,10 @@ def start_autocontrol_gui(*args, **kwargs):
     "to_be_stable",
     "Candidates",
     "from_boxes",
+    "build_cf_html",
+    "parse_cf_html",
+    "get_clipboard_html",
+    "set_clipboard_html",
     "emit_annotations", "format_annotation",
     "ClipboardHistory", "default_clipboard_history",
     "analyze_heal_log", "heal_stats", "scan_secrets",
diff --git a/je_auto_control/gui/script_builder/command_schema.py b/je_auto_control/gui/script_builder/command_schema.py
@@ -1044,6 +1044,19 @@ def _add_native_control_specs(specs: List[CommandSpec]) -> None:
 
 def _add_misc_specs(specs: List[CommandSpec]) -> None:
     _add_native_control_specs(specs)
+    specs.append(CommandSpec(
+        "AC_set_clipboard_html", "Data", "Set Clipboard HTML",
+        fields=(
+            FieldSpec("html", FieldType.STRING,
+                      placeholder="<b>Bold</b> rich text"),
+            FieldSpec("fragment_plaintext", FieldType.STRING, optional=True),
+        ),
+        description="Put rich HTML on the clipboard (CF_HTML, Windows).",
+    ))
+    specs.append(CommandSpec(
+        "AC_get_clipboard_html", "Data", "Get Clipboard HTML",
+        description="Read the clipboard's HTML fragment (CF_HTML, Windows).",
+    ))
     specs.append(CommandSpec(
         "AC_watchdog_add", "Flow", "Watchdog: Add Popup Rule",
         fields=(
diff --git a/je_auto_control/utils/executor/action_executor.py b/je_auto_control/utils/executor/action_executor.py
@@ -3661,6 +3661,21 @@ def _locate_chain(boxes: Any, ops: Any = None) -> Dict[str, Any]:
             "center": candidates.center()}
 
 
+def _set_clipboard_html(html: str, fragment_plaintext: Any = None
+                        ) -> Dict[str, Any]:
+    """Adapter: put an HTML fragment on the clipboard as CF_HTML (Windows)."""
+    from je_auto_control.utils.rich_clipboard import set_clipboard_html
+    set_clipboard_html(str(html), fragment_plaintext=fragment_plaintext)
+    return {"set": True, "length": len(str(html))}
+
+
+def _get_clipboard_html() -> Dict[str, Any]:
+    """Adapter: read the clipboard's HTML fragment (Windows)."""
+    from je_auto_control.utils.rich_clipboard import get_clipboard_html
+    html = get_clipboard_html()
+    return {"found": html is not None, "html": html}
+
+
 def _with_modifiers(modifiers: Any, actions: Any) -> Dict[str, Any]:
     """Adapter: run nested actions while modifier keys are held down."""
     import json
@@ -5402,6 +5417,8 @@ def __init__(self):
             "AC_find_separators": _find_separators,
             "AC_expect_poll": _expect_poll,
             "AC_locate_chain": _locate_chain,
+            "AC_set_clipboard_html": _set_clipboard_html,
+            "AC_get_clipboard_html": _get_clipboard_html,
             "AC_tile_rect": _tile_rect,
             "AC_grid_rects": _grid_rects,
             "AC_cascade_rects": _cascade_rects,
diff --git a/je_auto_control/utils/mcp_server/tools/_factories.py b/je_auto_control/utils/mcp_server/tools/_factories.py
@@ -3060,6 +3060,32 @@ def locator_chain_tools() -> List[MCPTool]:
     ]
 
 
+def rich_clipboard_tools() -> List[MCPTool]:
+    return [
+        MCPTool(
+            name="ac_set_clipboard_html",
+            description=("Put an HTML fragment on the clipboard as CF_HTML for rich "
+                         "paste into Word / Outlook / rich editors (Windows). "
+                         "'fragment_plaintext' is also set as plain text. Returns "
+                         "{set, length}."),
+            input_schema=schema({
+                "html": {"type": "string"},
+                "fragment_plaintext": {"type": "string"}},
+                required=["html"]),
+            handler=h.set_clipboard_html,
+            annotations=SIDE_EFFECT_ONLY,
+        ),
+        MCPTool(
+            name="ac_get_clipboard_html",
+            description=("Read the clipboard's HTML fragment (CF_HTML, Windows). "
+                         "Returns {found, html}."),
+            input_schema=schema({}, required=[]),
+            handler=h.get_clipboard_html,
+            annotations=READ_ONLY,
+        ),
+    ]
+
+
 def ssim_tools() -> List[MCPTool]:
     return [
         MCPTool(
@@ -6565,7 +6591,7 @@ def media_assert_tools() -> List[MCPTool]:
     window_layout_tools, window_arrange_tools, preprocess_tools,
     monitor_layout_tools, actionability_tools, element_parse_tools,
     hsv_segment_tools, text_regions_tools, edge_lines_tools, expect_poll_tools,
-    locator_chain_tools, plugin_sdk_tools, governance_tools,
+    locator_chain_tools, rich_clipboard_tools, plugin_sdk_tools, governance_tools,
     credential_lease_tools, egress_tools, approval_testing_tools,
     trajectory_eval_tools, compliance_tools, agent_trace_tools,
     video_report_tools, fuzzy_tools, artifact_store_tools, image_dedup_tools,
diff --git a/je_auto_control/utils/mcp_server/tools/_handlers.py b/je_auto_control/utils/mcp_server/tools/_handlers.py
@@ -2237,6 +2237,16 @@ def locate_chain(boxes, ops=None):
     return _locate_chain(boxes, ops)
 
 
+def set_clipboard_html(html, fragment_plaintext=None):
+    from je_auto_control.utils.executor.action_executor import _set_clipboard_html
+    return _set_clipboard_html(html, fragment_plaintext)
+
+
+def get_clipboard_html():
+    from je_auto_control.utils.executor.action_executor import _get_clipboard_html
+    return _get_clipboard_html()
+
+
 def detect_drift(reference, current, threshold=0.25, bins=10):
     from je_auto_control.utils.executor.action_executor import _detect_drift
     return _detect_drift(reference, current, threshold, bins)
diff --git a/je_auto_control/utils/rich_clipboard/__init__.py b/je_auto_control/utils/rich_clipboard/__init__.py
@@ -0,0 +1,7 @@
+"""Rich clipboard formats — HTML (CF_HTML) build / parse / get / set."""
+from je_auto_control.utils.rich_clipboard.rich_clipboard import (
+    build_cf_html, get_clipboard_html, parse_cf_html, set_clipboard_html,
+)
+
+__all__ = ["build_cf_html", "get_clipboard_html", "parse_cf_html",
+           "set_clipboard_html"]
diff --git a/je_auto_control/utils/rich_clipboard/rich_clipboard.py b/je_auto_control/utils/rich_clipboard/rich_clipboard.py
diff --git a/test/unit_test/headless/test_rich_clipboard_batch.py b/test/unit_test/headless/test_rich_clipboard_batch.py
