Add Unicode text entry via clipboard (emoji / CJK)

README/WHATS_NEW_zh-CN.md

@@ -1,5 +1,11 @@
 # 本次更新 — AutoControl
 
+## 本次更新 (2026-06-23) — Unicode 文本输入(emoji / CJK)
+
+输入 `write` 无法处理的任何 Unicode(emoji / CJK / 重音)。完整参考:[`docs/source/Zh/doc/new_features/v122_features_doc.rst`](../docs/source/Zh/doc/new_features/v122_features_doc.rst)。
+
+- **`type_unicode` / `plan_paste` / `unicode_code_units`**(`AC_type_unicode`):`write` 通过虚拟键表输入,对 emoji/CJK/许多重音字会*抛异常*。`type_unicode` 以设定剪贴板再粘贴(`modifier` ctrl/command)可靠地输入任何文本。`unicode_code_units` 将文本拆成 UTF-16 码元(代理对)供 KEYEVENTF_UNICODE 后端使用。纯计划 + 可注入 sink、确定。
+
 ## 本次更新 (2026-06-23) — 等待区域颜色
 
 阻塞直到某颜色填满(或离开)屏幕区域。完整参考:[`docs/source/Zh/doc/new_features/v121_features_doc.rst`](../docs/source/Zh/doc/new_features/v121_features_doc.rst)。

README/WHATS_NEW_zh-TW.md

@@ -1,5 +1,11 @@
 # 本次更新 — AutoControl
 
+## 本次更新 (2026-06-23) — Unicode 文字輸入(emoji / CJK)
+
+輸入 `write` 無法處理的任何 Unicode(emoji / CJK / 重音)。完整參考:[`docs/source/Zh/doc/new_features/v122_features_doc.rst`](../docs/source/Zh/doc/new_features/v122_features_doc.rst)。
+
+- **`type_unicode` / `plan_paste` / `unicode_code_units`**(`AC_type_unicode`):`write` 透過虛擬鍵表輸入,對 emoji/CJK/許多重音字會*拋例外*。`type_unicode` 以設定剪貼簿再貼上(`modifier` ctrl/command)可靠地輸入任何文字。`unicode_code_units` 將文字拆成 UTF-16 碼元(代理對)供 KEYEVENTF_UNICODE 後端使用。純計畫 + 可注入 sink、具決定性。
+
 ## 本次更新 (2026-06-23) — 等待區域顏色
 
 阻塞直到某顏色填滿(或離開)螢幕區域。完整參考:[`docs/source/Zh/doc/new_features/v121_features_doc.rst`](../docs/source/Zh/doc/new_features/v121_features_doc.rst)。

WHATS_NEW.md

@@ -1,5 +1,11 @@
 # What's New — AutoControl
 
+## What's new (2026-06-23) — Unicode Text Entry (Emoji / CJK)
+
+Type any Unicode (emoji / CJK / accented) that `write` can't. Full reference: [`docs/source/Eng/doc/new_features/v122_features_doc.rst`](docs/source/Eng/doc/new_features/v122_features_doc.rst).
+
+- **`type_unicode` / `plan_paste` / `unicode_code_units`** (`AC_type_unicode`): `write` types through the virtual-key table and *raises* on emoji/CJK/many accented chars. `type_unicode` enters any text reliably by setting the clipboard and pasting (`modifier` ctrl/command). `unicode_code_units` splits text into UTF-16 code units (surrogate pairs) for KEYEVENTF_UNICODE backends. Pure-planning + injectable sink, deterministic.
+
 ## What's new (2026-06-23) — Wait for Region Colour
 
 Block until a colour fills (or leaves) a screen region. Full reference: [`docs/source/Eng/doc/new_features/v121_features_doc.rst`](docs/source/Eng/doc/new_features/v121_features_doc.rst).

docs/source/Eng/doc/new_features/v122_features_doc.rst

@@ -0,0 +1,41 @@
+Unicode Text Entry (Emoji / CJK)
+================================
+
+``write`` types through the platform virtual-key table and *raises* on any
+character outside it — emoji, CJK, many accented letters — so non-ASCII text entry
+was impossible through the normal path. The reliable, cross-platform way to enter
+arbitrary Unicode is to put it on the clipboard and paste it.
+
+:func:`plan_paste` builds the deterministic op-plan and :func:`unicode_code_units`
+splits text into UTF-16 code units (for a backend that can do
+``KEYEVENTF_UNICODE``); both are pure and unit-testable. :func:`type_unicode`
+dispatches the paste plan through an injectable ``sink`` so it is tested without
+touching the real clipboard. Imports no ``PySide6``.
+
+Headless API
+------------
+
+.. code-block:: python
+
+    from je_auto_control import type_unicode, plan_paste, unicode_code_units
+
+    type_unicode("café 🚀 値")              # clipboard set + Ctrl+V
+    type_unicode("値", modifier="command")  # macOS
+
+    unicode_code_units("🚀")               # [0xD83D, 0xDE80] (surrogate pair)
+    plan_paste("hi")
+    # [{'op': 'set_clipboard', 'text': 'hi'},
+    #  {'op': 'hotkey', 'keys': ['ctrl', 'v']}]
+
+``type_unicode`` sets the clipboard to the text and sends the paste hotkey
+(``modifier`` defaults to ``"ctrl"``; use ``"command"`` on macOS), so it enters
+*any* text regardless of keyboard layout — emoji, CJK, RTL, accented. It returns
+the dispatched plan plus the UTF-16 code-unit count. ``unicode_code_units`` is
+provided for backends that want to inject code units directly.
+
+Executor commands
+-----------------
+
+``AC_type_unicode`` takes ``text`` plus an optional ``modifier`` and returns
+``{ops, plan, code_units}``. It is exposed as the MCP tool ``ac_type_unicode`` and
+as a Script Builder command under **Keyboard**.

docs/source/Eng/eng_index.rst

@@ -144,6 +144,7 @@ Comprehensive guides for all AutoControl features.
    doc/new_features/v119_features_doc
    doc/new_features/v120_features_doc
    doc/new_features/v121_features_doc
+   doc/new_features/v122_features_doc
    doc/ocr_backends/ocr_backends_doc
    doc/observability/observability_doc
    doc/operations_layer/operations_layer_doc

docs/source/Zh/doc/new_features/v122_features_doc.rst

@@ -0,0 +1,34 @@
+Unicode 文字輸入(emoji / CJK)
+==============================
+
+``write`` 透過平台虛擬鍵表輸入,對任何不在表內的字元會*拋例外*——emoji、CJK、許多重音字母——因此無法以正常
+途徑輸入非 ASCII 文字。可靠的跨平台輸入任意 Unicode 的方法,是將其放上剪貼簿再貼上。
+
+:func:`plan_paste` 建立決定性操作計畫,:func:`unicode_code_units` 將文字拆成 UTF-16 碼元(供能做
+``KEYEVENTF_UNICODE`` 的後端使用);兩者皆為純函式、可單元測試。:func:`type_unicode` 透過可注入的 ``sink``
+派發貼上計畫,因此可在不觸碰真實剪貼簿下測試。不匯入 ``PySide6``。
+
+無頭 API
+--------
+
+.. code-block:: python
+
+    from je_auto_control import type_unicode, plan_paste, unicode_code_units
+
+    type_unicode("café 🚀 値")              # 設定剪貼簿 + Ctrl+V
+    type_unicode("値", modifier="command")  # macOS
+
+    unicode_code_units("🚀")               # [0xD83D, 0xDE80](代理對)
+    plan_paste("hi")
+    # [{'op': 'set_clipboard', 'text': 'hi'},
+    #  {'op': 'hotkey', 'keys': ['ctrl', 'v']}]
+
+``type_unicode`` 將剪貼簿設為該文字並送出貼上熱鍵(``modifier`` 預設 ``"ctrl"``;macOS 用 ``"command"``),
+因此無論鍵盤配置都能輸入*任何*文字——emoji、CJK、RTL、重音。它回傳派發的計畫加上 UTF-16 碼元數。
+``unicode_code_units`` 供想直接注入碼元的後端使用。
+
+執行器命令
+----------
+
+``AC_type_unicode`` 接受 ``text`` 與選用的 ``modifier``,並回傳 ``{ops, plan, code_units}``。它以 MCP 工具
+``ac_type_unicode`` 以及 Script Builder 中 **Keyboard** 分類下的命令提供。

docs/source/Zh/zh_index.rst

@@ -144,6 +144,7 @@ AutoControl 所有功能的完整使用指南。
    doc/new_features/v119_features_doc
    doc/new_features/v120_features_doc
    doc/new_features/v121_features_doc
+   doc/new_features/v122_features_doc
    doc/ocr_backends/ocr_backends_doc
    doc/observability/observability_doc
    doc/operations_layer/operations_layer_doc

je_auto_control/__init__.py

@@ -263,6 +263,10 @@
 from je_auto_control.utils.mouse_relative import (
     move_mouse_relative, relative_target,
 )
+# Type arbitrary Unicode (emoji / CJK) via the clipboard
+from je_auto_control.utils.text_unicode import (
+    plan_paste, type_unicode, unicode_code_units,
+)
 # CI workflow annotations (GitHub Actions)
 from je_auto_control.utils.ci_annotations import (
     emit_annotations, format_annotation,
@@ -1045,6 +1049,9 @@ def start_autocontrol_gui(*args, **kwargs):
     "hold_key",
     "move_mouse_relative",
     "relative_target",
+    "type_unicode",
+    "plan_paste",
+    "unicode_code_units",
     "emit_annotations", "format_annotation",
     "ClipboardHistory", "default_clipboard_history",
     "analyze_heal_log", "heal_stats", "scan_secrets",

je_auto_control/gui/script_builder/command_schema.py

@@ -185,6 +185,15 @@ def _add_keyboard_specs(specs: List[CommandSpec]) -> None:
         ),
         description="Hold a key for a duration, or auto-repeat it at rate_hz.",
     ))
+    specs.append(CommandSpec(
+        "AC_type_unicode", "Keyboard", "Type Unicode (emoji / CJK)",
+        fields=(
+            FieldSpec("text", FieldType.STRING, placeholder="café 🚀 値"),
+            FieldSpec("modifier", FieldType.STRING, optional=True,
+                      default="ctrl", placeholder="ctrl | command"),
+        ),
+        description="Enter any Unicode text via clipboard paste (write can't).",
+    ))
     specs.append(CommandSpec(
         "AC_hotkey", "Keyboard", "Hotkey",
         fields=(

je_auto_control/utils/executor/action_executor.py

@@ -3155,6 +3155,12 @@ def _move_mouse_relative(dx: Any, dy: Any) -> Dict[str, Any]:
     return move_mouse_relative(int(dx), int(dy))
 
 
+def _type_unicode(text: str, modifier: str = "ctrl") -> Dict[str, Any]:
+    """Adapter: enter arbitrary Unicode text via clipboard paste."""
+    from je_auto_control.utils.text_unicode import type_unicode
+    return type_unicode(text, modifier=modifier)
+
+
 def _cas_put(name: str, key: str, value: Any,
              expected_version: Any = None) -> Dict[str, Any]:
     """Adapter: optimistic put into a named versioned store."""
@@ -4855,6 +4861,7 @@ def __init__(self):
             "AC_set_field_text": _set_field_text,
             "AC_hold_key": _hold_key,
             "AC_move_mouse_relative": _move_mouse_relative,
+            "AC_type_unicode": _type_unicode,
             "AC_detect_drift": _detect_drift,
             "AC_categorical_drift": _categorical_drift,
             "AC_diff_rows": _diff_rows,

je_auto_control/utils/mcp_server/tools/_factories.py

@@ -2596,6 +2596,23 @@ def tween_drag_tools() -> List[MCPTool]:
     ]
 
 
+def text_unicode_tools() -> List[MCPTool]:
+    return [
+        MCPTool(
+            name="ac_type_unicode",
+            description=("Enter arbitrary Unicode 'text' (emoji / CJK / accented "
+                         "that the normal type cannot) via clipboard paste. "
+                         "'modifier' is the paste key (ctrl / command). "
+                         "Returns {ops, plan, code_units}."),
+            input_schema=schema(
+                {"text": {"type": "string"}, "modifier": {"type": "string"}},
+                required=["text"]),
+            handler=h.type_unicode,
+            annotations=SIDE_EFFECT_ONLY,
+        ),
+    ]
+
+
 def mouse_relative_tools() -> List[MCPTool]:
     return [
         MCPTool(
@@ -5929,7 +5946,8 @@ def media_assert_tools() -> List[MCPTool]:
     input_macro_tools, resilience_tools,
     ci_annotation_tools, clipboard_history_tools, audit_analysis_tools,
     process_doc_tools, tween_drag_tools, mouse_path_tools, field_entry_tools,
-    key_hold_tools, mouse_relative_tools, plugin_sdk_tools, governance_tools,
+    key_hold_tools, mouse_relative_tools, text_unicode_tools, plugin_sdk_tools,
+    governance_tools,
     credential_lease_tools, egress_tools, approval_testing_tools,
     trajectory_eval_tools, compliance_tools, agent_trace_tools,
     video_report_tools, fuzzy_tools, artifact_store_tools, image_dedup_tools,

je_auto_control/utils/mcp_server/tools/_handlers.py

@@ -2053,6 +2053,11 @@ def move_mouse_relative(dx, dy):
     return _move_mouse_relative(dx, dy)
 
 
+def type_unicode(text, modifier="ctrl"):
+    from je_auto_control.utils.executor.action_executor import _type_unicode
+    return _type_unicode(text, modifier)
+
+
 def detect_drift(reference, current, threshold=0.25, bins=10):
     from je_auto_control.utils.executor.action_executor import _detect_drift
     return _detect_drift(reference, current, threshold, bins)

je_auto_control/utils/text_unicode/__init__.py

@@ -0,0 +1,6 @@
+"""Type arbitrary Unicode (emoji / CJK / accented) via the clipboard."""
+from je_auto_control.utils.text_unicode.text_unicode import (
+    plan_paste, type_unicode, unicode_code_units,
+)
+
+__all__ = ["plan_paste", "type_unicode", "unicode_code_units"]

je_auto_control/utils/text_unicode/text_unicode.py

@@ -0,0 +1,62 @@
+"""Type arbitrary Unicode (emoji / CJK / accented) via the clipboard.
+
+``write`` types through the platform virtual-key table and *raises* on any
+character outside it — emoji, CJK, many accented letters — so non-ASCII text
+entry is impossible through the normal path. The reliable, cross-platform way to
+enter arbitrary Unicode is to put it on the clipboard and paste it.
+
+:func:`plan_paste` builds the deterministic op-plan and :func:`unicode_code_units`
+splits text into UTF-16 code units (for a backend that can do
+``KEYEVENTF_UNICODE``); both are pure and unit-testable. :func:`type_unicode`
+dispatches the paste plan through an injectable ``sink`` so it is tested without
+touching the real clipboard. Imports no ``PySide6``.
+"""
+from typing import Any, Callable, Dict, List, Optional
+
+Sink = Callable[[Dict[str, Any]], None]
+
+
+def unicode_code_units(text: str) -> List[int]:
+    """Return the UTF-16 code units of ``text`` (surrogate pairs for > U+FFFF)."""
+    units: List[int] = []
+    for char in text or "":
+        code = ord(char)
+        if code > 0xFFFF:
+            code -= 0x10000
+            units.append(0xD800 + (code >> 10))
+            units.append(0xDC00 + (code & 0x3FF))
+        else:
+            units.append(code)
+    return units
+
+
+def plan_paste(text: str, *, modifier: str = "ctrl") -> List[Dict[str, Any]]:
+    """Return the op-plan to enter ``text`` via clipboard paste."""
+    return [{"op": "set_clipboard", "text": text},
+            {"op": "hotkey", "keys": [modifier, "v"]}]
+
+
+def _default_sink(event: Dict[str, Any]) -> None:
+    """Default dispatch: drive the real clipboard / keyboard backend."""
+    op = event["op"]
+    if op == "set_clipboard":
+        from je_auto_control.utils.clipboard.clipboard import set_clipboard
+        set_clipboard(event["text"])
+    elif op == "hotkey":
+        from je_auto_control.wrapper.auto_control_keyboard import hotkey
+        hotkey(list(event["keys"]))
+
+
+def type_unicode(text: str, *, modifier: str = "ctrl",
+                 sink: Optional[Sink] = None) -> Dict[str, Any]:
+    """Enter ``text`` (any Unicode) by setting the clipboard then pasting.
+
+    ``modifier`` is the platform paste key (``"ctrl"``; use ``"command"`` on
+    macOS). Returns the dispatched plan plus the UTF-16 code-unit count.
+    """
+    plan = plan_paste(text, modifier=modifier)
+    dispatch = sink or _default_sink
+    for event in plan:
+        dispatch(event)
+    return {"ops": len(plan), "plan": plan,
+            "code_units": len(unicode_code_units(text))}

test/unit_test/headless/test_text_unicode_batch.py

@@ -0,0 +1,58 @@
+"""Headless tests for Unicode text entry via clipboard. No Qt."""
+import je_auto_control as ac
+from je_auto_control.utils.text_unicode import (
+    plan_paste, type_unicode, unicode_code_units,
+)
+
+
+def test_code_units_ascii_and_bmp():
+    assert unicode_code_units("Hi") == [72, 105]
+    assert unicode_code_units("値") == [0x5024]
+    assert unicode_code_units("") == []
+
+
+def test_code_units_astral_surrogate_pair():
+    assert unicode_code_units("🚀") == [0xD83D, 0xDE80]   # rocket > U+FFFF
+    assert len(unicode_code_units("a🚀b")) == 4            # 1 + 2 + 1
+
+
+def test_plan_paste_is_clipboard_then_hotkey():
+    assert plan_paste("café 🚀") == [
+        {"op": "set_clipboard", "text": "café 🚀"},
+        {"op": "hotkey", "keys": ["ctrl", "v"]},
+    ]
+    assert plan_paste("x", modifier="command")[1]["keys"] == ["command", "v"]
+
+
+def test_type_unicode_dispatches_plan():
+    events = []
+    result = type_unicode("café 🚀", sink=events.append)
+    assert [e["op"] for e in events] == ["set_clipboard", "hotkey"]
+    assert events[0]["text"] == "café 🚀"
+    assert result["ops"] == 2 and result["code_units"] == 7
+
+
+# --- wiring ---------------------------------------------------------------
+
+def test_executor_adapter_dispatches_via_sink():
+    # the executor default dispatch is device-bound; verify the planning the
+    # adapter delegates to instead.
+    events = []
+    type_unicode("値", sink=events.append, modifier="ctrl")
+    assert events[-1] == {"op": "hotkey", "keys": ["ctrl", "v"]}
+
+
+def test_wiring():
+    known = ac.executor.known_commands()
+    assert "AC_type_unicode" in set(known)
+    from je_auto_control.utils.mcp_server.tools import build_default_tool_registry
+    names = {t.name for t in build_default_tool_registry()}
+    assert "ac_type_unicode" in names
+    from je_auto_control.gui.script_builder.command_schema import _build_specs
+    specs = {s.command for s in _build_specs()}
+    assert "AC_type_unicode" in specs
+
+
+def test_facade_exports():
+    for attr in ("type_unicode", "plan_paste", "unicode_code_units"):
+        assert hasattr(ac, attr) and attr in ac.__all__