diff --git a/README/WHATS_NEW_zh-CN.md b/README/WHATS_NEW_zh-CN.md index 76883031..b8e95cee 100644 --- a/README/WHATS_NEW_zh-CN.md +++ b/README/WHATS_NEW_zh-CN.md @@ -1,5 +1,11 @@ # 本次更新 — AutoControl +## 本次更新 (2026-06-23) — 按住按键 / 自动重复 + +按住一个键一段时间,或以固定频率自动重复。完整参考:[`docs/source/Zh/doc/new_features/v119_features_doc.rst`](../docs/source/Zh/doc/new_features/v119_features_doc.rst)。 + +- **`hold_key` / `plan_key_hold`**(`AC_hold_key`):`type_keyboard` 是瞬间按下+放开——先前没有「按住此键 N 秒」(游戏移动、按住滚动)或「每秒送 R 次」(自动重复)。`plan_key_hold` 建立确定性操作计划(按下/等待/放开,或为 `rate_hz` 产生 N 个间隔按键事件);`hold_key` 将等待导向可注入的 `sleep`、按键导向可注入的 `sink`。纯计划、确定。 + ## 本次更新 (2026-06-23) — 等待消失(阻塞式 vanish 等待) 阻塞直到转圈圈 / toast / 对话框消失。完整参考:[`docs/source/Zh/doc/new_features/v118_features_doc.rst`](../docs/source/Zh/doc/new_features/v118_features_doc.rst)。 diff --git a/README/WHATS_NEW_zh-TW.md b/README/WHATS_NEW_zh-TW.md index ec7eb96f..ae61b4bd 100644 --- a/README/WHATS_NEW_zh-TW.md +++ b/README/WHATS_NEW_zh-TW.md @@ -1,5 +1,11 @@ # 本次更新 — AutoControl +## 本次更新 (2026-06-23) — 按住按鍵 / 自動重複 + +按住一個鍵一段時間,或以固定頻率自動重複。完整參考:[`docs/source/Zh/doc/new_features/v119_features_doc.rst`](../docs/source/Zh/doc/new_features/v119_features_doc.rst)。 + +- **`hold_key` / `plan_key_hold`**(`AC_hold_key`):`type_keyboard` 是瞬間按下+放開——先前沒有「按住此鍵 N 秒」(遊戲移動、按住捲動)或「每秒送 R 次」(自動重複)。`plan_key_hold` 建立決定性操作計畫(按下/等待/放開,或為 `rate_hz` 產生 N 個間隔按鍵事件);`hold_key` 將等待導向可注入的 `sleep`、按鍵導向可注入的 `sink`。純計畫、具決定性。 + ## 本次更新 (2026-06-23) — 等待消失(阻塞式 vanish 等待) 阻塞直到轉圈圈 / toast / 對話框消失。完整參考:[`docs/source/Zh/doc/new_features/v118_features_doc.rst`](../docs/source/Zh/doc/new_features/v118_features_doc.rst)。 diff --git a/WHATS_NEW.md b/WHATS_NEW.md index 1c299998..e9005de0 100644 --- a/WHATS_NEW.md +++ b/WHATS_NEW.md @@ -1,5 +1,11 @@ # What's New — AutoControl +## What's new (2026-06-23) — Hold Key / Auto-Repeat + +Hold a key for a duration, or auto-repeat it at a fixed rate. Full reference: [`docs/source/Eng/doc/new_features/v119_features_doc.rst`](docs/source/Eng/doc/new_features/v119_features_doc.rst). + +- **`hold_key` / `plan_key_hold`** (`AC_hold_key`): `type_keyboard` is an instant down+up — there was no "hold this key for N seconds" (game movement, hold-to-scroll) or "send it at R presses/second" (auto-repeat). `plan_key_hold` builds the deterministic op-plan (press/wait/release, or N spaced key events for `rate_hz`); `hold_key` routes waits to an injectable `sleep` and keys to an injectable `sink`. Pure-planning, deterministic. + ## What's new (2026-06-23) — Wait Until Gone (Blocking Vanish Waits) Block until a spinner / toast / dialog disappears. Full reference: [`docs/source/Eng/doc/new_features/v118_features_doc.rst`](docs/source/Eng/doc/new_features/v118_features_doc.rst). diff --git a/docs/source/Eng/doc/new_features/v119_features_doc.rst b/docs/source/Eng/doc/new_features/v119_features_doc.rst new file mode 100644 index 00000000..68edaa18 --- /dev/null +++ b/docs/source/Eng/doc/new_features/v119_features_doc.rst @@ -0,0 +1,39 @@ +Hold Key / Auto-Repeat +====================== + +``type_keyboard`` is an instant down+up and ``input_macro.run_sequence`` can +hand-roll a press / wait / release, but there was no primitive for "hold this key +for N seconds" (game movement, hold-to-scroll) or "send it at R presses per +second" (auto-repeat). + +:func:`plan_key_hold` builds the deterministic op-plan (pure, unit-testable); +:func:`hold_key` dispatches it through an injectable ``sink`` and ``sleep`` so it +is tested without real input or real waiting. Imports no ``PySide6``. + +Headless API +------------ + +.. code-block:: python + + from je_auto_control import hold_key, plan_key_hold + + hold_key("key_d", duration_s=1.5) # press, hold 1.5s, release + hold_key("key_down", duration_s=2.0, rate_hz=20) # 40 key events @ 50ms + + plan_key_hold("space", 1.0) + # [{'op': 'press', 'key': 'space'}, + # {'op': 'wait', 'seconds': 1.0}, + # {'op': 'release', 'key': 'space'}] + +With ``rate_hz`` unset the key is pressed, held for ``duration_s``, then released. +With ``rate_hz`` set it is sent as ``round(duration_s * rate_hz)`` discrete key +events spaced ``1 / rate_hz`` apart — simulated auto-repeat for movement / scroll +loops. A non-positive duration or rate raises ``ValueError``. ``hold_key`` routes +the ``wait`` steps to ``sleep`` and the key steps to ``sink``, both injectable. + +Executor commands +----------------- + +``AC_hold_key`` takes ``key`` plus ``duration_s`` and an optional ``rate_hz`` and +returns ``{ops, plan}``. It is exposed as the MCP tool ``ac_hold_key`` and as a +Script Builder command under **Keyboard**. diff --git a/docs/source/Eng/eng_index.rst b/docs/source/Eng/eng_index.rst index b9203ae9..5945bab3 100644 --- a/docs/source/Eng/eng_index.rst +++ b/docs/source/Eng/eng_index.rst @@ -141,6 +141,7 @@ Comprehensive guides for all AutoControl features. doc/new_features/v116_features_doc doc/new_features/v117_features_doc doc/new_features/v118_features_doc + doc/new_features/v119_features_doc doc/ocr_backends/ocr_backends_doc doc/observability/observability_doc doc/operations_layer/operations_layer_doc diff --git a/docs/source/Zh/doc/new_features/v119_features_doc.rst b/docs/source/Zh/doc/new_features/v119_features_doc.rst new file mode 100644 index 00000000..10df75cf --- /dev/null +++ b/docs/source/Zh/doc/new_features/v119_features_doc.rst @@ -0,0 +1,34 @@ +按住按鍵 / 自動重複 +================== + +``type_keyboard`` 是瞬間的按下+放開,``input_macro.run_sequence`` 雖可手動拼出按下 / 等待 / 放開,但先前沒有 +「按住此鍵 N 秒」(遊戲移動、按住捲動)或「每秒送 R 次」(自動重複)的基本元件。 + +:func:`plan_key_hold` 建立決定性的操作計畫(純函式、可單元測試);:func:`hold_key` 透過可注入的 ``sink`` +與 ``sleep`` 派發,因此可在無真實輸入、無真實等待下測試。不匯入 ``PySide6``。 + +無頭 API +-------- + +.. code-block:: python + + from je_auto_control import hold_key, plan_key_hold + + hold_key("key_d", duration_s=1.5) # 按下、按住 1.5 秒、放開 + hold_key("key_down", duration_s=2.0, rate_hz=20) # 40 個按鍵事件 @ 50ms + + plan_key_hold("space", 1.0) + # [{'op': 'press', 'key': 'space'}, + # {'op': 'wait', 'seconds': 1.0}, + # {'op': 'release', 'key': 'space'}] + +未設定 ``rate_hz`` 時,鍵會被按下、按住 ``duration_s``、再放開。設定 ``rate_hz`` 時,會送出 +``round(duration_s * rate_hz)`` 個相隔 ``1 / rate_hz`` 的離散按鍵事件——用於移動 / 捲動迴圈的模擬自動重複。 +非正數的時長或頻率會拋出 ``ValueError``。``hold_key`` 將 ``wait`` 步驟導向 ``sleep``、按鍵步驟導向 ``sink``, +兩者皆可注入。 + +執行器命令 +---------- + +``AC_hold_key`` 接受 ``key`` 以及 ``duration_s`` 與選用的 ``rate_hz``,並回傳 ``{ops, plan}``。它以 MCP 工具 +``ac_hold_key`` 以及 Script Builder 中 **Keyboard** 分類下的命令提供。 diff --git a/docs/source/Zh/zh_index.rst b/docs/source/Zh/zh_index.rst index a1152f4c..7f5c98a5 100644 --- a/docs/source/Zh/zh_index.rst +++ b/docs/source/Zh/zh_index.rst @@ -141,6 +141,7 @@ AutoControl 所有功能的完整使用指南。 doc/new_features/v116_features_doc doc/new_features/v117_features_doc doc/new_features/v118_features_doc + doc/new_features/v119_features_doc doc/ocr_backends/ocr_backends_doc doc/observability/observability_doc doc/operations_layer/operations_layer_doc diff --git a/je_auto_control/__init__.py b/je_auto_control/__init__.py index 1494b92b..ae9dacb0 100644 --- a/je_auto_control/__init__.py +++ b/je_auto_control/__init__.py @@ -257,6 +257,8 @@ ) # Clear-then-type a text field (Playwright `fill` idiom; paste for Unicode) from je_auto_control.utils.field_entry import plan_field_set, set_field_text +# Hold a key for a duration / auto-repeat at a fixed rate +from je_auto_control.utils.key_hold import hold_key, plan_key_hold # CI workflow annotations (GitHub Actions) from je_auto_control.utils.ci_annotations import ( emit_annotations, format_annotation, @@ -1034,6 +1036,8 @@ def start_autocontrol_gui(*args, **kwargs): "path_easings", "plan_field_set", "set_field_text", + "plan_key_hold", + "hold_key", "emit_annotations", "format_annotation", "ClipboardHistory", "default_clipboard_history", "analyze_heal_log", "heal_stats", "scan_secrets", diff --git a/je_auto_control/gui/script_builder/command_schema.py b/je_auto_control/gui/script_builder/command_schema.py index 18b01c1a..1c11146e 100644 --- a/je_auto_control/gui/script_builder/command_schema.py +++ b/je_auto_control/gui/script_builder/command_schema.py @@ -174,6 +174,17 @@ def _add_keyboard_specs(specs: List[CommandSpec]) -> None: ), description="Clear the focused field then enter text (paste for Unicode).", )) + specs.append(CommandSpec( + "AC_hold_key", "Keyboard", "Hold Key", + fields=( + FieldSpec("key", FieldType.STRING, placeholder="e.g. key_d, space"), + FieldSpec("duration_s", FieldType.FLOAT, default=1.0, + min_value=0.01), + FieldSpec("rate_hz", FieldType.FLOAT, optional=True, + placeholder="auto-repeat presses/sec (blank = hold)"), + ), + description="Hold a key for a duration, or auto-repeat it at rate_hz.", + )) specs.append(CommandSpec( "AC_hotkey", "Keyboard", "Hotkey", fields=( diff --git a/je_auto_control/utils/executor/action_executor.py b/je_auto_control/utils/executor/action_executor.py index ede6aca7..aa21e73b 100644 --- a/je_auto_control/utils/executor/action_executor.py +++ b/je_auto_control/utils/executor/action_executor.py @@ -3123,6 +3123,14 @@ def _set_field_text(text: str, clear: str = "select_all", paste: Any = False, modifier=modifier) +def _hold_key(key: str, duration_s: Any = 1.0, + rate_hz: Any = None) -> Dict[str, Any]: + """Adapter: hold a key for a duration (or auto-repeat at rate_hz).""" + from je_auto_control.utils.key_hold import hold_key + rate = float(rate_hz) if rate_hz not in (None, "") else None + return hold_key(key, float(duration_s), rate_hz=rate) + + def _cas_put(name: str, key: str, value: Any, expected_version: Any = None) -> Dict[str, Any]: """Adapter: optimistic put into a named versioned store.""" @@ -4821,6 +4829,7 @@ def __init__(self): "AC_move_along_path": _move_along_path, "AC_drag_path": _drag_path, "AC_set_field_text": _set_field_text, + "AC_hold_key": _hold_key, "AC_detect_drift": _detect_drift, "AC_categorical_drift": _categorical_drift, "AC_diff_rows": _diff_rows, diff --git a/je_auto_control/utils/key_hold/__init__.py b/je_auto_control/utils/key_hold/__init__.py new file mode 100644 index 00000000..4fb6c7be --- /dev/null +++ b/je_auto_control/utils/key_hold/__init__.py @@ -0,0 +1,4 @@ +"""Hold a key for a duration, or auto-repeat it at a fixed rate.""" +from je_auto_control.utils.key_hold.key_hold import hold_key, plan_key_hold + +__all__ = ["hold_key", "plan_key_hold"] diff --git a/je_auto_control/utils/key_hold/key_hold.py b/je_auto_control/utils/key_hold/key_hold.py new file mode 100644 index 00000000..cdc79102 --- /dev/null +++ b/je_auto_control/utils/key_hold/key_hold.py @@ -0,0 +1,73 @@ +"""Hold a key for a duration, or auto-repeat it at a fixed rate. + +``type_keyboard`` is an instant down+up and ``input_macro.run_sequence`` can hand- +roll a press / wait / release, but there is no primitive for "hold this key for N +seconds" (game movement, hold-to-scroll) or "send it at R presses per second" +(auto-repeat). :func:`plan_key_hold` builds the deterministic op-plan (pure, +unit-testable); :func:`hold_key` dispatches it through an injectable ``sink`` and +``sleep`` so it is tested without real input or real waiting. Imports no +``PySide6``. +""" +import time +from typing import Any, Callable, Dict, List, Optional + +Sink = Callable[[Dict[str, Any]], None] + + +def plan_key_hold(key: str, duration_s: float, *, + rate_hz: Optional[float] = None) -> List[Dict[str, Any]]: + """Return the op-plan to hold (or auto-repeat) ``key`` for ``duration_s``. + + With ``rate_hz`` unset the key is pressed, held for ``duration_s``, then + released. With ``rate_hz`` set it is sent as ``round(duration_s * rate_hz)`` + discrete key events spaced ``1 / rate_hz`` apart (simulated auto-repeat). + Raises ``ValueError`` on a non-positive duration or rate. + """ + if duration_s <= 0: + raise ValueError("duration_s must be positive") + if rate_hz is None: + return [{"op": "press", "key": key}, + {"op": "wait", "seconds": float(duration_s)}, + {"op": "release", "key": key}] + if rate_hz <= 0: + raise ValueError("rate_hz must be positive") + interval = 1.0 / float(rate_hz) + count = max(1, round(float(duration_s) * float(rate_hz))) + plan: List[Dict[str, Any]] = [] + for index in range(count): + plan.append({"op": "key", "key": key}) + if index != count - 1: + plan.append({"op": "wait", "seconds": interval}) + return plan + + +def _default_sink(event: Dict[str, Any]) -> None: + """Default dispatch: drive the real keyboard backend.""" + from je_auto_control.wrapper.auto_control_keyboard import ( + press_keyboard_key, release_keyboard_key, type_keyboard) + op = event["op"] + if op == "press": + press_keyboard_key(event["key"]) + elif op == "release": + release_keyboard_key(event["key"]) + elif op == "key": + type_keyboard(event["key"]) + + +def hold_key(key: str, duration_s: float, *, rate_hz: Optional[float] = None, + sink: Optional[Sink] = None, + sleep: Optional[Callable[[float], None]] = None) -> Dict[str, Any]: + """Hold or auto-repeat ``key`` for ``duration_s``; return the dispatched plan. + + ``wait`` ops go to ``sleep`` (default :func:`time.sleep`); key ops go to + ``sink`` (default: the real keyboard backend). + """ + plan = plan_key_hold(key, duration_s, rate_hz=rate_hz) + dispatch = sink or _default_sink + pause = sleep or time.sleep + for event in plan: + if event["op"] == "wait": + pause(event["seconds"]) + else: + dispatch(event) + return {"ops": len(plan), "plan": plan} diff --git a/je_auto_control/utils/mcp_server/tools/_factories.py b/je_auto_control/utils/mcp_server/tools/_factories.py index 76c83357..b62f009a 100644 --- a/je_auto_control/utils/mcp_server/tools/_factories.py +++ b/je_auto_control/utils/mcp_server/tools/_factories.py @@ -2578,6 +2578,24 @@ def tween_drag_tools() -> List[MCPTool]: ] +def key_hold_tools() -> List[MCPTool]: + return [ + MCPTool( + name="ac_hold_key", + description=("Hold 'key' for 'duration_s' seconds, or set 'rate_hz' " + "to auto-repeat it at that many presses/second. " + "Returns {ops, plan}."), + input_schema=schema({ + "key": {"type": "string"}, + "duration_s": {"type": "number"}, + "rate_hz": {"type": "number"}}, + required=["key", "duration_s"]), + handler=h.hold_key, + annotations=SIDE_EFFECT_ONLY, + ), + ] + + def field_entry_tools() -> List[MCPTool]: return [ MCPTool( @@ -5878,7 +5896,7 @@ def media_assert_tools() -> List[MCPTool]: input_macro_tools, resilience_tools, ci_annotation_tools, clipboard_history_tools, audit_analysis_tools, process_doc_tools, tween_drag_tools, mouse_path_tools, field_entry_tools, - plugin_sdk_tools, governance_tools, + key_hold_tools, plugin_sdk_tools, governance_tools, credential_lease_tools, egress_tools, approval_testing_tools, trajectory_eval_tools, compliance_tools, agent_trace_tools, video_report_tools, fuzzy_tools, artifact_store_tools, image_dedup_tools, diff --git a/je_auto_control/utils/mcp_server/tools/_handlers.py b/je_auto_control/utils/mcp_server/tools/_handlers.py index c685a332..1d25ee3b 100644 --- a/je_auto_control/utils/mcp_server/tools/_handlers.py +++ b/je_auto_control/utils/mcp_server/tools/_handlers.py @@ -2043,6 +2043,11 @@ def set_field_text(text, clear="select_all", paste=False, modifier="ctrl"): return _set_field_text(text, clear, paste, modifier) +def hold_key(key, duration_s=1.0, rate_hz=None): + from je_auto_control.utils.executor.action_executor import _hold_key + return _hold_key(key, duration_s, rate_hz) + + def detect_drift(reference, current, threshold=0.25, bins=10): from je_auto_control.utils.executor.action_executor import _detect_drift return _detect_drift(reference, current, threshold, bins) diff --git a/test/unit_test/headless/test_key_hold_batch.py b/test/unit_test/headless/test_key_hold_batch.py new file mode 100644 index 00000000..6cc0fffc --- /dev/null +++ b/test/unit_test/headless/test_key_hold_batch.py @@ -0,0 +1,77 @@ +"""Headless tests for key hold / auto-repeat. No Qt.""" +import pytest + +import je_auto_control as ac +from je_auto_control.utils.key_hold import hold_key, plan_key_hold + + +def test_hold_plan_press_wait_release(): + plan = plan_key_hold("key_d", 1.5) + assert plan == [ + {"op": "press", "key": "key_d"}, + {"op": "wait", "seconds": 1.5}, + {"op": "release", "key": "key_d"}, + ] + + +def test_repeat_plan_emits_n_key_events(): + plan = plan_key_hold("a", 1.0, rate_hz=20) + keys = [event for event in plan if event["op"] == "key"] + waits = [event for event in plan if event["op"] == "wait"] + assert len(keys) == 20 + assert len(waits) == 19 # one fewer gap than events + assert waits[0]["seconds"] == pytest.approx(0.05) # 1 / 20 Hz + + +def test_repeat_rounds_count_min_one(): + assert len([e for e in plan_key_hold("a", 0.01, rate_hz=10) + if e["op"] == "key"]) == 1 # round(0.1) -> 0 -> clamped to 1 + + +def test_hold_key_routes_waits_to_sleep(): + events, slept = [], [] + result = hold_key("a", 0.5, sink=events.append, sleep=slept.append) + assert [e["op"] for e in events] == ["press", "release"] + assert slept == [0.5] + assert result["ops"] == 3 + + +def test_validation(): + for bad in (("a", 0), ("a", -1)): + try: + plan_key_hold(*bad) + except ValueError: + pass + else: # pragma: no cover + raise AssertionError("expected ValueError") + try: + plan_key_hold("a", 1.0, rate_hz=0) + except ValueError: + pass + else: # pragma: no cover + raise AssertionError("expected ValueError for rate_hz=0") + + +# --- wiring --------------------------------------------------------------- + +def test_executor_adapter_planning(): + events, slept = [], [] + from je_auto_control.utils.key_hold import hold_key as _hk + _hk("space", 0.2, sink=events.append, sleep=slept.append) + assert events and events[0]["op"] == "press" and events[-1]["op"] == "release" + + +def test_wiring(): + known = ac.executor.known_commands() + assert "AC_hold_key" in set(known) + from je_auto_control.utils.mcp_server.tools import build_default_tool_registry + names = {t.name for t in build_default_tool_registry()} + assert "ac_hold_key" in names + from je_auto_control.gui.script_builder.command_schema import _build_specs + specs = {s.command for s in _build_specs()} + assert "AC_hold_key" in specs + + +def test_facade_exports(): + for attr in ("plan_key_hold", "hold_key"): + assert hasattr(ac, attr) and attr in ac.__all__