diff --git a/README/WHATS_NEW_zh-CN.md b/README/WHATS_NEW_zh-CN.md index 4915c9b3..8e404b1b 100644 --- a/README/WHATS_NEW_zh-CN.md +++ b/README/WHATS_NEW_zh-CN.md @@ -1,5 +1,11 @@ # 本次更新 — AutoControl +## 本次更新 (2026-06-23) — 窗口 Z-order(置顶 / 最前 / 最后) + +把窗口钉在最上层、移到最前、或推到后面。完整参考:[`docs/source/Zh/doc/new_features/v147_features_doc.rst`](../docs/source/Zh/doc/new_features/v147_features_doc.rst)。 + +- **`set_topmost` / `bring_to_front` / `send_to_back` / `plan_zorder`**(`AC_set_topmost`、`AC_bring_to_front`、`AC_send_to_back`):原始 `set_window_position` 存在但未在 facade、无标题包装也无 topmost 语意——缺少标准 RPA 的「置顶」。`plan_zorder` 是纯动作→`SetWindowPos` 常数查找(可无头测试);以标题操作的设定器透过可注入 driver(`snap_window` 接缝模式)套用,默认为 Win32。 + ## 本次更新 (2026-06-23) — 局部动态 / 活动检测 找出两帧之间哪些子区域在动。完整参考:[`docs/source/Zh/doc/new_features/v146_features_doc.rst`](../docs/source/Zh/doc/new_features/v146_features_doc.rst)。 diff --git a/README/WHATS_NEW_zh-TW.md b/README/WHATS_NEW_zh-TW.md index 947b52cd..2b99f10d 100644 --- a/README/WHATS_NEW_zh-TW.md +++ b/README/WHATS_NEW_zh-TW.md @@ -1,5 +1,11 @@ # 本次更新 — AutoControl +## 本次更新 (2026-06-23) — 視窗 Z-order(置頂 / 最前 / 最後) + +把視窗釘在最上層、移到最前、或推到後面。完整參考:[`docs/source/Zh/doc/new_features/v147_features_doc.rst`](../docs/source/Zh/doc/new_features/v147_features_doc.rst)。 + +- **`set_topmost` / `bring_to_front` / `send_to_back` / `plan_zorder`**(`AC_set_topmost`、`AC_bring_to_front`、`AC_send_to_back`):原始 `set_window_position` 存在但未在 facade、無標題包裝也無 topmost 語意——缺少標準 RPA 的「置頂」。`plan_zorder` 是純動作→`SetWindowPos` 常數查找(可無頭測試);以標題操作的設定器透過可注入 driver(`snap_window` 接縫模式)套用,預設為 Win32。 + ## 本次更新 (2026-06-23) — 局部動態 / 活動偵測 找出兩幀之間哪些子區域在動。完整參考:[`docs/source/Zh/doc/new_features/v146_features_doc.rst`](../docs/source/Zh/doc/new_features/v146_features_doc.rst)。 diff --git a/WHATS_NEW.md b/WHATS_NEW.md index 1c821895..2fab21c0 100644 --- a/WHATS_NEW.md +++ b/WHATS_NEW.md @@ -1,5 +1,11 @@ # What's New — AutoControl +## What's new (2026-06-23) — Window Z-Order (Always-On-Top / Front / Back) + +Pin a window on top, raise it, or push it behind. Full reference: [`docs/source/Eng/doc/new_features/v147_features_doc.rst`](docs/source/Eng/doc/new_features/v147_features_doc.rst). + +- **`set_topmost` / `bring_to_front` / `send_to_back` / `plan_zorder`** (`AC_set_topmost`, `AC_bring_to_front`, `AC_send_to_back`): the raw `set_window_position` existed but wasn't in the facade, had no title wrapper and no topmost semantics — the standard RPA "always-on-top" was missing. `plan_zorder` is a pure action→`SetWindowPos` constant lookup (headless-testable); the title-based setters apply it through an injectable driver (the `snap_window` seam pattern), Win32 by default. + ## What's new (2026-06-23) — Localized Motion / Activity Detection Find which sub-regions are animating between two frames. Full reference: [`docs/source/Eng/doc/new_features/v146_features_doc.rst`](docs/source/Eng/doc/new_features/v146_features_doc.rst). diff --git a/docs/source/Eng/doc/new_features/v147_features_doc.rst b/docs/source/Eng/doc/new_features/v147_features_doc.rst new file mode 100644 index 00000000..d6fda0bb --- /dev/null +++ b/docs/source/Eng/doc/new_features/v147_features_doc.rst @@ -0,0 +1,42 @@ +Window Z-Order — Always-On-Top / Front / Back +============================================= + +``windows_window_manage.set_window_position`` exists at the raw Win32 layer but is not +exported in the package facade, has no title-based wrapper, and no topmost / not-topmost +semantics — and there was no ``always_on_top`` anywhere outside GUI overlay code. This +adds the standard RPA z-order primitive: a pure ``plan_zorder`` that maps an action to +the ``SetWindowPos`` insert-after constant, plus title-based ``set_topmost`` / +``bring_to_front`` / ``send_to_back`` over an injectable driver (the same seam pattern +as ``snap_window``). + +The planning is pure and headless-testable; only the default driver touches Win32 +(returning ``False`` on other platforms). Imports no ``PySide6``. + +Headless API +------------ + +.. code-block:: python + + from je_auto_control import (set_topmost, bring_to_front, send_to_back, + plan_zorder) + + set_topmost("Media Player") # pin always-on-top + set_topmost("Media Player", False) # release + bring_to_front("Editor") + send_to_back("Background Monitor") + + plan_zorder("topmost")["insert_after"] # -1 (HWND_TOPMOST), pure / testable + +``plan_zorder(action)`` (``top`` / ``bottom`` / ``topmost`` / ``notopmost``) returns the +``SetWindowPos`` descriptor (``insert_after`` constant + ``SWP_NOMOVE`` / ``SWP_NOSIZE`` +flags); unknown actions raise ``ValueError``. ``set_topmost`` / ``bring_to_front`` / +``send_to_back`` resolve the window by title and apply the action through the default +Win32 driver (or an injected one in tests), returning whether it was applied. + +Executor commands +----------------- + +``AC_set_topmost`` (``title`` / ``on`` → ``{applied}``), ``AC_bring_to_front`` and +``AC_send_to_back`` (``title`` → ``{applied}``). They are exposed as the MCP tools +``ac_set_topmost`` / ``ac_bring_to_front`` / ``ac_send_to_back`` and as Script Builder +commands under **Window**. diff --git a/docs/source/Eng/eng_index.rst b/docs/source/Eng/eng_index.rst index 7e8a7540..41045d39 100644 --- a/docs/source/Eng/eng_index.rst +++ b/docs/source/Eng/eng_index.rst @@ -169,6 +169,7 @@ Comprehensive guides for all AutoControl features. doc/new_features/v144_features_doc doc/new_features/v145_features_doc doc/new_features/v146_features_doc + doc/new_features/v147_features_doc doc/ocr_backends/ocr_backends_doc doc/observability/observability_doc doc/operations_layer/operations_layer_doc diff --git a/docs/source/Zh/doc/new_features/v147_features_doc.rst b/docs/source/Zh/doc/new_features/v147_features_doc.rst new file mode 100644 index 00000000..54f50614 --- /dev/null +++ b/docs/source/Zh/doc/new_features/v147_features_doc.rst @@ -0,0 +1,35 @@ +視窗 Z-order——置頂 / 移到最前 / 移到最後 +========================================= + +``windows_window_manage.set_window_position`` 在原始 Win32 層存在,但未在套件 facade 匯出、沒有以標題為基礎的包裝、 +也沒有 topmost / not-topmost 語意——而 GUI 疊圖以外的地方完全沒有 ``always_on_top``。本功能加入標準 RPA z-order +基本能力:純函式 ``plan_zorder`` 將動作對應到 ``SetWindowPos`` 的 insert-after 常數,以及以標題操作、可注入 driver +的 ``set_topmost`` / ``bring_to_front`` / ``send_to_back``(與 ``snap_window`` 相同的接縫模式)。 + +規劃為純函式且可無頭測試;只有預設 driver 觸及 Win32(其他平台回傳 ``False``)。不匯入 ``PySide6``。 + +無頭 API +-------- + +.. code-block:: python + + from je_auto_control import (set_topmost, bring_to_front, send_to_back, + plan_zorder) + + set_topmost("Media Player") # 置頂 + set_topmost("Media Player", False) # 取消置頂 + bring_to_front("Editor") + send_to_back("Background Monitor") + + plan_zorder("topmost")["insert_after"] # -1(HWND_TOPMOST),純 / 可測 + +``plan_zorder(action)``(``top`` / ``bottom`` / ``topmost`` / ``notopmost``)回傳 ``SetWindowPos`` 描述符 +(``insert_after`` 常數 + ``SWP_NOMOVE`` / ``SWP_NOSIZE`` 旗標);未知動作丟出 ``ValueError``。``set_topmost`` / +``bring_to_front`` / ``send_to_back`` 以標題解析視窗並透過預設 Win32 driver(測試時注入)套用動作,回傳是否已套用。 + +執行器命令 +---------- + +``AC_set_topmost``(``title`` / ``on`` → ``{applied}``)、``AC_bring_to_front`` 與 ``AC_send_to_back`` +(``title`` → ``{applied}``)。它們以 MCP 工具 ``ac_set_topmost`` / ``ac_bring_to_front`` / ``ac_send_to_back`` +以及 Script Builder 中 **Window** 分類下的命令提供。 diff --git a/docs/source/Zh/zh_index.rst b/docs/source/Zh/zh_index.rst index 5c77e039..fb1dbf0b 100644 --- a/docs/source/Zh/zh_index.rst +++ b/docs/source/Zh/zh_index.rst @@ -169,6 +169,7 @@ AutoControl 所有功能的完整使用指南。 doc/new_features/v144_features_doc doc/new_features/v145_features_doc doc/new_features/v146_features_doc + doc/new_features/v147_features_doc doc/ocr_backends/ocr_backends_doc doc/observability/observability_doc doc/operations_layer/operations_layer_doc diff --git a/je_auto_control/__init__.py b/je_auto_control/__init__.py index ed962943..4c36f5b4 100644 --- a/je_auto_control/__init__.py +++ b/je_auto_control/__init__.py @@ -347,6 +347,10 @@ from je_auto_control.utils.motion_regions import ( activity_score, changed_regions, has_motion, ) +# Window z-order control (topmost / bring-to-front / send-to-back) +from je_auto_control.utils.window_zorder import ( + bring_to_front, plan_zorder, send_to_back, set_topmost, +) # CI workflow annotations (GitHub Actions) from je_auto_control.utils.ci_annotations import ( emit_annotations, format_annotation, @@ -1209,6 +1213,10 @@ def start_autocontrol_gui(*args, **kwargs): "changed_regions", "has_motion", "activity_score", + "plan_zorder", + "set_topmost", + "bring_to_front", + "send_to_back", "emit_annotations", "format_annotation", "ClipboardHistory", "default_clipboard_history", "analyze_heal_log", "heal_stats", "scan_secrets", diff --git a/je_auto_control/gui/script_builder/command_schema.py b/je_auto_control/gui/script_builder/command_schema.py index 6180abb7..00d83a68 100644 --- a/je_auto_control/gui/script_builder/command_schema.py +++ b/je_auto_control/gui/script_builder/command_schema.py @@ -676,6 +676,24 @@ def _add_window_specs(specs: List[CommandSpec]) -> None: ), description="Cascade a list of windows diagonally.", )) + specs.append(CommandSpec( + "AC_set_topmost", "Window", "Set Always-On-Top", + fields=( + FieldSpec("title", FieldType.STRING), + FieldSpec("on", FieldType.BOOL, optional=True, default=True), + ), + description="Pin a window always-on-top (or release it).", + )) + specs.append(CommandSpec( + "AC_bring_to_front", "Window", "Bring Window to Front", + fields=(FieldSpec("title", FieldType.STRING),), + description="Raise a window to the top of the z-order.", + )) + specs.append(CommandSpec( + "AC_send_to_back", "Window", "Send Window to Back", + fields=(FieldSpec("title", FieldType.STRING),), + description="Send a window to the bottom of the z-order.", + )) specs.append(CommandSpec( "AC_wait_window_closed", "Window", "Wait for Window to Close", fields=( diff --git a/je_auto_control/utils/executor/action_executor.py b/je_auto_control/utils/executor/action_executor.py index 888991f4..3bbec21e 100644 --- a/je_auto_control/utils/executor/action_executor.py +++ b/je_auto_control/utils/executor/action_executor.py @@ -3735,6 +3735,24 @@ def _resolve_after(after: Any): return np.asarray(pil_screenshot().convert("RGB")) +def _set_topmost(title: str, on: Any = True) -> Dict[str, Any]: + """Adapter: pin a window always-on-top (or release it).""" + from je_auto_control.utils.window_zorder import set_topmost + return {"applied": set_topmost(title, bool(on))} + + +def _bring_to_front(title: str) -> Dict[str, Any]: + """Adapter: raise a window to the top of the z-order.""" + from je_auto_control.utils.window_zorder import bring_to_front + return {"applied": bring_to_front(title)} + + +def _send_to_back(title: str) -> Dict[str, Any]: + """Adapter: send a window to the bottom of the z-order.""" + from je_auto_control.utils.window_zorder import send_to_back + return {"applied": send_to_back(title)} + + def _with_modifiers(modifiers: Any, actions: Any) -> Dict[str, Any]: """Adapter: run nested actions while modifier keys are held down.""" import json @@ -5482,6 +5500,9 @@ def __init__(self): "AC_histogram_changed": _histogram_changed, "AC_changed_regions": _changed_regions, "AC_has_motion": _has_motion, + "AC_set_topmost": _set_topmost, + "AC_bring_to_front": _bring_to_front, + "AC_send_to_back": _send_to_back, "AC_tile_rect": _tile_rect, "AC_grid_rects": _grid_rects, "AC_cascade_rects": _cascade_rects, diff --git a/je_auto_control/utils/mcp_server/tools/_factories.py b/je_auto_control/utils/mcp_server/tools/_factories.py index cbaa14f8..d24ca817 100644 --- a/je_auto_control/utils/mcp_server/tools/_factories.py +++ b/je_auto_control/utils/mcp_server/tools/_factories.py @@ -3158,6 +3158,38 @@ def motion_regions_tools() -> List[MCPTool]: ] +def window_zorder_tools() -> List[MCPTool]: + return [ + MCPTool( + name="ac_set_topmost", + description=("Pin the window matching 'title' always-on-top (or release " + "it with on=false). Returns {applied}. Windows only."), + input_schema=schema({ + "title": {"type": "string"}, + "on": {"type": "boolean"}}, + required=["title"]), + handler=h.set_topmost, + annotations=SIDE_EFFECT_ONLY, + ), + MCPTool( + name="ac_bring_to_front", + description=("Raise the window matching 'title' to the top of the " + "z-order. Returns {applied}. Windows only."), + input_schema=schema({"title": {"type": "string"}}, required=["title"]), + handler=h.bring_to_front, + annotations=SIDE_EFFECT_ONLY, + ), + MCPTool( + name="ac_send_to_back", + description=("Send the window matching 'title' to the bottom of the " + "z-order. Returns {applied}. Windows only."), + input_schema=schema({"title": {"type": "string"}}, required=["title"]), + handler=h.send_to_back, + annotations=SIDE_EFFECT_ONLY, + ), + ] + + def ssim_tools() -> List[MCPTool]: return [ MCPTool( @@ -6664,7 +6696,7 @@ def media_assert_tools() -> List[MCPTool]: monitor_layout_tools, actionability_tools, element_parse_tools, hsv_segment_tools, text_regions_tools, edge_lines_tools, expect_poll_tools, locator_chain_tools, rich_clipboard_tools, img_histogram_tools, - motion_regions_tools, plugin_sdk_tools, governance_tools, + motion_regions_tools, window_zorder_tools, plugin_sdk_tools, governance_tools, credential_lease_tools, egress_tools, approval_testing_tools, trajectory_eval_tools, compliance_tools, agent_trace_tools, video_report_tools, fuzzy_tools, artifact_store_tools, image_dedup_tools, diff --git a/je_auto_control/utils/mcp_server/tools/_handlers.py b/je_auto_control/utils/mcp_server/tools/_handlers.py index 09b0ceaf..f783770b 100644 --- a/je_auto_control/utils/mcp_server/tools/_handlers.py +++ b/je_auto_control/utils/mcp_server/tools/_handlers.py @@ -2268,6 +2268,21 @@ def has_motion(before, after=None, threshold=25, min_area=80): return _has_motion(before, after, threshold, min_area) +def set_topmost(title, on=True): + from je_auto_control.utils.executor.action_executor import _set_topmost + return _set_topmost(title, on) + + +def bring_to_front(title): + from je_auto_control.utils.executor.action_executor import _bring_to_front + return _bring_to_front(title) + + +def send_to_back(title): + from je_auto_control.utils.executor.action_executor import _send_to_back + return _send_to_back(title) + + def detect_drift(reference, current, threshold=0.25, bins=10): from je_auto_control.utils.executor.action_executor import _detect_drift return _detect_drift(reference, current, threshold, bins) diff --git a/je_auto_control/utils/window_zorder/__init__.py b/je_auto_control/utils/window_zorder/__init__.py new file mode 100644 index 00000000..7c3aee9b --- /dev/null +++ b/je_auto_control/utils/window_zorder/__init__.py @@ -0,0 +1,7 @@ +"""Window z-order control (topmost / bring-to-front / send-to-back).""" +from je_auto_control.utils.window_zorder.window_zorder import ( + available_actions, bring_to_front, plan_zorder, send_to_back, set_topmost, +) + +__all__ = ["available_actions", "bring_to_front", "plan_zorder", "send_to_back", + "set_topmost"] diff --git a/je_auto_control/utils/window_zorder/window_zorder.py b/je_auto_control/utils/window_zorder/window_zorder.py new file mode 100644 index 00000000..b228ee29 --- /dev/null +++ b/je_auto_control/utils/window_zorder/window_zorder.py @@ -0,0 +1,69 @@ +"""Window z-order control — always-on-top / topmost, bring-to-front, send-to-back. + +``windows_window_manage.set_window_position(hwnd, position)`` exists at the raw Win32 +layer but is **not** exported in the package facade, has no title-based wrapper, and no +topmost / not-topmost semantics — and there is no ``always_on_top`` anywhere outside +GUI overlay code. This adds the standard RPA z-order primitive: a pure ``plan_zorder`` +that maps an action to the ``SetWindowPos`` insert-after constant, plus title-based +``set_topmost`` / ``bring_to_front`` / ``send_to_back`` over an injectable driver (the +same seam pattern as ``snap_window``). + +The planning is pure and headless-testable; only the default driver touches Win32 +(returning ``False`` on other platforms). Imports no ``PySide6``. +""" +import sys +from typing import Any, Callable, Dict + +ZOrderDriver = Callable[[str, str], bool] + +# action -> SetWindowPos hwndInsertAfter constant +_ACTIONS = {"top": 0, "bottom": 1, "topmost": -1, "notopmost": -2} + + +def available_actions() -> list: + """Return the supported z-order action names.""" + return list(_ACTIONS) + + +def plan_zorder(action: str) -> Dict[str, Any]: + """Return the ``SetWindowPos`` descriptor for a z-order ``action``. + + ``action`` is one of ``top`` / ``bottom`` / ``topmost`` / ``notopmost``; the + result carries the ``insert_after`` HWND constant and the move/size-preserving + flags. Raises ``ValueError`` for an unknown action. + """ + if action not in _ACTIONS: + raise ValueError(f"unknown z-order action: {action!r}") + return {"action": action, "insert_after": _ACTIONS[action], + "flags": ["SWP_NOMOVE", "SWP_NOSIZE"]} + + +def _default_driver(title: str, action: str) -> bool: + """Apply a z-order action to the window matching ``title`` (Win32 only).""" + if not sys.platform.startswith("win"): + return False + from je_auto_control.wrapper.auto_control_window import find_window + hit = find_window(title) + if hit is None: + return False + from je_auto_control.windows.window import windows_window_manage as wm + wm.set_window_position(int(hit[0]), plan_zorder(action)["insert_after"]) + return True + + +def set_topmost(title: str, on: bool = True, *, + driver: Callable[[str, str], bool] = None) -> bool: + """Pin the window matching ``title`` always-on-top (or release it when ``on`` is False).""" + return (driver or _default_driver)(title, "topmost" if on else "notopmost") + + +def bring_to_front(title: str, *, + driver: Callable[[str, str], bool] = None) -> bool: + """Raise the window matching ``title`` to the top of the z-order.""" + return (driver or _default_driver)(title, "top") + + +def send_to_back(title: str, *, + driver: Callable[[str, str], bool] = None) -> bool: + """Send the window matching ``title`` to the bottom of the z-order.""" + return (driver or _default_driver)(title, "bottom") diff --git a/test/unit_test/headless/test_window_zorder_batch.py b/test/unit_test/headless/test_window_zorder_batch.py new file mode 100644 index 00000000..9809ba54 --- /dev/null +++ b/test/unit_test/headless/test_window_zorder_batch.py @@ -0,0 +1,67 @@ +"""Headless tests for window z-order control. No Qt; driver is injected.""" +import je_auto_control as ac +from je_auto_control.utils.window_zorder import ( + available_actions, bring_to_front, plan_zorder, send_to_back, set_topmost, +) + + +def _recorder(): + calls = [] + + def driver(title, action): + calls.append((title, action)) + return True + + return driver, calls + + +def test_plan_zorder_maps_constants(): + assert plan_zorder("top")["insert_after"] == 0 + assert plan_zorder("bottom")["insert_after"] == 1 + assert plan_zorder("topmost")["insert_after"] == -1 + assert plan_zorder("notopmost")["insert_after"] == -2 + assert plan_zorder("top")["flags"] == ["SWP_NOMOVE", "SWP_NOSIZE"] + + +def test_plan_zorder_unknown_raises(): + try: + plan_zorder("sideways") + except ValueError: + return + raise AssertionError("expected ValueError") + + +def test_set_topmost_on_and_off(): + driver, calls = _recorder() + assert set_topmost("Editor", True, driver=driver) is True + assert set_topmost("Editor", False, driver=driver) is True + assert calls == [("Editor", "topmost"), ("Editor", "notopmost")] + + +def test_bring_to_front_and_send_to_back(): + driver, calls = _recorder() + bring_to_front("Editor", driver=driver) + send_to_back("Editor", driver=driver) + assert calls == [("Editor", "top"), ("Editor", "bottom")] + + +def test_available_actions(): + assert set(available_actions()) == {"top", "bottom", "topmost", "notopmost"} + + +# --- wiring --------------------------------------------------------------- + +def test_wiring(): + known = set(ac.executor.known_commands()) + assert {"AC_set_topmost", "AC_bring_to_front", "AC_send_to_back"} <= known + from je_auto_control.utils.mcp_server.tools import build_default_tool_registry + names = {t.name for t in build_default_tool_registry()} + assert {"ac_set_topmost", "ac_bring_to_front", "ac_send_to_back"} <= names + from je_auto_control.gui.script_builder.command_schema import _build_specs + specs = {s.command for s in _build_specs()} + assert {"AC_set_topmost", "AC_bring_to_front", "AC_send_to_back"} <= specs + + +def test_facade_exports(): + for attr in ("plan_zorder", "set_topmost", "bring_to_front", "send_to_back"): + assert hasattr(ac, attr) and attr in ac.__all__