Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions README/WHATS_NEW_zh-CN.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
# 本次更新 — AutoControl

## 本次更新 (2026-06-23) — 窗口 Z-order(置顶 / 最前 / 最后)

把窗口钉在最上层、移到最前、或推到后面。完整参考:[`docs/source/Zh/doc/new_features/v147_features_doc.rst`](../docs/source/Zh/doc/new_features/v147_features_doc.rst)。

- **`set_topmost` / `bring_to_front` / `send_to_back` / `plan_zorder`**(`AC_set_topmost`、`AC_bring_to_front`、`AC_send_to_back`):原始 `set_window_position` 存在但未在 facade、无标题包装也无 topmost 语意——缺少标准 RPA 的「置顶」。`plan_zorder` 是纯动作→`SetWindowPos` 常数查找(可无头测试);以标题操作的设定器透过可注入 driver(`snap_window` 接缝模式)套用,默认为 Win32。

## 本次更新 (2026-06-23) — 局部动态 / 活动检测

找出两帧之间哪些子区域在动。完整参考:[`docs/source/Zh/doc/new_features/v146_features_doc.rst`](../docs/source/Zh/doc/new_features/v146_features_doc.rst)。
Expand Down
6 changes: 6 additions & 0 deletions README/WHATS_NEW_zh-TW.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
# 本次更新 — AutoControl

## 本次更新 (2026-06-23) — 視窗 Z-order(置頂 / 最前 / 最後)

把視窗釘在最上層、移到最前、或推到後面。完整參考:[`docs/source/Zh/doc/new_features/v147_features_doc.rst`](../docs/source/Zh/doc/new_features/v147_features_doc.rst)。

- **`set_topmost` / `bring_to_front` / `send_to_back` / `plan_zorder`**(`AC_set_topmost`、`AC_bring_to_front`、`AC_send_to_back`):原始 `set_window_position` 存在但未在 facade、無標題包裝也無 topmost 語意——缺少標準 RPA 的「置頂」。`plan_zorder` 是純動作→`SetWindowPos` 常數查找(可無頭測試);以標題操作的設定器透過可注入 driver(`snap_window` 接縫模式)套用,預設為 Win32。

## 本次更新 (2026-06-23) — 局部動態 / 活動偵測

找出兩幀之間哪些子區域在動。完整參考:[`docs/source/Zh/doc/new_features/v146_features_doc.rst`](../docs/source/Zh/doc/new_features/v146_features_doc.rst)。
Expand Down
6 changes: 6 additions & 0 deletions WHATS_NEW.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
# What's New — AutoControl

## What's new (2026-06-23) — Window Z-Order (Always-On-Top / Front / Back)

Pin a window on top, raise it, or push it behind. Full reference: [`docs/source/Eng/doc/new_features/v147_features_doc.rst`](docs/source/Eng/doc/new_features/v147_features_doc.rst).

- **`set_topmost` / `bring_to_front` / `send_to_back` / `plan_zorder`** (`AC_set_topmost`, `AC_bring_to_front`, `AC_send_to_back`): the raw `set_window_position` existed but wasn't in the facade, had no title wrapper and no topmost semantics — the standard RPA "always-on-top" was missing. `plan_zorder` is a pure action→`SetWindowPos` constant lookup (headless-testable); the title-based setters apply it through an injectable driver (the `snap_window` seam pattern), Win32 by default.

## What's new (2026-06-23) — Localized Motion / Activity Detection

Find which sub-regions are animating between two frames. Full reference: [`docs/source/Eng/doc/new_features/v146_features_doc.rst`](docs/source/Eng/doc/new_features/v146_features_doc.rst).
Expand Down
42 changes: 42 additions & 0 deletions docs/source/Eng/doc/new_features/v147_features_doc.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
Window Z-Order — Always-On-Top / Front / Back
=============================================

``windows_window_manage.set_window_position`` exists at the raw Win32 layer but is not
exported in the package facade, has no title-based wrapper, and no topmost / not-topmost
semantics — and there was no ``always_on_top`` anywhere outside GUI overlay code. This
adds the standard RPA z-order primitive: a pure ``plan_zorder`` that maps an action to
the ``SetWindowPos`` insert-after constant, plus title-based ``set_topmost`` /
``bring_to_front`` / ``send_to_back`` over an injectable driver (the same seam pattern
as ``snap_window``).

The planning is pure and headless-testable; only the default driver touches Win32
(returning ``False`` on other platforms). Imports no ``PySide6``.

Headless API
------------

.. code-block:: python

from je_auto_control import (set_topmost, bring_to_front, send_to_back,
plan_zorder)

set_topmost("Media Player") # pin always-on-top
set_topmost("Media Player", False) # release
bring_to_front("Editor")
send_to_back("Background Monitor")

plan_zorder("topmost")["insert_after"] # -1 (HWND_TOPMOST), pure / testable

``plan_zorder(action)`` (``top`` / ``bottom`` / ``topmost`` / ``notopmost``) returns the
``SetWindowPos`` descriptor (``insert_after`` constant + ``SWP_NOMOVE`` / ``SWP_NOSIZE``
flags); unknown actions raise ``ValueError``. ``set_topmost`` / ``bring_to_front`` /
``send_to_back`` resolve the window by title and apply the action through the default
Win32 driver (or an injected one in tests), returning whether it was applied.

Executor commands
-----------------

``AC_set_topmost`` (``title`` / ``on`` → ``{applied}``), ``AC_bring_to_front`` and
``AC_send_to_back`` (``title`` → ``{applied}``). They are exposed as the MCP tools
``ac_set_topmost`` / ``ac_bring_to_front`` / ``ac_send_to_back`` and as Script Builder
commands under **Window**.
1 change: 1 addition & 0 deletions docs/source/Eng/eng_index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -169,6 +169,7 @@ Comprehensive guides for all AutoControl features.
doc/new_features/v144_features_doc
doc/new_features/v145_features_doc
doc/new_features/v146_features_doc
doc/new_features/v147_features_doc
doc/ocr_backends/ocr_backends_doc
doc/observability/observability_doc
doc/operations_layer/operations_layer_doc
Expand Down
35 changes: 35 additions & 0 deletions docs/source/Zh/doc/new_features/v147_features_doc.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
視窗 Z-order——置頂 / 移到最前 / 移到最後
=========================================

``windows_window_manage.set_window_position`` 在原始 Win32 層存在,但未在套件 facade 匯出、沒有以標題為基礎的包裝、
也沒有 topmost / not-topmost 語意——而 GUI 疊圖以外的地方完全沒有 ``always_on_top``。本功能加入標準 RPA z-order
基本能力:純函式 ``plan_zorder`` 將動作對應到 ``SetWindowPos`` 的 insert-after 常數,以及以標題操作、可注入 driver
的 ``set_topmost`` / ``bring_to_front`` / ``send_to_back``(與 ``snap_window`` 相同的接縫模式)。

規劃為純函式且可無頭測試;只有預設 driver 觸及 Win32(其他平台回傳 ``False``)。不匯入 ``PySide6``。

無頭 API
--------

.. code-block:: python

from je_auto_control import (set_topmost, bring_to_front, send_to_back,
plan_zorder)

set_topmost("Media Player") # 置頂
set_topmost("Media Player", False) # 取消置頂
bring_to_front("Editor")
send_to_back("Background Monitor")

plan_zorder("topmost")["insert_after"] # -1(HWND_TOPMOST),純 / 可測

``plan_zorder(action)``(``top`` / ``bottom`` / ``topmost`` / ``notopmost``)回傳 ``SetWindowPos`` 描述符
(``insert_after`` 常數 + ``SWP_NOMOVE`` / ``SWP_NOSIZE`` 旗標);未知動作丟出 ``ValueError``。``set_topmost`` /
``bring_to_front`` / ``send_to_back`` 以標題解析視窗並透過預設 Win32 driver(測試時注入)套用動作,回傳是否已套用。

執行器命令
----------

``AC_set_topmost``(``title`` / ``on`` → ``{applied}``)、``AC_bring_to_front`` 與 ``AC_send_to_back``
(``title`` → ``{applied}``)。它們以 MCP 工具 ``ac_set_topmost`` / ``ac_bring_to_front`` / ``ac_send_to_back``
以及 Script Builder 中 **Window** 分類下的命令提供。
1 change: 1 addition & 0 deletions docs/source/Zh/zh_index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -169,6 +169,7 @@ AutoControl 所有功能的完整使用指南。
doc/new_features/v144_features_doc
doc/new_features/v145_features_doc
doc/new_features/v146_features_doc
doc/new_features/v147_features_doc
doc/ocr_backends/ocr_backends_doc
doc/observability/observability_doc
doc/operations_layer/operations_layer_doc
Expand Down
8 changes: 8 additions & 0 deletions je_auto_control/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -347,6 +347,10 @@
from je_auto_control.utils.motion_regions import (
activity_score, changed_regions, has_motion,
)
# Window z-order control (topmost / bring-to-front / send-to-back)
from je_auto_control.utils.window_zorder import (
bring_to_front, plan_zorder, send_to_back, set_topmost,
)
# CI workflow annotations (GitHub Actions)
from je_auto_control.utils.ci_annotations import (
emit_annotations, format_annotation,
Expand Down Expand Up @@ -1209,6 +1213,10 @@ def start_autocontrol_gui(*args, **kwargs):
"changed_regions",
"has_motion",
"activity_score",
"plan_zorder",
"set_topmost",
"bring_to_front",
"send_to_back",
"emit_annotations", "format_annotation",
"ClipboardHistory", "default_clipboard_history",
"analyze_heal_log", "heal_stats", "scan_secrets",
Expand Down
18 changes: 18 additions & 0 deletions je_auto_control/gui/script_builder/command_schema.py
Original file line number Diff line number Diff line change
Expand Up @@ -676,6 +676,24 @@ def _add_window_specs(specs: List[CommandSpec]) -> None:
),
description="Cascade a list of windows diagonally.",
))
specs.append(CommandSpec(
"AC_set_topmost", "Window", "Set Always-On-Top",
fields=(
FieldSpec("title", FieldType.STRING),
FieldSpec("on", FieldType.BOOL, optional=True, default=True),
),
description="Pin a window always-on-top (or release it).",
))
specs.append(CommandSpec(
"AC_bring_to_front", "Window", "Bring Window to Front",
fields=(FieldSpec("title", FieldType.STRING),),
description="Raise a window to the top of the z-order.",
))
specs.append(CommandSpec(
"AC_send_to_back", "Window", "Send Window to Back",
fields=(FieldSpec("title", FieldType.STRING),),
description="Send a window to the bottom of the z-order.",
))
specs.append(CommandSpec(
"AC_wait_window_closed", "Window", "Wait for Window to Close",
fields=(
Expand Down
21 changes: 21 additions & 0 deletions je_auto_control/utils/executor/action_executor.py
Original file line number Diff line number Diff line change
Expand Up @@ -3735,6 +3735,24 @@ def _resolve_after(after: Any):
return np.asarray(pil_screenshot().convert("RGB"))


def _set_topmost(title: str, on: Any = True) -> Dict[str, Any]:
"""Adapter: pin a window always-on-top (or release it)."""
from je_auto_control.utils.window_zorder import set_topmost
return {"applied": set_topmost(title, bool(on))}


def _bring_to_front(title: str) -> Dict[str, Any]:
"""Adapter: raise a window to the top of the z-order."""
from je_auto_control.utils.window_zorder import bring_to_front
return {"applied": bring_to_front(title)}


def _send_to_back(title: str) -> Dict[str, Any]:
"""Adapter: send a window to the bottom of the z-order."""
from je_auto_control.utils.window_zorder import send_to_back
return {"applied": send_to_back(title)}


def _with_modifiers(modifiers: Any, actions: Any) -> Dict[str, Any]:
"""Adapter: run nested actions while modifier keys are held down."""
import json
Expand Down Expand Up @@ -5482,6 +5500,9 @@ def __init__(self):
"AC_histogram_changed": _histogram_changed,
"AC_changed_regions": _changed_regions,
"AC_has_motion": _has_motion,
"AC_set_topmost": _set_topmost,
"AC_bring_to_front": _bring_to_front,
"AC_send_to_back": _send_to_back,
"AC_tile_rect": _tile_rect,
"AC_grid_rects": _grid_rects,
"AC_cascade_rects": _cascade_rects,
Expand Down
34 changes: 33 additions & 1 deletion je_auto_control/utils/mcp_server/tools/_factories.py
Original file line number Diff line number Diff line change
Expand Up @@ -3158,6 +3158,38 @@ def motion_regions_tools() -> List[MCPTool]:
]


def window_zorder_tools() -> List[MCPTool]:
return [
MCPTool(
name="ac_set_topmost",
description=("Pin the window matching 'title' always-on-top (or release "
"it with on=false). Returns {applied}. Windows only."),
input_schema=schema({
"title": {"type": "string"},
"on": {"type": "boolean"}},
required=["title"]),
handler=h.set_topmost,
annotations=SIDE_EFFECT_ONLY,
),
MCPTool(
name="ac_bring_to_front",
description=("Raise the window matching 'title' to the top of the "
"z-order. Returns {applied}. Windows only."),
input_schema=schema({"title": {"type": "string"}}, required=["title"]),
handler=h.bring_to_front,
annotations=SIDE_EFFECT_ONLY,
),
MCPTool(
name="ac_send_to_back",
description=("Send the window matching 'title' to the bottom of the "
"z-order. Returns {applied}. Windows only."),
input_schema=schema({"title": {"type": "string"}}, required=["title"]),
handler=h.send_to_back,
annotations=SIDE_EFFECT_ONLY,
),
]


def ssim_tools() -> List[MCPTool]:
return [
MCPTool(
Expand Down Expand Up @@ -6664,7 +6696,7 @@ def media_assert_tools() -> List[MCPTool]:
monitor_layout_tools, actionability_tools, element_parse_tools,
hsv_segment_tools, text_regions_tools, edge_lines_tools, expect_poll_tools,
locator_chain_tools, rich_clipboard_tools, img_histogram_tools,
motion_regions_tools, plugin_sdk_tools, governance_tools,
motion_regions_tools, window_zorder_tools, plugin_sdk_tools, governance_tools,
credential_lease_tools, egress_tools, approval_testing_tools,
trajectory_eval_tools, compliance_tools, agent_trace_tools,
video_report_tools, fuzzy_tools, artifact_store_tools, image_dedup_tools,
Expand Down
15 changes: 15 additions & 0 deletions je_auto_control/utils/mcp_server/tools/_handlers.py
Original file line number Diff line number Diff line change
Expand Up @@ -2268,6 +2268,21 @@ def has_motion(before, after=None, threshold=25, min_area=80):
return _has_motion(before, after, threshold, min_area)


def set_topmost(title, on=True):
from je_auto_control.utils.executor.action_executor import _set_topmost
return _set_topmost(title, on)


def bring_to_front(title):
from je_auto_control.utils.executor.action_executor import _bring_to_front
return _bring_to_front(title)


def send_to_back(title):
from je_auto_control.utils.executor.action_executor import _send_to_back
return _send_to_back(title)


def detect_drift(reference, current, threshold=0.25, bins=10):
from je_auto_control.utils.executor.action_executor import _detect_drift
return _detect_drift(reference, current, threshold, bins)
Expand Down
7 changes: 7 additions & 0 deletions je_auto_control/utils/window_zorder/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
"""Window z-order control (topmost / bring-to-front / send-to-back)."""
from je_auto_control.utils.window_zorder.window_zorder import (
available_actions, bring_to_front, plan_zorder, send_to_back, set_topmost,
)

__all__ = ["available_actions", "bring_to_front", "plan_zorder", "send_to_back",
"set_topmost"]
69 changes: 69 additions & 0 deletions je_auto_control/utils/window_zorder/window_zorder.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
"""Window z-order control — always-on-top / topmost, bring-to-front, send-to-back.

``windows_window_manage.set_window_position(hwnd, position)`` exists at the raw Win32
layer but is **not** exported in the package facade, has no title-based wrapper, and no
topmost / not-topmost semantics — and there is no ``always_on_top`` anywhere outside
GUI overlay code. This adds the standard RPA z-order primitive: a pure ``plan_zorder``
that maps an action to the ``SetWindowPos`` insert-after constant, plus title-based
``set_topmost`` / ``bring_to_front`` / ``send_to_back`` over an injectable driver (the
same seam pattern as ``snap_window``).

The planning is pure and headless-testable; only the default driver touches Win32
(returning ``False`` on other platforms). Imports no ``PySide6``.
"""
import sys
from typing import Any, Callable, Dict

ZOrderDriver = Callable[[str, str], bool]

# action -> SetWindowPos hwndInsertAfter constant
_ACTIONS = {"top": 0, "bottom": 1, "topmost": -1, "notopmost": -2}


def available_actions() -> list:
"""Return the supported z-order action names."""
return list(_ACTIONS)


def plan_zorder(action: str) -> Dict[str, Any]:
"""Return the ``SetWindowPos`` descriptor for a z-order ``action``.

``action`` is one of ``top`` / ``bottom`` / ``topmost`` / ``notopmost``; the
result carries the ``insert_after`` HWND constant and the move/size-preserving
flags. Raises ``ValueError`` for an unknown action.
"""
if action not in _ACTIONS:
raise ValueError(f"unknown z-order action: {action!r}")
return {"action": action, "insert_after": _ACTIONS[action],
"flags": ["SWP_NOMOVE", "SWP_NOSIZE"]}


def _default_driver(title: str, action: str) -> bool:
"""Apply a z-order action to the window matching ``title`` (Win32 only)."""
if not sys.platform.startswith("win"):
return False
from je_auto_control.wrapper.auto_control_window import find_window
hit = find_window(title)
if hit is None:
return False
from je_auto_control.windows.window import windows_window_manage as wm
wm.set_window_position(int(hit[0]), plan_zorder(action)["insert_after"])
return True


def set_topmost(title: str, on: bool = True, *,
driver: Callable[[str, str], bool] = None) -> bool:
"""Pin the window matching ``title`` always-on-top (or release it when ``on`` is False)."""
return (driver or _default_driver)(title, "topmost" if on else "notopmost")


def bring_to_front(title: str, *,
driver: Callable[[str, str], bool] = None) -> bool:
"""Raise the window matching ``title`` to the top of the z-order."""
return (driver or _default_driver)(title, "top")


def send_to_back(title: str, *,
driver: Callable[[str, str], bool] = None) -> bool:
"""Send the window matching ``title`` to the bottom of the z-order."""
return (driver or _default_driver)(title, "bottom")
Loading
Loading