Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions README/WHATS_NEW_zh-CN.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
# 本次更新 — AutoControl

## 本次更新 (2026-06-23) — 等待消失(阻塞式 vanish 等待)

阻塞直到转圈圈 / toast / 对话框消失。完整参考:[`docs/source/Zh/doc/new_features/v118_features_doc.rst`](../docs/source/Zh/doc/new_features/v118_features_doc.rst)。

- **`wait_until_gone` / `wait_until_image_gone` / `wait_until_text_gone`**(`AC_wait_image_gone`、`AC_wait_text_gone`):`wait_for_image`/`wait_for_text` 只阻塞到某物*出现*,`observer` 则以异步回调在消失时触发——先前没有*阻塞式*的「等到此图像/文本消失再继续」。通用的 `wait_until_gone` 接受任意谓词(可无头测试);图像/文本辅助函数从定位函数建立。`gone_for_s` 可消抖。返回 `WaitOutcome`。纯标准库。

## 本次更新 (2026-06-23) — 清空再输入字段

可靠地设定文本字段的值(Playwright 的 `fill` 惯用法)。完整参考:[`docs/source/Zh/doc/new_features/v117_features_doc.rst`](../docs/source/Zh/doc/new_features/v117_features_doc.rst)。
Expand Down
6 changes: 6 additions & 0 deletions README/WHATS_NEW_zh-TW.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
# 本次更新 — AutoControl

## 本次更新 (2026-06-23) — 等待消失(阻塞式 vanish 等待)

阻塞直到轉圈圈 / toast / 對話框消失。完整參考:[`docs/source/Zh/doc/new_features/v118_features_doc.rst`](../docs/source/Zh/doc/new_features/v118_features_doc.rst)。

- **`wait_until_gone` / `wait_until_image_gone` / `wait_until_text_gone`**(`AC_wait_image_gone`、`AC_wait_text_gone`):`wait_for_image`/`wait_for_text` 只阻塞到某物*出現*,`observer` 則以非同步回呼在消失時觸發——先前沒有*阻塞式*的「等到此影像/文字消失再繼續」。通用的 `wait_until_gone` 接受任意述詞(可無頭測試);影像/文字輔助函式從定位函式建立。`gone_for_s` 可消抖。回傳 `WaitOutcome`。純標準函式庫。

## 本次更新 (2026-06-23) — 清空再輸入欄位

可靠地設定文字欄位的值(Playwright 的 `fill` 慣用法)。完整參考:[`docs/source/Zh/doc/new_features/v117_features_doc.rst`](../docs/source/Zh/doc/new_features/v117_features_doc.rst)。
Expand Down
6 changes: 6 additions & 0 deletions WHATS_NEW.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
# What's New — AutoControl

## What's new (2026-06-23) — Wait Until Gone (Blocking Vanish Waits)

Block until a spinner / toast / dialog disappears. Full reference: [`docs/source/Eng/doc/new_features/v118_features_doc.rst`](docs/source/Eng/doc/new_features/v118_features_doc.rst).

- **`wait_until_gone` / `wait_until_image_gone` / `wait_until_text_gone`** (`AC_wait_image_gone`, `AC_wait_text_gone`): `wait_for_image`/`wait_for_text` only block until something *appears*, and `observer` fires async callbacks on vanish — there was no *blocking* "wait until this image/text disappears then continue" call. The generic `wait_until_gone` takes any predicate (headless-testable); the image/text helpers build it from the locate functions. `gone_for_s` debounces flicker. Returns a `WaitOutcome`. Pure-stdlib.

## What's new (2026-06-23) — Clear-Then-Type Field Entry

Reliably set a text field's value (the Playwright `fill` idiom). Full reference: [`docs/source/Eng/doc/new_features/v117_features_doc.rst`](docs/source/Eng/doc/new_features/v117_features_doc.rst).
Expand Down
41 changes: 41 additions & 0 deletions docs/source/Eng/doc/new_features/v118_features_doc.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
Wait Until Gone (Blocking Vanish Waits)
=======================================

``wait_for_image`` / ``wait_for_text`` block until something *appears*, and the
``observer`` fires async callbacks on vanish — but there was no *blocking* "wait
until this spinner / toast / dialog **disappears** then continue" call for an
image or text. ``wait_until_window_closed`` covers windows only. This adds the
missing vanish waits to the ``smart_waits`` family.

The generic :func:`wait_until_gone` takes any predicate, so its loop is
headless-testable without a real screen; the image / text helpers build that
predicate from the locate functions. Imports no ``PySide6``.

Headless API
------------

.. code-block:: python

from je_auto_control import (
wait_until_gone, wait_until_image_gone, wait_until_text_gone,
)

# generic: wait until any predicate has been falsey
wait_until_gone(lambda: spinner_is_visible(), timeout_s=15)

wait_until_image_gone("spinner.png", timeout_s=15) # image left the screen
wait_until_text_gone("Loading...", timeout_s=15) # OCR text disappeared

Each returns a ``WaitOutcome`` (``succeeded`` / ``reason`` / ``elapsed_s`` /
``samples_taken``) — the same result type as the other smart waits. ``gone_for_s``
requires the target to stay absent for that long before succeeding (debounces a
flickering element); ``poll_interval_s`` / ``timeout_s`` bound the loop.

Executor commands
-----------------

``AC_wait_image_gone`` and ``AC_wait_text_gone`` take the target plus
``timeout_s`` / ``poll_interval_s`` / ``gone_for_s`` (and ``detect_threshold`` for
the image) and return the ``WaitOutcome`` dict. Both are exposed as MCP tools
(``ac_wait_image_gone`` / ``ac_wait_text_gone``) and as Script Builder commands
under **Flow**.
1 change: 1 addition & 0 deletions docs/source/Eng/eng_index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -140,6 +140,7 @@ Comprehensive guides for all AutoControl features.
doc/new_features/v115_features_doc
doc/new_features/v116_features_doc
doc/new_features/v117_features_doc
doc/new_features/v118_features_doc
doc/ocr_backends/ocr_backends_doc
doc/observability/observability_doc
doc/operations_layer/operations_layer_doc
Expand Down
35 changes: 35 additions & 0 deletions docs/source/Zh/doc/new_features/v118_features_doc.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
等待消失(阻塞式 vanish 等待)
==============================

``wait_for_image`` / ``wait_for_text`` 會阻塞直到某物*出現*,``observer`` 則以非同步回呼在消失時觸發——但先前
沒有針對影像或文字的*阻塞式*「等到這個轉圈圈 / toast / 對話框**消失**再繼續」呼叫。``wait_until_window_closed``
只涵蓋視窗。本功能為 ``smart_waits`` 家族補上缺少的 vanish 等待。

通用的 :func:`wait_until_gone` 接受任意述詞,因此其迴圈可在無真實螢幕下做無頭測試;影像 / 文字輔助函式則
從定位函式建立該述詞。不匯入 ``PySide6``。

無頭 API
--------

.. code-block:: python

from je_auto_control import (
wait_until_gone, wait_until_image_gone, wait_until_text_gone,
)

# 通用:等到任意述詞變為 falsey
wait_until_gone(lambda: spinner_is_visible(), timeout_s=15)

wait_until_image_gone("spinner.png", timeout_s=15) # 影像離開螢幕
wait_until_text_gone("Loading...", timeout_s=15) # OCR 文字消失

每個皆回傳 ``WaitOutcome``(``succeeded`` / ``reason`` / ``elapsed_s`` / ``samples_taken``)——與其他 smart
waits 相同的結果型別。``gone_for_s`` 要求目標需持續缺席該段時間才算成功(可消抖閃爍的元素);
``poll_interval_s`` / ``timeout_s`` 界定迴圈。

執行器命令
----------

``AC_wait_image_gone`` 與 ``AC_wait_text_gone`` 接受目標以及 ``timeout_s`` / ``poll_interval_s`` /
``gone_for_s``(影像另含 ``detect_threshold``),並回傳 ``WaitOutcome`` dict。兩者皆以 MCP 工具
(``ac_wait_image_gone`` / ``ac_wait_text_gone``)以及 Script Builder 中 **Flow** 分類下的命令提供。
1 change: 1 addition & 0 deletions docs/source/Zh/zh_index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -140,6 +140,7 @@ AutoControl 所有功能的完整使用指南。
doc/new_features/v115_features_doc
doc/new_features/v116_features_doc
doc/new_features/v117_features_doc
doc/new_features/v118_features_doc
doc/ocr_backends/ocr_backends_doc
doc/observability/observability_doc
doc/operations_layer/operations_layer_doc
Expand Down
6 changes: 4 additions & 2 deletions je_auto_control/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -605,8 +605,9 @@
# Smart waits (frame-diff replacements for time.sleep)
from je_auto_control.utils.smart_waits import (
WaitOutcome, wait_until_clipboard_changes, wait_until_file,
wait_until_pixel_changes, wait_until_port, wait_until_process,
wait_until_region_idle, wait_until_screen_stable, wait_until_window_closed,
wait_until_gone, wait_until_image_gone, wait_until_pixel_changes,
wait_until_port, wait_until_process, wait_until_region_idle,
wait_until_screen_stable, wait_until_text_gone, wait_until_window_closed,
)
# Visual regression (golden-image comparison)
from je_auto_control.utils.visual_regression import (
Expand Down Expand Up @@ -1230,6 +1231,7 @@ def start_autocontrol_gui(*args, **kwargs):
"wait_until_region_idle", "wait_until_screen_stable",
"wait_until_clipboard_changes", "wait_until_window_closed",
"wait_until_file", "wait_until_port", "wait_until_process",
"wait_until_gone", "wait_until_image_gone", "wait_until_text_gone",
# Visual regression + state machine
"take_golden", "compare_to_golden", "image_difference",
"DiffResult", "MaskRegion",
Expand Down
24 changes: 24 additions & 0 deletions je_auto_control/gui/script_builder/command_schema.py
Original file line number Diff line number Diff line change
Expand Up @@ -443,6 +443,30 @@ def _add_flow_specs(specs: List[CommandSpec]) -> None:
),
description="Wait until the clipboard changes or matches target.",
))
specs.append(CommandSpec(
"AC_wait_image_gone", "Flow", "Wait for Image to Vanish",
fields=(
FieldSpec("image", FieldType.STRING, placeholder="path/to/spinner.png"),
FieldSpec("detect_threshold", FieldType.FLOAT, optional=True,
default=1.0),
FieldSpec("timeout_s", FieldType.FLOAT, optional=True, default=10.0),
FieldSpec("poll_interval_s", FieldType.FLOAT, optional=True,
default=0.2, min_value=0.01),
FieldSpec("gone_for_s", FieldType.FLOAT, optional=True, default=0.0),
),
description="Block until an image (spinner/toast) leaves the screen.",
))
specs.append(CommandSpec(
"AC_wait_text_gone", "Flow", "Wait for Text to Vanish",
fields=(
FieldSpec("text", FieldType.STRING, placeholder="Loading..."),
FieldSpec("timeout_s", FieldType.FLOAT, optional=True, default=10.0),
FieldSpec("poll_interval_s", FieldType.FLOAT, optional=True,
default=0.2, min_value=0.01),
FieldSpec("gone_for_s", FieldType.FLOAT, optional=True, default=0.0),
),
description="Block until on-screen text (OCR) disappears.",
))
specs.append(CommandSpec(
"AC_loop", "Flow", "Loop (N times)",
fields=(
Expand Down
25 changes: 25 additions & 0 deletions je_auto_control/utils/executor/action_executor.py
Original file line number Diff line number Diff line change
Expand Up @@ -361,6 +361,29 @@ def _wait_clipboard_change(baseline: Optional[str] = None,
).to_dict()


def _wait_image_gone(image: Any, detect_threshold: float = 1.0,
timeout_s: float = 10.0, poll_interval_s: float = 0.2,
gone_for_s: float = 0.0) -> Dict[str, Any]:
"""Executor adapter: wait until an image is no longer on screen."""
from je_auto_control.utils.smart_waits import wait_until_image_gone
return wait_until_image_gone(
image, detect_threshold=float(detect_threshold),
timeout_s=float(timeout_s), poll_interval_s=float(poll_interval_s),
gone_for_s=float(gone_for_s),
).to_dict()


def _wait_text_gone(text: str, timeout_s: float = 10.0,
poll_interval_s: float = 0.2,
gone_for_s: float = 0.0) -> Dict[str, Any]:
"""Executor adapter: wait until text is no longer on screen (OCR)."""
from je_auto_control.utils.smart_waits import wait_until_text_gone
return wait_until_text_gone(
text, timeout_s=float(timeout_s),
poll_interval_s=float(poll_interval_s), gone_for_s=float(gone_for_s),
).to_dict()


def _wait_window_closed(title: str, case_sensitive: bool = False,
timeout_s: float = 10.0,
poll_interval_s: float = 0.2) -> Dict[str, Any]:
Expand Down Expand Up @@ -4971,6 +4994,8 @@ def __init__(self):
"AC_wait_for_port": _wait_for_port,
"AC_wait_for_process": _wait_for_process,
"AC_wait_clipboard_change": _wait_clipboard_change,
"AC_wait_image_gone": _wait_image_gone,
"AC_wait_text_gone": _wait_text_gone,
"AC_wait_window_closed": _wait_window_closed,

# Cost telemetry (LLM token + USD tracking)
Expand Down
28 changes: 28 additions & 0 deletions je_auto_control/utils/mcp_server/tools/_factories.py
Original file line number Diff line number Diff line change
Expand Up @@ -1277,6 +1277,34 @@ def cost_telemetry_tools() -> List[MCPTool]:

def smart_wait_tools() -> List[MCPTool]:
return [
MCPTool(
name="ac_wait_image_gone",
description=("Block until 'image' is no longer found on screen "
"(spinner/toast/dialog vanished). 'detect_threshold', "
"'timeout_s', 'poll_interval_s', 'gone_for_s'."),
input_schema=schema({
"image": {"type": "string"},
"detect_threshold": {"type": "number"},
"timeout_s": {"type": "number"},
"poll_interval_s": {"type": "number"},
"gone_for_s": {"type": "number"}},
required=["image"]),
handler=h.wait_image_gone,
annotations=READ_ONLY,
),
MCPTool(
name="ac_wait_text_gone",
description=("Block until 'text' is no longer found on screen (OCR). "
"'timeout_s', 'poll_interval_s', 'gone_for_s'."),
input_schema=schema({
"text": {"type": "string"},
"timeout_s": {"type": "number"},
"poll_interval_s": {"type": "number"},
"gone_for_s": {"type": "number"}},
required=["text"]),
handler=h.wait_text_gone,
annotations=READ_ONLY,
),
MCPTool(
name="ac_wait_screen_stable",
description=("Block until the screen stops moving (consecutive "
Expand Down
15 changes: 15 additions & 0 deletions je_auto_control/utils/mcp_server/tools/_handlers.py
Original file line number Diff line number Diff line change
Expand Up @@ -2418,6 +2418,21 @@ def wait_screen_stable(region: Optional[List[int]] = None,
).to_dict()


def wait_image_gone(image, detect_threshold: float = 1.0,
timeout_s: float = 10.0, poll_interval_s: float = 0.2,
gone_for_s: float = 0.0) -> Dict[str, Any]:
from je_auto_control.utils.executor.action_executor import _wait_image_gone
return _wait_image_gone(image, detect_threshold, timeout_s,
poll_interval_s, gone_for_s)


def wait_text_gone(text: str, timeout_s: float = 10.0,
poll_interval_s: float = 0.2,
gone_for_s: float = 0.0) -> Dict[str, Any]:
from je_auto_control.utils.executor.action_executor import _wait_text_gone
return _wait_text_gone(text, timeout_s, poll_interval_s, gone_for_s)


def wait_for_file(path: str, timeout_s: float = 30.0,
poll_interval_s: float = 0.25,
stable_for_s: float = 1.0,
Expand Down
13 changes: 7 additions & 6 deletions je_auto_control/utils/smart_waits/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,17 +10,18 @@
from je_auto_control.utils.smart_waits.waits import (
ClipboardReader, FileStatReader, Frame, PortConnector, ProcessLister,
ScreenSampler, WaitOutcome, WindowFinder, wait_until_clipboard_changes,
wait_until_file, wait_until_pixel_changes, wait_until_port,
wait_until_process, wait_until_region_idle, wait_until_screen_stable,
wait_until_file, wait_until_gone, wait_until_image_gone,
wait_until_pixel_changes, wait_until_port, wait_until_process,
wait_until_region_idle, wait_until_screen_stable, wait_until_text_gone,
wait_until_window_closed,
)


__all__ = [
"ClipboardReader", "FileStatReader", "Frame", "PortConnector",
"ProcessLister", "ScreenSampler", "WaitOutcome", "WindowFinder",
"wait_until_clipboard_changes", "wait_until_file",
"wait_until_pixel_changes", "wait_until_port", "wait_until_process",
"wait_until_region_idle", "wait_until_screen_stable",
"wait_until_window_closed",
"wait_until_clipboard_changes", "wait_until_file", "wait_until_gone",
"wait_until_image_gone", "wait_until_pixel_changes", "wait_until_port",
"wait_until_process", "wait_until_region_idle", "wait_until_screen_stable",
"wait_until_text_gone", "wait_until_window_closed",
]
68 changes: 68 additions & 0 deletions je_auto_control/utils/smart_waits/waits.py
Original file line number Diff line number Diff line change
Expand Up @@ -378,6 +378,74 @@ def wait_until_process(name: str, *, present: bool = True,
started, samples)


def wait_until_gone(present: Callable[[], bool], *,
timeout_s: float = 10.0, poll_interval_s: float = 0.2,
gone_for_s: float = 0.0) -> WaitOutcome:
"""Return once ``present()`` has been falsey for ``gone_for_s`` seconds.

The blocking complement of ``wait_for_image`` / ``wait_for_text``: wait for a
spinner / toast / dialog to *disappear*. ``present`` is any predicate (e.g.
"is this image still on screen"); it is polled every ``poll_interval_s`` up to
``timeout_s``. Injecting ``present`` keeps the loop headless-testable.
"""
if timeout_s <= 0:
raise ValueError(_TIMEOUT_POSITIVE)
if poll_interval_s <= 0:
raise ValueError(_POLL_POSITIVE)
if gone_for_s < 0:
raise ValueError("gone_for_s must be >= 0")
started = time.monotonic()
deadline = started + float(timeout_s)
samples = 0
gone_since: Optional[float] = None
while time.monotonic() < deadline:
samples += 1
if present():
gone_since = None
else:
if gone_since is None:
gone_since = time.monotonic()
if time.monotonic() - gone_since >= float(gone_for_s):
return _finish(True, "target gone", started, samples)
time.sleep(float(poll_interval_s))
return _finish(False, "timeout while waiting for target to vanish",
started, samples)


def _image_present(image: Any, detect_threshold: float) -> bool:
"""Whether ``image`` is currently locatable on screen."""
from je_auto_control.utils.exception.exceptions import ImageNotFoundException
from je_auto_control.wrapper.auto_control_image import locate_image_center
try:
locate_image_center(image, detect_threshold=detect_threshold)
return True
except ImageNotFoundException:
return False


def wait_until_image_gone(image: Any, *, detect_threshold: float = 1.0,
timeout_s: float = 10.0, poll_interval_s: float = 0.2,
gone_for_s: float = 0.0) -> WaitOutcome:
"""Wait until ``image`` is no longer found on screen."""
return wait_until_gone(lambda: _image_present(image, detect_threshold),
timeout_s=timeout_s, poll_interval_s=poll_interval_s,
gone_for_s=gone_for_s)


def _text_present(text: str) -> bool:
"""Whether ``text`` is currently found on screen via OCR."""
from je_auto_control.utils.ocr.ocr_engine import find_text_matches
return bool(find_text_matches(text))


def wait_until_text_gone(text: str, *, timeout_s: float = 10.0,
poll_interval_s: float = 0.2,
gone_for_s: float = 0.0) -> WaitOutcome:
"""Wait until ``text`` is no longer found on screen (OCR)."""
return wait_until_gone(lambda: _text_present(text), timeout_s=timeout_s,
poll_interval_s=poll_interval_s, gone_for_s=gone_for_s)


def _default_process_lister(name: str) -> List[str]:
"""List running process names matching ``name`` (requires psutil)."""
from je_auto_control.utils.assertion.assertions import _running_process_names
Expand Down
Loading
Loading