Skip to content

Commit 11d58ea

Browse files
authored
Merge pull request #441 from Integration-Automation/dev
Release: input-fidelity lane (retry budget, field verify, adaptive timeout, ensure state, app idle) v209-v213
2 parents 1629faf + 1d51f60 commit 11d58ea

31 files changed

Lines changed: 2075 additions & 0 deletions

WHATS_NEW.md

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,36 @@
22

33
## What's new (2026-06-26)
44

5+
### Wait Until the App Is Idle
6+
7+
Hold off the next click until the busy/wait cursor settles — don't act mid-churn. Full reference: [`docs/source/Eng/doc/new_features/v213_features_doc.rst`](docs/source/Eng/doc/new_features/v213_features_doc.rst).
8+
9+
- **`wait_until_app_idle` / `idle_point`** (`AC_wait_until_app_idle`, `AC_idle_point`): a click fired while the app is still churning (busy cursor up, dialog mid-paint, long handler running) is dropped or mis-targeted. `smart_waits` watches *pixels* settle; this watches the app's *busy signal* settle, which is cheaper and survives animated-but-idle UI. It reuses `settle_detector.SettleTracker` — each poll feeds 1.0 when busy / 0.0 when idle, and returns once the app has read idle for `quiet_samples` polls in a row (a busy spike resets the run). `wait_until_app_idle` polls an injectable `busy_probe` (default = Windows busy/app-starting cursor) with injectable `clock`/`sleep`; `idle_point` is the pure analyser over a recorded busy/idle trace. Fully testable without an app. Fifth feature of the ROUND-15 input-fidelity lane. No `PySide6`.
10+
11+
### Ensure a Control Is in the Desired State (Idempotent)
12+
13+
Read-compare-act-verify instead of acting blind — don't double-toggle an already-checked box. Full reference: [`docs/source/Eng/doc/new_features/v212_features_doc.rst`](docs/source/Eng/doc/new_features/v212_features_doc.rst).
14+
15+
- **`ensure_state` / `ensure_toggle`** (`AC_ensure_field_value`): automation that acts *unconditionally* double-toggles a box that was already checked or re-enters an already-correct field, and can't be safely re-run. The robust shape is read-compare-act-verify. `ensure_state` reads via an injectable `reader`, and only if it doesn't equal `desired` applies `setter` and re-reads (up to `attempts`); `ensure_toggle` is the boolean specialization that calls `toggle` only while the state differs. A control already in the desired state is left untouched (`changed=False`) — idempotent and safe to re-run, distinct from `idempotency` (a request-key replay cache) since this converges *device state*. The executor's `AC_ensure_field_value` idempotently sets a native control's value via the accessibility backend. Fourth feature of the ROUND-15 input-fidelity lane. No `PySide6`.
16+
17+
### Adaptive Timeout from Observed Durations
18+
19+
Stop guessing wait timeouts — learn them from how long the step actually takes. Full reference: [`docs/source/Eng/doc/new_features/v211_features_doc.rst`](docs/source/Eng/doc/new_features/v211_features_doc.rst).
20+
21+
- **`recommend_timeout` / `timeout_stats`** (`AC_adaptive_timeout`, `AC_timeout_stats`): hard-coded waits are a perennial flakiness source — too short races a slow machine, too long makes every failure pay the full timeout. This learns the timeout from observed step durations: a high percentile (the slow-but-real case) scaled by a safety `factor`, clamped to a sane `[min_s, max_s]` band. `recommend_timeout` is the single number to feed a `wait_for_*` / actionability `GateConfig`; `timeout_stats` also exposes the percentiles and `floored`/`capped` flags for tuning. Both are pure and reuse `stats.percentile`; with no samples they fall back to `default_s`. Third feature of the ROUND-15 input-fidelity lane. No `PySide6`.
22+
23+
### Verify a Field After Typing
24+
25+
Read the field back and confirm the value actually landed — don't type and hope. Full reference: [`docs/source/Eng/doc/new_features/v210_features_doc.rst`](docs/source/Eng/doc/new_features/v210_features_doc.rst).
26+
27+
- **`compare_field_value` / `verify_field_value` / `fill_and_verify`** (`AC_compare_field_value`, `AC_verify_field_value`): `field_entry` types into a control and *hopes* — a slow IME, focus steal, input mask or auto-format can silently mangle or drop characters, and nothing reads the field back. This is distinct from `action_effect` (did *anything* change near the target?) and `postcondition.text_present` (does the text appear *anywhere*?) — neither confirms *this* field equals *this* value. `compare_field_value` is the pure comparator (`exact`/`trim`/`ci`/`normalized` NFKC/`contains`); `verify_field_value` reads through an injectable `reader` (native accessibility value in the executor); `fill_and_verify` types via an injectable `filler`, reads back, and retries (optionally clearing first) until it matches or attempts run out. Every comparison and retry decision is pure and unit-tested without a real control. Second feature of the ROUND-15 input-fidelity lane. No `PySide6`.
28+
29+
### Retry Budget — Deadline + Jitter
30+
31+
Retry a flaky step bounded by a total time budget, with jittered backoff. Full reference: [`docs/source/Eng/doc/new_features/v209_features_doc.rst`](docs/source/Eng/doc/new_features/v209_features_doc.rst).
32+
33+
- **`RetryBudget` / `run_with_budget` / `backoff_delay` / `jittered_delay`** (`AC_retry_delay`, `AC_plan_retry_delays`): `resilience.RetryPolicy` retries a fixed attempt count with plain exponential backoff — it can't express a *wall-clock deadline* ("give up after 30 s total, however many attempts that is") or *jitter* (randomized backoff so retrying workers don't resynchronize into a thundering herd). `RetryBudget` adds both: bounded by `max_attempts` *and/or* `deadline_s`, `run_with_budget` honours whichever is hit first and never sleeps past the deadline; delays use capped exponential backoff with a selectable `full`/`equal`/`none` jitter strategy. The randomness (`uniform`), clock and sleeper are all injectable, so every delay and giveup decision is deterministic in tests. First feature of the ROUND-15 input-fidelity lane. No `PySide6`.
34+
535
### Live IME State for Safe CJK Entry
636

737
Wait for the input method to commit before reading a Japanese/Chinese/Korean field. Full reference: [`docs/source/Eng/doc/new_features/v208_features_doc.rst`](docs/source/Eng/doc/new_features/v208_features_doc.rst).
Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
Retry Budget — Deadline + Jitter
2+
================================
3+
4+
:class:`resilience.RetryPolicy` retries a fixed number of attempts with plain
5+
exponential backoff. Two things it can't express are exactly what flaky,
6+
contended UI automation needs:
7+
8+
* a **wall-clock deadline** — "keep retrying, but give up after 30 s total",
9+
independent of how many attempts that takes; and
10+
* **jitter** — randomized backoff so many retrying workers don't resynchronize
11+
into a thundering herd.
12+
13+
``retry_budget`` adds both. :class:`RetryBudget` is bounded by ``max_attempts``
14+
*and / or* ``deadline_s``; :func:`run_with_budget` honours whichever is hit
15+
first and never sleeps past the deadline. Delays use capped exponential backoff
16+
with a selectable jitter strategy (``full`` / ``equal`` / ``none``). The
17+
randomness source (``uniform``), the clock and the sleeper are all injectable,
18+
so every delay and decision is deterministic in tests. Imports no ``PySide6``.
19+
20+
Headless API
21+
------------
22+
23+
.. code-block:: python
24+
25+
from je_auto_control import RetryBudget, run_with_budget
26+
27+
budget = RetryBudget(max_attempts=8, deadline_s=30.0,
28+
base_delay_s=0.2, max_delay_s=5.0)
29+
30+
# Retry the click until it lands, capped at 8 tries OR 30 seconds total
31+
run_with_budget(lambda: click_and_verify("Save"), budget)
32+
33+
``RetryBudget`` is bounded by attempts and / or a deadline — set either to
34+
``None`` to bound only by the other. :func:`backoff_delay` (pure, no jitter) and
35+
:meth:`RetryBudget.plan` give the delay schedule for inspection:
36+
37+
.. code-block:: python
38+
39+
RetryBudget(jitter="none").plan(4) # [0.1, 0.2, 0.4, 0.8]
40+
41+
For deterministic tests inject ``uniform`` / ``clock`` / ``sleep``:
42+
43+
.. code-block:: python
44+
45+
run_with_budget(flaky, budget, clock=fake_clock, sleep=fake_sleep,
46+
uniform=lambda lo, hi: lo) # always the low bound
47+
48+
Executor commands
49+
-----------------
50+
51+
``AC_retry_delay`` (``attempt`` / ``base`` / ``max_delay`` / ``multiplier`` /
52+
``jitter`` → ``{delay}``) and ``AC_plan_retry_delays`` (``attempts`` … →
53+
``{delays}``) expose the pure backoff schedule (``jitter`` defaults to ``none``
54+
for a deterministic result). They are the matching read-only ``ac_*`` MCP tools
55+
and Script Builder commands under **Flow**. :func:`run_with_budget` (which wraps
56+
a callable) is the Python-API surface.
Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
Verify a Field After Typing
2+
===========================
3+
4+
``field_entry`` types into a control and *hopes* it landed. A slow IME, a focus
5+
steal, an input mask or an auto-format can silently mangle or drop characters,
6+
and nothing reads the field back to notice. This is distinct from
7+
``action_effect`` (did *anything* change near the target?) and
8+
``postcondition.text_present`` (does the text appear *anywhere* on screen?) —
9+
neither confirms *this* field now equals *this* value. ``verify_field`` closes
10+
the read-back gap.
11+
12+
* :func:`compare_field_value` — pure: compare an expected and actual value under
13+
a match ``mode`` — ``exact`` / ``trim`` / ``ci`` (case-insensitive) /
14+
``normalized`` (Unicode NFKC + case-fold + whitespace) / ``contains``.
15+
* :func:`verify_field_value` — read the field through an injectable ``reader``
16+
and compare.
17+
* :func:`fill_and_verify` — type through an injectable ``filler``, read back, and
18+
retry (optionally clearing first) until it matches or attempts run out.
19+
20+
In the executor the reader is the native accessibility value, but every
21+
comparison and retry decision is pure and testable without a real control.
22+
Imports no ``PySide6``.
23+
24+
Headless API
25+
------------
26+
27+
.. code-block:: python
28+
29+
from je_auto_control import (
30+
compare_field_value, verify_field_value, fill_and_verify,
31+
)
32+
33+
compare_field_value("café", "café", mode="normalized")["match"] # True
34+
35+
# Read a control back and assert it took the value
36+
ok = verify_field_value("invoice.pdf",
37+
reader=lambda: read_control_value())["match"]
38+
39+
# Type, read back, and retry up to 3 times (clearing before each retry)
40+
fill_and_verify("2026-06-26", filler=type_into_field,
41+
reader=read_control_value, attempts=3, clear=select_all_del)
42+
43+
``fill_and_verify`` returns the final :func:`compare_field_value` result plus an
44+
``attempts`` count, so a flow can branch on a persistent mismatch instead of
45+
typing blind. ``filler`` / ``reader`` / ``clear`` are injectable, so the retry
46+
logic is fully unit-tested without a real field.
47+
48+
Executor commands
49+
-----------------
50+
51+
``AC_compare_field_value`` (``expected`` / ``actual`` / ``mode`` → ``{match,
52+
mode, expected, actual}``, pure) and ``AC_verify_field_value`` (``expected`` +
53+
``name`` / ``role`` / ``app_name`` / ``automation_id`` / ``mode`` → the match
54+
result, reading the control's value through the accessibility backend). They are
55+
the matching read-only ``ac_*`` MCP tools and Script Builder commands under
56+
**Flow**. :func:`fill_and_verify` (which wraps a typing callable) is the
57+
Python-API surface.
Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
Adaptive Timeout from Observed Durations
2+
========================================
3+
4+
Hard-coded waits are a perennial source of flakiness: too short and a slow
5+
machine races the UI; too long and every failure pays the full timeout. The
6+
durable fix is to *learn* the timeout from how long a step has actually taken.
7+
``adaptive_timeout`` turns a sample of observed durations into a robust timeout
8+
— a high percentile (the slow-but-real case) scaled by a safety ``factor``,
9+
then clamped to a sane ``[min_s, max_s]`` band.
10+
11+
* :func:`recommend_timeout` — the single number to feed a wait or ``GateConfig``.
12+
* :func:`timeout_stats` — the same with the percentiles and clamp flags exposed
13+
for logging / tuning.
14+
15+
Both are pure and reuse :func:`stats.percentile`; with no samples they fall back
16+
to ``default_s`` (or ``min_s``). Imports no ``PySide6``.
17+
18+
Headless API
19+
------------
20+
21+
.. code-block:: python
22+
23+
from je_auto_control import recommend_timeout, timeout_stats
24+
25+
# The dialog has historically taken these seconds to appear:
26+
seen = [0.8, 1.1, 0.9, 3.2, 1.0, 1.3]
27+
28+
recommend_timeout(seen) # ~ p95 * 1.5, clamped to [1, 60]
29+
recommend_timeout(seen, percentile_q=99.0, factor=2.0, max_s=30.0)
30+
31+
timeout_stats(seen)
32+
# {'n': 6, 'p50': 1.05, 'p_high': 2.7..., 'percentile_q': 95.0,
33+
# 'recommended': 4.1..., 'floored': False, 'capped': False}
34+
35+
Use the recommendation as the ``timeout_s`` for the next ``wait_for_*`` /
36+
actionability gate, recomputing it as the duration sample grows. With no samples
37+
yet, pass ``default_s`` for the cold-start value.
38+
39+
Executor commands
40+
-----------------
41+
42+
``AC_adaptive_timeout`` (``durations`` + ``percentile_q`` / ``factor`` /
43+
``min_s`` / ``max_s`` → ``{timeout_s}``) and ``AC_timeout_stats`` (same inputs →
44+
``{n, p50, p_high, percentile_q, recommended, floored, capped}``). ``durations``
45+
accepts a JSON list. They are the matching read-only ``ac_*`` MCP tools and
46+
Script Builder commands under **Flow**.
Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
Ensure a Control Is in the Desired State (Idempotent)
2+
=====================================================
3+
4+
Automation that *acts unconditionally* — "click the checkbox", "type the value"
5+
— double-toggles a box that was already checked, or re-enters a field that was
6+
already correct, and can't be safely re-run. The robust shape is
7+
read-compare-act-verify: look at the current state, do nothing if it already
8+
matches, otherwise apply the change and confirm it took. ``ensure_state`` is
9+
that primitive.
10+
11+
* :func:`ensure_state` — generic: read via ``reader``, and if it doesn't equal
12+
``desired`` apply ``setter`` and re-read, up to ``attempts`` times.
13+
* :func:`ensure_toggle` — the boolean specialization for a stateless flip: read
14+
``is_on`` and call ``toggle`` only while it differs from ``desired``.
15+
16+
A control already in the desired state is left untouched (``changed=False``), so
17+
the call is idempotent and safe to re-run. This is distinct from
18+
:mod:`idempotency` (a request-key replay cache) — ``ensure_state`` converges
19+
*device state*, not call results. The reader / setter / toggle seams are
20+
injectable, so the logic is fully testable without a real control. Imports no
21+
``PySide6``.
22+
23+
Headless API
24+
------------
25+
26+
.. code-block:: python
27+
28+
from je_auto_control import ensure_state, ensure_toggle
29+
30+
# Idempotently make a setting "on" — no write if it already is
31+
ensure_state("on", reader=read_combo, setter=write_combo)
32+
# -> {'ok': True, 'changed': False, 'value': 'on', 'attempts': 0}
33+
34+
# Flip a checkbox to checked only if it isn't already
35+
ensure_toggle(True, is_on=is_checked, toggle=click_checkbox)
36+
37+
Both return ``{ok, changed, value, attempts}``: ``changed`` tells you whether an
38+
action was actually performed (useful for "did I have to fix this?" reporting),
39+
and ``ok`` whether the desired state was reached within ``attempts``. Pass a
40+
custom ``equals`` to :func:`ensure_state` for case-insensitive or normalized
41+
comparisons.
42+
43+
Executor commands
44+
-----------------
45+
46+
``AC_ensure_field_value`` (``desired`` + ``name`` / ``role`` / ``app_name`` /
47+
``automation_id`` / ``attempts`` → ``{ok, changed, value, attempts}``)
48+
idempotently sets a native control's value through the accessibility backend —
49+
reading first and doing nothing if it already matches. It is the matching
50+
``ac_ensure_field_value`` MCP tool and a Script Builder command under **Flow**.
51+
:func:`ensure_state` / :func:`ensure_toggle` (which take arbitrary callables) are
52+
the Python-API surface.
Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
Wait Until the App Is Idle
2+
==========================
3+
4+
A click fired while the app is still churning — the busy / wait cursor is up, a
5+
dialog is mid-paint, a long handler is running — is dropped or mis-targeted.
6+
``smart_waits`` watches *pixels* settling; ``app_idle`` watches the app's *busy
7+
signal* settle instead, which is cheaper and survives animated-but-idle UI. It
8+
reuses :class:`settle_detector.SettleTracker`: each poll feeds ``1.0`` when busy
9+
and ``0.0`` when idle, and the wait returns once the app has read idle for
10+
``quiet_samples`` polls in a row (a fresh busy spike resets the run).
11+
12+
* :func:`wait_until_app_idle` — poll a ``busy_probe`` until the app settles idle
13+
or a timeout, with injectable ``clock`` / ``sleep`` / ``busy_probe``.
14+
* :func:`idle_point` — pure: the index in a recorded busy/idle sample series at
15+
which it first becomes settled-idle.
16+
17+
The default probe reports the Windows busy / app-starting cursor; every wait and
18+
settle decision runs through the injectable seam, so it is fully testable
19+
without an app. Imports no ``PySide6``.
20+
21+
Headless API
22+
------------
23+
24+
.. code-block:: python
25+
26+
from je_auto_control import wait_until_app_idle, idle_point
27+
28+
# Launch something, then wait for its busy cursor to settle before clicking
29+
start_exe("setup.exe")
30+
if wait_until_app_idle(quiet_samples=3, timeout_s=30)["idle"]:
31+
click_next()
32+
33+
# Pure: analyse a recorded busy/idle trace
34+
idle_point([True, True, False, False, False], quiet_samples=3) # 4
35+
36+
``wait_until_app_idle`` returns ``{idle, polls, quiet_run, elapsed_s}``. Pass a
37+
custom ``busy_probe`` (a ``() -> bool``) to gate on any busy signal — a spinner
38+
image match, a process-CPU threshold, an accessibility "busy" flag — not just the
39+
cursor.
40+
41+
Executor commands
42+
-----------------
43+
44+
``AC_wait_until_app_idle`` (``quiet_samples`` / ``timeout`` / ``interval`` →
45+
``{idle, polls, quiet_run, elapsed_s}``, using the Windows busy cursor) and
46+
``AC_idle_point`` (``busy_samples`` JSON list + ``quiet_samples`` → ``{index}``,
47+
pure). They are the matching read-only ``ac_*`` MCP tools and Script Builder
48+
commands under **Flow**.
Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
重試預算——截止時間 + 抖動
2+
==========================
3+
4+
:class:`resilience.RetryPolicy` 以固定次數搭配單純指數退避重試。有兩件它無法表達的事,正是不穩定、
5+
高競爭的 UI 自動化所需:
6+
7+
* **掛鐘截止時間**——「持續重試,但總共超過 30 秒就放棄」,與嘗試了幾次無關;以及
8+
* **抖動(jitter)**——隨機化退避,讓眾多重試中的工作者不會重新同步成驚群效應。
9+
10+
``retry_budget`` 兩者皆補上。:class:`RetryBudget` 由 ``max_attempts`` *與 / 或* ``deadline_s``
11+
界定;:func:`run_with_budget` 以先達到者為準,且絕不會睡過截止時間。延遲採用有上限的指數退避,
12+
搭配可選的抖動策略(``full`` / ``equal`` / ``none``)。隨機來源(``uniform``)、時鐘與睡眠器
13+
皆可注入,故每個延遲與決策在測試中都是確定的。不匯入 ``PySide6``。
14+
15+
無頭 API
16+
--------
17+
18+
.. code-block:: python
19+
20+
from je_auto_control import RetryBudget, run_with_budget
21+
22+
budget = RetryBudget(max_attempts=8, deadline_s=30.0,
23+
base_delay_s=0.2, max_delay_s=5.0)
24+
25+
# 重試點擊直到成功,上限為 8 次嘗試 或 總共 30 秒
26+
run_with_budget(lambda: click_and_verify("Save"), budget)
27+
28+
``RetryBudget`` 由嘗試次數與 / 或截止時間界定——把其一設為 ``None`` 即只以另一者界定。
29+
:func:`backoff_delay`(純函式,無抖動)與 :meth:`RetryBudget.plan` 提供延遲排程以供檢視:
30+
31+
.. code-block:: python
32+
33+
RetryBudget(jitter="none").plan(4) # [0.1, 0.2, 0.4, 0.8]
34+
35+
確定性測試可注入 ``uniform`` / ``clock`` / ``sleep``:
36+
37+
.. code-block:: python
38+
39+
run_with_budget(flaky, budget, clock=fake_clock, sleep=fake_sleep,
40+
uniform=lambda lo, hi: lo) # 永遠取下界
41+
42+
執行器指令
43+
----------
44+
45+
``AC_retry_delay``(``attempt`` / ``base`` / ``max_delay`` / ``multiplier`` /
46+
``jitter`` → ``{delay}``)與 ``AC_plan_retry_delays``(``attempts`` … →
47+
``{delays}``)暴露純退避排程(``jitter`` 預設為 ``none`` 以得確定結果)。皆以對應的唯讀
48+
``ac_*`` MCP 工具及 Script Builder 指令(位於 **Flow** 分類下)形式提供。
49+
:func:`run_with_budget`(包裹一個 callable)則是 Python API 介面。

0 commit comments

Comments
 (0)