Skip to content

Commit 2dd1b96

Browse files
authored
Merge pull request #434 from Integration-Automation/feat/ime-state-batch
Add ime_state: live IME composition state for safe CJK entry
2 parents 7f5fd42 + 70047df commit 2dd1b96

11 files changed

Lines changed: 528 additions & 0 deletions

File tree

WHATS_NEW.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,12 @@
22

33
## What's new (2026-06-26)
44

5+
### Live IME State for Safe CJK Entry
6+
7+
Wait for the input method to commit before reading a Japanese/Chinese/Korean field. Full reference: [`docs/source/Eng/doc/new_features/v208_features_doc.rst`](docs/source/Eng/doc/new_features/v208_features_doc.rst).
8+
9+
- **`ime_state` / `is_composing` / `wait_for_composition_commit` / `decode_conversion_mode`** (`AC_ime_state`, `AC_is_composing`, `AC_wait_for_composition_commit`, `AC_decode_conversion_mode`): typing into a CJK field is unsafe while an IME is *composing* — the candidate text isn't committed, so reading the field back returns half-entered glyphs and the next keystroke edits the composition. `text_unicode` (`VK_PACKET`) is blind to this. `ime_state` exposes the focused window's live `{open, composing, composition, conversion}` (Windows IMM32, read-only) through an injectable `reader`; `is_composing` is the boolean gate; `wait_for_composition_commit` blocks until the IME commits (injectable `clock`/`sleep`/`reader`); `decode_conversion_mode` is the pure `IME_CMODE_*` bitmask decoder. All decode/wait logic is unit-tested without an IME. Sixth feature of the ROUND-15 cross-app OS lane. No `PySide6`.
10+
511
### Lock the Workstation + Wait for Unlock
612

713
Lock the box at the end of a run, and block until a human unlocks it before resuming. Full reference: [`docs/source/Eng/doc/new_features/v207_features_doc.rst`](docs/source/Eng/doc/new_features/v207_features_doc.rst).
Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
Live IME State for Safe CJK Entry
2+
=================================
3+
4+
Typing into a CJK / Japanese / Korean field is unsafe while an IME (input method
5+
editor) is *composing*: the candidate text has not been committed yet, so
6+
reading the field back returns half-entered glyphs and the next keystroke edits
7+
the composition instead of the field. ``text_unicode`` (``VK_PACKET``) is blind
8+
to this. ``ime_state`` exposes the live composition and conversion state so a
9+
flow can wait for the IME to commit before it reads or acts.
10+
11+
* :func:`ime_state` — ``{open, composing, composition, conversion,
12+
conversion_flags}`` for the focused window's IME, through an injectable
13+
``reader``.
14+
* :func:`is_composing` — ``True`` while the IME has an uncommitted composition.
15+
* :func:`wait_for_composition_commit` — block until composition ends (or a
16+
timeout), with injectable ``clock`` / ``sleep`` / ``reader``.
17+
* :func:`decode_conversion_mode` — pure: the IMM32 ``IME_CMODE_*`` conversion
18+
bitmask to ``{native, katakana, full_shape, roman, char_code}``.
19+
20+
The default ``reader`` queries Windows IMM32 (``ImmGetContext`` /
21+
``ImmGetOpenStatus`` / ``ImmGetConversionStatus`` / ``ImmGetCompositionStringW``)
22+
read-only; all decoding / waiting logic runs through the injectable seam, so it
23+
is fully testable without an IME. Imports no ``PySide6``.
24+
25+
Headless API
26+
------------
27+
28+
.. code-block:: python
29+
30+
from je_auto_control import (
31+
ime_state, is_composing, wait_for_composition_commit,
32+
)
33+
34+
# Before reading a CJK field, make sure the IME has committed
35+
if wait_for_composition_commit(timeout_s=3):
36+
value = read_field()
37+
38+
is_composing() # True while candidate text is still on screen
39+
ime_state() # {'open': True, 'composing': True, 'composition': 'あ', ...}
40+
41+
For tests (or any non-Windows host) pass a ``reader`` — a
42+
``() -> {open, conversion, composition}``:
43+
44+
.. code-block:: python
45+
46+
busy = lambda: {"open": True, "conversion": 0, "composition": ""}
47+
is_composing(reader=busy) # True
48+
ime_state(reader=busy)["composition"] # 'あ'
49+
50+
Executor commands
51+
-----------------
52+
53+
``AC_ime_state`` (→ the full state), ``AC_is_composing`` (→ ``{composing}``),
54+
``AC_wait_for_composition_commit`` (``timeout`` / ``interval`` →
55+
``{committed}``) and ``AC_decode_conversion_mode`` (``flags`` → the decoded
56+
modes). They are exposed as the matching read-only ``ac_*`` MCP tools and as
57+
Script Builder commands under **Shell**.
Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
即時 IME 狀態以利安全的 CJK 輸入
2+
================================
3+
4+
在 IME(輸入法)*組字中*對 CJK / 日文 / 韓文欄位輸入並不安全:候選字尚未送出,故讀回欄位會得到
5+
半成形的字,而下一個按鍵會編輯組字而非欄位。``text_unicode``(``VK_PACKET``)對此一無所知。
6+
``ime_state`` 暴露即時的組字與轉換狀態,讓流程能在讀取或操作前等待 IME 送出。
7+
8+
* :func:`ime_state` ——聚焦視窗 IME 的 ``{open, composing, composition, conversion,
9+
conversion_flags}``,透過可注入的 ``reader``。
10+
* :func:`is_composing` ——當 IME 有尚未送出的組字時回傳 ``True``。
11+
* :func:`wait_for_composition_commit` ——阻塞直到組字結束(或逾時),``clock`` / ``sleep`` /
12+
``reader`` 皆可注入。
13+
* :func:`decode_conversion_mode` ——純函式:把 IMM32 ``IME_CMODE_*`` 轉換位元遮罩解碼為
14+
``{native, katakana, full_shape, roman, char_code}``。
15+
16+
預設 ``reader`` 以唯讀方式查詢 Windows IMM32(``ImmGetContext`` / ``ImmGetOpenStatus`` /
17+
``ImmGetConversionStatus`` / ``ImmGetCompositionStringW``);所有解碼 / 等待邏輯都透過可注入接縫
18+
執行,故能在沒有 IME 的情況下完整測試。不匯入 ``PySide6``。
19+
20+
無頭 API
21+
--------
22+
23+
.. code-block:: python
24+
25+
from je_auto_control import (
26+
ime_state, is_composing, wait_for_composition_commit,
27+
)
28+
29+
# 讀取 CJK 欄位前,先確認 IME 已送出
30+
if wait_for_composition_commit(timeout_s=3):
31+
value = read_field()
32+
33+
is_composing() # 候選字仍在畫面上時為 True
34+
ime_state() # {'open': True, 'composing': True, 'composition': 'あ', ...}
35+
36+
測試時(或任何非 Windows 主機)可傳入 ``reader`` ——一個
37+
``() -> {open, conversion, composition}``:
38+
39+
.. code-block:: python
40+
41+
busy = lambda: {"open": True, "conversion": 0, "composition": ""}
42+
is_composing(reader=busy) # True
43+
ime_state(reader=busy)["composition"] # 'あ'
44+
45+
執行器指令
46+
----------
47+
48+
``AC_ime_state``(→ 完整狀態)、``AC_is_composing``(→ ``{composing}``)、
49+
``AC_wait_for_composition_commit``(``timeout`` / ``interval`` → ``{committed}``)
50+
與 ``AC_decode_conversion_mode``(``flags`` → 解碼後的模式)。皆以對應的唯讀 ``ac_*`` MCP 工具
51+
及 Script Builder 指令(位於 **Shell** 分類下)形式提供。

je_auto_control/__init__.py

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -110,6 +110,11 @@
110110
classify_lock_transitions, lock_session, plan_lock_session,
111111
wait_for_lock, wait_for_unlock,
112112
)
113+
# Read the live IME composition / conversion state for safe CJK entry
114+
from je_auto_control.utils.ime_state import (
115+
decode_conversion_mode, ime_state, is_composing,
116+
wait_for_composition_commit,
117+
)
113118
# Rich clipboard formats — RTF + CSV/TSV codecs and Windows get / set
114119
from je_auto_control.utils.clipboard_rich_formats import (
115120
build_rtf, csv_to_rows, get_clipboard_csv, get_clipboard_rtf, rows_to_csv,
@@ -1727,6 +1732,8 @@ def start_autocontrol_gui(*args, **kwargs):
17271732
"is_muted", "set_mute", "mute", "unmute", "toggle_mute",
17281733
"lock_session", "plan_lock_session", "wait_for_unlock",
17291734
"wait_for_lock", "classify_lock_transitions",
1735+
"ime_state", "is_composing", "wait_for_composition_commit",
1736+
"decode_conversion_mode",
17301737
"build_rtf", "rtf_to_text", "rows_to_csv", "csv_to_rows",
17311738
"set_clipboard_rtf", "get_clipboard_rtf",
17321739
"set_clipboard_csv", "get_clipboard_csv",

je_auto_control/gui/script_builder/command_schema.py

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4373,6 +4373,34 @@ def _add_work_queue_specs(specs: List[CommandSpec]) -> None:
43734373
),
43744374
description="Reduce lock-state samples to lock / unlock events.",
43754375
))
4376+
specs.append(CommandSpec(
4377+
"AC_ime_state", "Shell", "IME State",
4378+
fields=(),
4379+
description="Read the focused window's live IME composition state.",
4380+
))
4381+
specs.append(CommandSpec(
4382+
"AC_is_composing", "Shell", "Is IME Composing",
4383+
fields=(),
4384+
description="True while the IME has an uncommitted composition.",
4385+
))
4386+
specs.append(CommandSpec(
4387+
"AC_wait_for_composition_commit", "Shell", "Wait for IME Commit",
4388+
fields=(
4389+
FieldSpec("timeout", FieldType.FLOAT, optional=True, default=5.0,
4390+
placeholder="timeout seconds"),
4391+
FieldSpec("interval", FieldType.FLOAT, optional=True, default=0.1,
4392+
placeholder="poll interval seconds"),
4393+
),
4394+
description="Block until the IME finishes composing or timeout.",
4395+
))
4396+
specs.append(CommandSpec(
4397+
"AC_decode_conversion_mode", "Shell", "Decode IME Conversion Mode",
4398+
fields=(
4399+
FieldSpec("flags", FieldType.INT, default=0,
4400+
placeholder="IMM32 conversion bitmask"),
4401+
),
4402+
description="Decode an IMM32 conversion bitmask into named flags.",
4403+
))
43764404
specs.append(CommandSpec(
43774405
"AC_normalize_ext", "Shell", "Normalize Extension",
43784406
fields=(

je_auto_control/utils/executor/action_executor.py

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2711,6 +2711,33 @@ def _classify_lock_transitions(states: Any) -> Dict[str, Any]:
27112711
return {"events": classify_lock_transitions(samples)}
27122712

27132713

2714+
def _ime_state() -> Dict[str, Any]:
2715+
"""Adapter: the focused window's live IME composition / conversion state."""
2716+
from je_auto_control.utils.ime_state import ime_state
2717+
return ime_state()
2718+
2719+
2720+
def _is_composing() -> Dict[str, Any]:
2721+
"""Adapter: whether the IME has an uncommitted composition."""
2722+
from je_auto_control.utils.ime_state import is_composing
2723+
return {"composing": bool(is_composing())}
2724+
2725+
2726+
def _wait_for_composition_commit(timeout: Any = 5.0, interval: Any = 0.1
2727+
) -> Dict[str, Any]:
2728+
"""Adapter: block until the IME finishes composing or timeout."""
2729+
from je_auto_control.utils.ime_state import wait_for_composition_commit
2730+
committed = wait_for_composition_commit(timeout_s=float(timeout),
2731+
interval_s=float(interval))
2732+
return {"committed": bool(committed)}
2733+
2734+
2735+
def _decode_conversion_mode(flags: Any) -> Dict[str, Any]:
2736+
"""Adapter: decode an IMM32 conversion bitmask into named flags (pure)."""
2737+
from je_auto_control.utils.ime_state import decode_conversion_mode
2738+
return decode_conversion_mode(int(flags))
2739+
2740+
27142741
def _normalize_ext(target: str) -> Dict[str, Any]:
27152742
"""Adapter: the lowercased extension of a path / bare ext (pure)."""
27162743
from je_auto_control.utils.file_assoc import normalize_ext
@@ -6729,6 +6756,10 @@ def __init__(self):
67296756
"AC_plan_lock_session": _plan_lock_session,
67306757
"AC_wait_for_unlock": _wait_for_unlock,
67316758
"AC_classify_lock_transitions": _classify_lock_transitions,
6759+
"AC_ime_state": _ime_state,
6760+
"AC_is_composing": _is_composing,
6761+
"AC_wait_for_composition_commit": _wait_for_composition_commit,
6762+
"AC_decode_conversion_mode": _decode_conversion_mode,
67326763
"AC_normalize_ext": _normalize_ext,
67336764
"AC_file_association": _file_association,
67346765
"AC_get_control_text": _get_control_text,
Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
"""Read the live IME composition / conversion state for safe CJK entry."""
2+
from je_auto_control.utils.ime_state.ime_state import (
3+
decode_conversion_mode, ime_state, is_composing,
4+
wait_for_composition_commit,
5+
)
6+
7+
__all__ = [
8+
"ime_state", "is_composing", "wait_for_composition_commit",
9+
"decode_conversion_mode",
10+
]
Lines changed: 134 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,134 @@
1+
"""Read the live IME (input method editor) state for safe CJK entry.
2+
3+
Typing into a CJK / Japanese / Korean field is unsafe while an IME is *composing*:
4+
the candidate text has not been committed yet, so reading the field back returns
5+
half-entered glyphs and the next keystroke edits the composition instead of the
6+
field. ``text_unicode`` (``VK_PACKET``) is blind to this. ``ime_state`` exposes
7+
the live composition and conversion state so a flow can wait for the IME to
8+
commit before it reads or acts.
9+
10+
* :func:`ime_state` — ``{open, composing, composition, conversion}`` for the
11+
focused window's IME, through an injectable ``reader``.
12+
* :func:`is_composing` — ``True`` while the IME has an uncommitted composition.
13+
* :func:`wait_for_composition_commit` — block until composition ends (or a
14+
timeout), with injectable ``clock`` / ``sleep`` / ``reader``.
15+
* :func:`decode_conversion_mode` — pure: the IMM32 ``IME_CMODE_*`` conversion
16+
bitmask to ``{native, katakana, full_shape, roman, char_code}``.
17+
18+
The default ``reader`` queries Windows IMM32 (``ImmGetContext`` /
19+
``ImmGetOpenStatus`` / ``ImmGetConversionStatus`` / ``ImmGetCompositionStringW``)
20+
read-only; all decoding / waiting logic runs through the injectable seam, so it
21+
is fully testable without an IME. Imports no ``PySide6``.
22+
"""
23+
import sys
24+
import time
25+
from typing import Any, Callable, Dict, Optional
26+
27+
# IMM32 conversion-mode (IME_CMODE_*) bit flags.
28+
IME_CMODE_NATIVE = 0x0001
29+
IME_CMODE_KATAKANA = 0x0002
30+
IME_CMODE_FULLSHAPE = 0x0008
31+
IME_CMODE_ROMAN = 0x0010
32+
IME_CMODE_CHARCODE = 0x0020
33+
34+
# A reader returns the raw IME state: {open, conversion, composition}.
35+
ImeReader = Callable[[], Dict[str, Any]]
36+
37+
38+
def decode_conversion_mode(flags: int) -> Dict[str, bool]:
39+
"""Decode an IMM32 ``IME_CMODE_*`` bitmask into named booleans (pure)."""
40+
value = int(flags)
41+
return {
42+
"native": bool(value & IME_CMODE_NATIVE),
43+
"katakana": bool(value & IME_CMODE_KATAKANA),
44+
"full_shape": bool(value & IME_CMODE_FULLSHAPE),
45+
"roman": bool(value & IME_CMODE_ROMAN),
46+
"char_code": bool(value & IME_CMODE_CHARCODE),
47+
}
48+
49+
50+
def _normalize(raw: Dict[str, Any]) -> Dict[str, Any]:
51+
"""Turn a raw reader result into the public IME-state dict (pure)."""
52+
composition = str(raw.get("composition") or "")
53+
flags = int(raw.get("conversion") or 0)
54+
return {
55+
"open": bool(raw.get("open")),
56+
"composing": bool(composition),
57+
"composition": composition,
58+
"conversion": decode_conversion_mode(flags),
59+
"conversion_flags": flags,
60+
}
61+
62+
63+
def ime_state(*, reader: Optional[ImeReader] = None) -> Dict[str, Any]:
64+
"""Return the focused window's IME state.
65+
66+
``{open, composing, composition, conversion, conversion_flags}``. Pass
67+
``reader`` (a ``() -> {open, conversion, composition}``) to supply the
68+
reading in tests; the default queries Windows IMM32.
69+
"""
70+
source = reader if reader is not None else _default_reader
71+
return _normalize(source())
72+
73+
74+
def is_composing(*, reader: Optional[ImeReader] = None) -> bool:
75+
"""Return ``True`` while the IME has an uncommitted composition."""
76+
return bool(ime_state(reader=reader)["composing"])
77+
78+
79+
def wait_for_composition_commit(
80+
*, reader: Optional[ImeReader] = None, timeout_s: float = 5.0,
81+
interval_s: float = 0.1, clock: Callable[[], float] = time.monotonic,
82+
sleep: Callable[[float], None] = time.sleep) -> bool:
83+
"""Block until the IME is no longer composing; ``True``, or ``False`` on timeout.
84+
85+
``clock`` / ``sleep`` / ``reader`` are injectable for deterministic tests.
86+
"""
87+
deadline = clock() + float(timeout_s)
88+
while True:
89+
if not is_composing(reader=reader):
90+
return True
91+
if clock() >= deadline:
92+
return False
93+
sleep(float(interval_s))
94+
95+
96+
# IMM32 composition-string flag: read the in-progress composition (GCS_COMPSTR).
97+
_GCS_COMPSTR = 0x0008
98+
99+
100+
def _read_composition(imm32: Any, himc: int) -> str:
101+
"""Read the in-progress composition string from an IME context."""
102+
import ctypes
103+
byte_len = imm32.ImmGetCompositionStringW(himc, _GCS_COMPSTR, None, 0)
104+
if byte_len <= 0:
105+
return ""
106+
buffer = ctypes.create_unicode_buffer(byte_len // 2)
107+
imm32.ImmGetCompositionStringW(himc, _GCS_COMPSTR, buffer, byte_len)
108+
return buffer.value
109+
110+
111+
def _default_reader() -> Dict[str, Any]:
112+
"""Read the focused window's IME state from Windows IMM32 (read-only)."""
113+
if not sys.platform.startswith("win"):
114+
raise RuntimeError(
115+
"IME state has no OS reader on this platform; pass reader=")
116+
import ctypes
117+
user32 = ctypes.windll.user32
118+
imm32 = ctypes.windll.imm32
119+
hwnd = user32.GetForegroundWindow()
120+
himc = imm32.ImmGetContext(hwnd)
121+
if not himc:
122+
return {"open": False, "conversion": 0, "composition": ""}
123+
try:
124+
conversion = ctypes.c_uint(0)
125+
sentence = ctypes.c_uint(0)
126+
imm32.ImmGetConversionStatus(
127+
himc, ctypes.byref(conversion), ctypes.byref(sentence))
128+
return {
129+
"open": bool(imm32.ImmGetOpenStatus(himc)),
130+
"conversion": int(conversion.value),
131+
"composition": _read_composition(imm32, himc),
132+
}
133+
finally:
134+
imm32.ImmReleaseContext(hwnd, himc)

0 commit comments

Comments
 (0)