Add rotation- and scale-tolerant template matching

match_template sweeps scales but assumes axis-aligned templates; OpenCV's
matchTemplate is not rotation-invariant, so a skewed control, rotated icon or
dial is missed. Sweep angles (warpAffine) crossed with a linspace scale-space
and keep the best, reporting the recovered scale and angle. Reuses
visual_match's loaders, resize, method table and NMS.

Author: JE-Chen

README/WHATS_NEW_zh-CN.md

@@ -1,5 +1,11 @@
 # 本次更新 — AutoControl
 
+## 本次更新 (2026-06-23) — 旋转与缩放容忍的模板匹配
+
+不只缩放,还能找到旋转或倾斜的模板。完整参考:[`docs/source/Zh/doc/new_features/v158_features_doc.rst`](../docs/source/Zh/doc/new_features/v158_features_doc.rst)。
+
+- **`match_rotated` / `match_rotated_all` / `scale_space`**(`AC_match_rotated`、`AC_match_rotated_all`):`match_template` 只扫描*缩放*且假设轴对齐——OpenCV 的 `matchTemplate` 不具旋转不变性,因此倾斜的控件、旋转的图标,或转到不同角度的刻度盘都会匹配失败。本功能扫描 `angles`(每个以 `cv2.warpAffine` 变形)并与 `np.linspace` 缩放空间交叉,返回相关性最高、且带有还原 `scale` + `angle` 的 `RotatedMatch`(`*_all` 版本以 NMS 合并相邻角度 / 缩放)。重用 `visual_match` 的加载器 / resize / 方法表 / NMS——不重复任何匹配或几何代码。`haystack` 可注入;可无头测试;不导入 `PySide6`。
+
 ## 本次更新 (2026-06-23) — 一维条码解码
 
 从屏幕或图像读取 EAN / UPC / Code-128 条码。完整参考:[`docs/source/Zh/doc/new_features/v157_features_doc.rst`](../docs/source/Zh/doc/new_features/v157_features_doc.rst)。

README/WHATS_NEW_zh-TW.md

@@ -1,5 +1,11 @@
 # 本次更新 — AutoControl
 
+## 本次更新 (2026-06-23) — 旋轉與縮放容忍的樣板比對
+
+不只縮放,還能找到旋轉或傾斜的樣板。完整參考:[`docs/source/Zh/doc/new_features/v158_features_doc.rst`](../docs/source/Zh/doc/new_features/v158_features_doc.rst)。
+
+- **`match_rotated` / `match_rotated_all` / `scale_space`**(`AC_match_rotated`、`AC_match_rotated_all`):`match_template` 只掃描*縮放*且假設軸對齊——OpenCV 的 `matchTemplate` 不具旋轉不變性,因此傾斜的控制項、旋轉的圖示,或轉到不同角度的刻度盤都會比對失敗。本功能掃描 `angles`(每個以 `cv2.warpAffine` 變形)並與 `np.linspace` 縮放空間交叉,回傳相關性最高、且帶有還原 `scale` + `angle` 的 `RotatedMatch`(`*_all` 版本以 NMS 合併相鄰角度 / 縮放)。重用 `visual_match` 的載入器 / resize / 方法表 / NMS——不重複任何比對或幾何程式。`haystack` 可注入;可無頭測試;不匯入 `PySide6`。
+
 ## 本次更新 (2026-06-23) — 一維條碼解碼
 
 從螢幕或影像讀取 EAN / UPC / Code-128 條碼。完整參考:[`docs/source/Zh/doc/new_features/v157_features_doc.rst`](../docs/source/Zh/doc/new_features/v157_features_doc.rst)。

WHATS_NEW.md

@@ -1,5 +1,11 @@
 # What's New — AutoControl
 
+## What's new (2026-06-23) — Rotation- & Scale-Tolerant Template Matching
+
+Find templates that are rotated or skewed, not just scaled. Full reference: [`docs/source/Eng/doc/new_features/v158_features_doc.rst`](docs/source/Eng/doc/new_features/v158_features_doc.rst).
+
+- **`match_rotated` / `match_rotated_all` / `scale_space`** (`AC_match_rotated`, `AC_match_rotated_all`): `match_template` sweeps *scales* but assumes axis-aligned — OpenCV's `matchTemplate` isn't rotation-invariant, so a skewed control, a rotated icon or a dial at a different angle is missed. This sweeps `angles` (each warped with `cv2.warpAffine`) crossed with a `np.linspace` scale-space, returns the best-correlating `RotatedMatch` carrying the recovered `scale` + `angle` (the `*_all` form NMS-dedupes neighbouring angles/scales). Reuses `visual_match`'s loaders / resize / method table / NMS — no matching or geometry code duplicated. Injectable `haystack`; headless-testable; no `PySide6`.
+
 ## What's new (2026-06-23) — Barcode Decoding (1-D)
 
 Read EAN / UPC / Code-128 barcodes off the screen or an image. Full reference: [`docs/source/Eng/doc/new_features/v157_features_doc.rst`](docs/source/Eng/doc/new_features/v157_features_doc.rst).

docs/source/Eng/doc/new_features/v158_features_doc.rst

@@ -0,0 +1,49 @@
+Rotation- and Scale-Tolerant Template Matching
+==============================================
+
+``match_template`` searches a template across *scales* (DPI / zoom tolerance) but
+assumes it is axis-aligned — OpenCV's ``matchTemplate`` is not rotation-invariant,
+so a control rendered at a slight skew, a rotated icon, or a dial/knob at a different
+angle is missed. ``match_rotated`` adds a rotation sweep: each angle is applied to
+the template with ``cv2.warpAffine``, crossed with a ``np.linspace`` scale-space, and
+the best-correlating (scale, angle) is returned — so the caller also learns the
+recovered *pose*.
+
+It reuses ``visual_match``'s grayscale loaders, scale resize, correlation-method
+table and non-maximum suppression, so no matching or geometry code is duplicated.
+The ``haystack`` is injectable (ndarray / path / PIL), so the search is unit-testable
+on synthetic arrays; only the default (grab the screen) is device-bound. Imports no
+``PySide6``.
+
+Headless API
+------------
+
+.. code-block:: python
+
+    from je_auto_control import match_rotated, match_rotated_all, scale_space
+
+    # find a knob that may be turned to any of these angles, at any of these scales
+    hit = match_rotated("knob.png", angles=[-15, 0, 15, 30],
+                        scales=scale_space(0.9, 1.1, 3), min_score=0.85)
+    if hit:
+        print(hit.angle, hit.scale, hit.score, hit.center)
+
+    # every rotated occurrence, overlaps merged by NMS
+    for m in match_rotated_all("arrow.png", angles=[0, 90, 180, 270]):
+        print(m.center, m.angle)
+
+``match_rotated`` returns a single ``RotatedMatch`` (``x`` / ``y`` / ``width`` /
+``height`` / ``score`` / ``scale`` / ``angle`` + ``center``) or ``None``;
+``match_rotated_all`` returns every hit at or above ``min_score`` with overlapping
+detections from neighbouring angles / scales collapsed by NMS, ordered by score.
+``scale_space(min, max, steps)`` is a helper returning evenly spaced scales.
+
+Executor commands
+-----------------
+
+``AC_match_rotated`` (``template`` / ``min_score`` / ``angles`` / ``scales`` /
+``region`` / ``method`` → ``{found, match}``) and ``AC_match_rotated_all`` (adds
+``max_results`` / ``nms_iou`` → ``{count, matches}``). They are exposed as the MCP
+tools ``ac_match_rotated`` / ``ac_match_rotated_all`` (read-only) and as Script
+Builder commands **Match Template (rotated)** / **Match Template All (rotated)**
+under **Image**.

docs/source/Eng/eng_index.rst

@@ -180,6 +180,7 @@ Comprehensive guides for all AutoControl features.
    doc/new_features/v155_features_doc
    doc/new_features/v156_features_doc
    doc/new_features/v157_features_doc
+   doc/new_features/v158_features_doc
    doc/ocr_backends/ocr_backends_doc
    doc/observability/observability_doc
    doc/operations_layer/operations_layer_doc

docs/source/Zh/doc/new_features/v158_features_doc.rst

@@ -0,0 +1,44 @@
+旋轉與縮放容忍的樣板比對
+========================
+
+``match_template`` 能跨*縮放*搜尋樣板(容忍 DPI / 縮放),但假設樣板為軸對齊——
+OpenCV 的 ``matchTemplate`` 不具旋轉不變性,因此略為傾斜的控制項、旋轉的圖示,或
+轉到不同角度的旋鈕 / 刻度盤都會比對失敗。``match_rotated`` 加入旋轉掃描:每個角度
+以 ``cv2.warpAffine`` 套用到樣板上,並與 ``np.linspace`` 縮放空間交叉,回傳相關性
+最高的(scale, angle)——因此呼叫端也能得知還原出的*姿態*。
+
+本功能重用 ``visual_match`` 的灰階載入器、縮放 resize、相關性方法表與非極大值抑制,
+不重複任何比對或幾何程式。``haystack`` 可注入(ndarray / 路徑 / PIL),因此搜尋可在
+合成陣列上單元測試;只有預設(擷取螢幕)為裝置相依。不匯入 ``PySide6``。
+
+無頭 API
+--------
+
+.. code-block:: python
+
+    from je_auto_control import match_rotated, match_rotated_all, scale_space
+
+    # 尋找可能轉到任一角度、任一縮放的旋鈕
+    hit = match_rotated("knob.png", angles=[-15, 0, 15, 30],
+                        scales=scale_space(0.9, 1.1, 3), min_score=0.85)
+    if hit:
+        print(hit.angle, hit.scale, hit.score, hit.center)
+
+    # 每個旋轉後的出現位置,重疊以 NMS 合併
+    for m in match_rotated_all("arrow.png", angles=[0, 90, 180, 270]):
+        print(m.center, m.angle)
+
+``match_rotated`` 回傳單一 ``RotatedMatch``(``x`` / ``y`` / ``width`` / ``height`` /
+``score`` / ``scale`` / ``angle`` + ``center``)或 ``None``;``match_rotated_all``
+回傳所有達到 ``min_score`` 的命中,相鄰角度 / 縮放的重疊偵測以 NMS 合併,依分數排序。
+``scale_space(min, max, steps)`` 為回傳等間距縮放的輔助函式。
+
+執行器指令
+----------
+
+``AC_match_rotated``(``template`` / ``min_score`` / ``angles`` / ``scales`` /
+``region`` / ``method`` → ``{found, match}``)與 ``AC_match_rotated_all``(另加
+``max_results`` / ``nms_iou`` → ``{count, matches}``)。兩者以 MCP 工具
+``ac_match_rotated`` / ``ac_match_rotated_all``(唯讀)及 Script Builder 指令
+**Match Template (rotated)** / **Match Template All (rotated)**(位於 **Image**
+分類下)形式提供。

docs/source/Zh/zh_index.rst

@@ -180,6 +180,7 @@ AutoControl 所有功能的完整使用指南。
    doc/new_features/v155_features_doc
    doc/new_features/v156_features_doc
    doc/new_features/v157_features_doc
+   doc/new_features/v158_features_doc
    doc/ocr_backends/ocr_backends_doc
    doc/observability/observability_doc
    doc/operations_layer/operations_layer_doc

je_auto_control/__init__.py

@@ -279,6 +279,10 @@
     match_template_all,
 )
 from je_auto_control.utils.visual_match import Match as TemplateMatch
+# Rotation- and scale-tolerant template matching (scale-space x angle sweep)
+from je_auto_control.utils.rotated_match import (
+    RotatedMatch, match_rotated, match_rotated_all, scale_space,
+)
 # Locate on-screen regions by colour (mask + connected components)
 from je_auto_control.utils.color_region import (
     find_color_region, find_color_regions,
@@ -1182,6 +1186,10 @@ def start_autocontrol_gui(*args, **kwargs):
     "match_masked",
     "match_masked_all",
     "best_matches",
+    "RotatedMatch",
+    "match_rotated",
+    "match_rotated_all",
+    "scale_space",
     "find_color_region",
     "find_color_regions",
     "ssim_compare",

je_auto_control/gui/script_builder/command_schema.py

@@ -304,6 +304,37 @@ def _add_image_specs(specs: List[CommandSpec]) -> None:
         ),
         description="Find every masked match of a template (NMS-deduped).",
     ))
+    specs.append(CommandSpec(
+        "AC_match_rotated", "Image", "Match Template (rotated)",
+        fields=(
+            FieldSpec("template", FieldType.FILE_PATH),
+            FieldSpec("min_score", FieldType.FLOAT, optional=True, default=0.8,
+                      min_value=0.0, max_value=1.0),
+            FieldSpec("angles", FieldType.STRING, optional=True,
+                      placeholder="[-10, 0, 10]"),
+            FieldSpec("scales", FieldType.STRING, optional=True,
+                      placeholder="[0.9, 1.0, 1.1]"),
+            FieldSpec("region", FieldType.STRING, optional=True,
+                      placeholder=_REGION_PLACEHOLDER),
+        ),
+        description="Locate a template tolerating rotation + scale; reports angle.",
+    ))
+    specs.append(CommandSpec(
+        "AC_match_rotated_all", "Image", "Match Template All (rotated)",
+        fields=(
+            FieldSpec("template", FieldType.FILE_PATH),
+            FieldSpec("min_score", FieldType.FLOAT, optional=True, default=0.8,
+                      min_value=0.0, max_value=1.0),
+            FieldSpec("angles", FieldType.STRING, optional=True,
+                      placeholder="[-10, 0, 10]"),
+            FieldSpec("scales", FieldType.STRING, optional=True,
+                      placeholder="[0.9, 1.0, 1.1]"),
+            FieldSpec("max_results", FieldType.INT, optional=True, default=20),
+            FieldSpec("nms_iou", FieldType.FLOAT, optional=True, default=0.3,
+                      min_value=0.0, max_value=1.0),
+        ),
+        description="Find every rotation/scale-tolerant match (NMS-deduped).",
+    ))
     specs.append(CommandSpec(
         "AC_find_color_region", "Image", "Find Colour Region",
         fields=(

je_auto_control/utils/executor/action_executor.py

@@ -1,5 +1,5 @@
 import types
-from typing import Any, Callable, Dict, List, Optional, Union
+from typing import Any, Callable, Dict, List, Optional, Sequence, Union
 
 from je_auto_control.utils.exception.exception_tags import (
     action_is_null_error_message, add_command_exception_error_message,
@@ -3282,6 +3282,47 @@ def _match_masked_all(template: str, mask: Any = None, min_score: Any = 0.9,
     return {"count": len(matches), "matches": [m.to_dict() for m in matches]}
 
 
+def _seq_arg(value: Any, default: Sequence[float]) -> Sequence[float]:
+    """Coerce a JSON-string / list arg into a tuple of floats, or the default."""
+    import json
+    if isinstance(value, str):
+        value = json.loads(value) if value.strip() else None
+    return tuple(float(v) for v in value) if value else tuple(default)
+
+
+def _match_rotated(template: str, min_score: Any = 0.8, scales: Any = None,
+                   angles: Any = None, region: Any = None,
+                   method: str = "ccoeff_normed") -> Dict[str, Any]:
+    """Adapter: best rotation/scale-tolerant template match on the screen."""
+    import json
+    from je_auto_control.utils.rotated_match import match_rotated
+    if isinstance(region, str):
+        region = json.loads(region) if region.strip() else None
+    match = match_rotated(template, region=region,
+                          scales=_seq_arg(scales, (1.0,)),
+                          angles=_seq_arg(angles, (0.0,)),
+                          min_score=float(min_score), method=method)
+    return {"found": match is not None,
+            "match": match.to_dict() if match else None}
+
+
+def _match_rotated_all(template: str, min_score: Any = 0.8, scales: Any = None,
+                       angles: Any = None, max_results: Any = 20,
+                       nms_iou: Any = 0.3, region: Any = None) -> Dict[str, Any]:
+    """Adapter: every rotation/scale-tolerant template match (NMS)."""
+    import json
+    from je_auto_control.utils.rotated_match import match_rotated_all
+    if isinstance(region, str):
+        region = json.loads(region) if region.strip() else None
+    matches = match_rotated_all(template, region=region,
+                                scales=_seq_arg(scales, (1.0,)),
+                                angles=_seq_arg(angles, (0.0,)),
+                                min_score=float(min_score),
+                                max_results=int(max_results),
+                                nms_iou=float(nms_iou))
+    return {"count": len(matches), "matches": [m.to_dict() for m in matches]}
+
+
 def _find_color_region(rgb: Any, tolerance: Any = 20, min_area: Any = 50,
                        region: Any = None) -> Dict[str, Any]:
     """Adapter: locate coloured regions on the screen, largest first."""
@@ -5684,6 +5725,8 @@ def __init__(self):
             "AC_match_template_all": _match_template_all,
             "AC_match_masked": _match_masked,
             "AC_match_masked_all": _match_masked_all,
+            "AC_match_rotated": _match_rotated,
+            "AC_match_rotated_all": _match_rotated_all,
             "AC_ssim_compare": _ssim_compare,
             "AC_ssim_changed_regions": _ssim_changed_regions,
             "AC_feature_match": _feature_match,