diff --git a/README/WHATS_NEW_zh-CN.md b/README/WHATS_NEW_zh-CN.md index 96117a60..21761eac 100644 --- a/README/WHATS_NEW_zh-CN.md +++ b/README/WHATS_NEW_zh-CN.md @@ -1,5 +1,11 @@ # 本次更新 — AutoControl +## 本次更新 (2026-06-23) — 线条 / 网格 / 分隔线检测(Hough) + +从原始像素找出表格网格线与 UI 分隔线。完整参考:[`docs/source/Zh/doc/new_features/v141_features_doc.rst`](../docs/source/Zh/doc/new_features/v141_features_doc.rst)。 + +- **`find_lines` / `find_grid` / `find_separators`**(`AC_find_lines`、`AC_find_grid`、`AC_find_separators`):`grid_locator` 分群*已找到*的框、`shape_locator` 找封闭矩形——两者都无法从像素找出表格网格线或分隔线。Canny + 概率 Hough 检测直线段(分类水平/垂直/斜向),`find_grid` 还原 `{rows, cols, cells}` 让你定址「第 3 行、第 2 列」,`find_separators` 返回长分隔线坐标。可注入 haystack → 无头可测;OpenCV 核心(`cv2.HoughLinesP`)。 + ## 本次更新 (2026-06-23) — 免模型文字区检测(MSER) 不跑 OCR 也能找出画面上文字的位置。完整参考:[`docs/source/Zh/doc/new_features/v140_features_doc.rst`](../docs/source/Zh/doc/new_features/v140_features_doc.rst)。 diff --git a/README/WHATS_NEW_zh-TW.md b/README/WHATS_NEW_zh-TW.md index b1c0b8f3..8a3d40ea 100644 --- a/README/WHATS_NEW_zh-TW.md +++ b/README/WHATS_NEW_zh-TW.md @@ -1,5 +1,11 @@ # 本次更新 — AutoControl +## 本次更新 (2026-06-23) — 線條 / 網格 / 分隔線偵測(Hough) + +從原始像素找出表格格線與 UI 分隔線。完整參考:[`docs/source/Zh/doc/new_features/v141_features_doc.rst`](../docs/source/Zh/doc/new_features/v141_features_doc.rst)。 + +- **`find_lines` / `find_grid` / `find_separators`**(`AC_find_lines`、`AC_find_grid`、`AC_find_separators`):`grid_locator` 分群*已找到*的框、`shape_locator` 找封閉矩形——兩者都無法從像素找出表格格線或分隔線。Canny + 機率 Hough 偵測直線段(分類水平/垂直/斜向),`find_grid` 還原 `{rows, cols, cells}` 讓你定址「第 3 列、第 2 欄」,`find_separators` 回傳長分隔線座標。可注入 haystack → 無頭可測;OpenCV 核心(`cv2.HoughLinesP`)。 + ## 本次更新 (2026-06-23) — 免模型文字區偵測(MSER) 不跑 OCR 也能找出畫面上文字的位置。完整參考:[`docs/source/Zh/doc/new_features/v140_features_doc.rst`](../docs/source/Zh/doc/new_features/v140_features_doc.rst)。 diff --git a/WHATS_NEW.md b/WHATS_NEW.md index 464bde5d..a3324391 100644 --- a/WHATS_NEW.md +++ b/WHATS_NEW.md @@ -1,5 +1,11 @@ # What's New — AutoControl +## What's new (2026-06-23) — Line / Grid / Separator Detection (Hough) + +Find table grid lines and UI dividers from raw pixels. Full reference: [`docs/source/Eng/doc/new_features/v141_features_doc.rst`](docs/source/Eng/doc/new_features/v141_features_doc.rst). + +- **`find_lines` / `find_grid` / `find_separators`** (`AC_find_lines`, `AC_find_grid`, `AC_find_separators`): `grid_locator` clusters *already-found* boxes and `shape_locator` finds closed rectangles — neither finds a table's ruling lines or a divider from pixels. Canny + probabilistic Hough detects straight segments (classified horizontal/vertical/diagonal), `find_grid` recovers `{rows, cols, cells}` so you can address "row 3, col 2", and `find_separators` returns the coordinates of long dividers. Injectable haystack → headless-testable; base OpenCV (`cv2.HoughLinesP`). + ## What's new (2026-06-23) — Model-Free Text-Region Detection (MSER) Find where text is on screen without running OCR. Full reference: [`docs/source/Eng/doc/new_features/v140_features_doc.rst`](docs/source/Eng/doc/new_features/v140_features_doc.rst). diff --git a/docs/source/Eng/doc/new_features/v141_features_doc.rst b/docs/source/Eng/doc/new_features/v141_features_doc.rst new file mode 100644 index 00000000..8bf7b2f2 --- /dev/null +++ b/docs/source/Eng/doc/new_features/v141_features_doc.rst @@ -0,0 +1,48 @@ +Line / Grid / Separator Detection (Hough) +========================================= + +``grid_locator`` clusters *already-found* element boxes into a grid; it cannot find +the ruling lines of a table / spreadsheet or a UI divider from raw pixels, and +``shape_locator`` only finds closed rectangles. ``find_lines``, ``find_grid`` and +``find_separators`` detect straight line segments via Canny + the probabilistic Hough +transform, classify them horizontal / vertical / diagonal, recover a table's row and +column coordinates (and cells), and return the positions of long divider lines — so a +script can address "row 3, column 2" or split a panel at its separators with no +template. + +Runs on an injectable ``haystack`` (ndarray / path / PIL), so it is headless-testable +on synthetic arrays. ``cv2.HoughLinesP`` is base OpenCV; OpenCV + NumPy come in via +``je_open_cv``. Imports no ``PySide6``. + +Headless API +------------ + +.. code-block:: python + + from je_auto_control import find_lines, find_grid, find_separators + + for seg in find_lines(min_length=80, orientation="vertical"): + print(seg["x1"], seg["y1"], seg["x2"], seg["y2"], seg["length"]) + + grid = find_grid(min_length=120) + cell = grid["cells"][0] # {x, y, width, height} of row 0, col 0 + click(cell["x"] + cell["width"] // 2, cell["y"] + cell["height"] // 2) + + dividers = find_separators(axis="horizontal") # [y0, y1, ...] of the rules + +``find_lines`` returns ``{x1, y1, x2, y2, angle, length, orientation}`` per segment, +longest first; pass ``orientation`` other than ``any`` to keep only that kind. +``find_grid`` clusters the horizontal rules into row coordinates and the vertical rules +into columns, returning ``{rows, cols, cells}`` (cells are the rectangles between +consecutive rules). ``find_separators`` returns the merged coordinates of long divider +lines along ``axis``. A blank screen yields no lines / cells. + +Executor commands +----------------- + +``AC_find_lines`` (``min_length`` / ``max_gap`` / ``orientation`` / ``region`` → +``{count, lines}``), ``AC_find_grid`` (``min_length`` / ``tol`` / ``region`` → +``{rows, cols, cells}``) and ``AC_find_separators`` (``axis`` / ``min_length`` / +``tol`` / ``region`` → ``{count, axis, coordinates}``). They are exposed as the MCP +tools ``ac_find_lines`` / ``ac_find_grid`` / ``ac_find_separators`` and as Script +Builder commands under **Image**. diff --git a/docs/source/Eng/eng_index.rst b/docs/source/Eng/eng_index.rst index ba50be0a..4885c344 100644 --- a/docs/source/Eng/eng_index.rst +++ b/docs/source/Eng/eng_index.rst @@ -163,6 +163,7 @@ Comprehensive guides for all AutoControl features. doc/new_features/v138_features_doc doc/new_features/v139_features_doc doc/new_features/v140_features_doc + doc/new_features/v141_features_doc doc/ocr_backends/ocr_backends_doc doc/observability/observability_doc doc/operations_layer/operations_layer_doc diff --git a/docs/source/Zh/doc/new_features/v141_features_doc.rst b/docs/source/Zh/doc/new_features/v141_features_doc.rst new file mode 100644 index 00000000..92bad09b --- /dev/null +++ b/docs/source/Zh/doc/new_features/v141_features_doc.rst @@ -0,0 +1,38 @@ +線條 / 網格 / 分隔線偵測(Hough) +================================== + +``grid_locator`` 把*已找到的*元素框分群成網格;它無法從原始像素找出表格 / 試算表的格線或 UI 分隔線,而 +``shape_locator`` 只找封閉矩形。``find_lines``、``find_grid`` 與 ``find_separators`` 以 Canny + 機率 Hough 轉換 +偵測直線段、分類為水平 / 垂直 / 斜向、還原表格的列與欄座標(及儲存格),並回傳長分隔線的位置——讓腳本能在無模板下 +定址「第 3 列、第 2 欄」或在分隔處切分面板。 + +在可注入的 ``haystack``(ndarray / 路徑 / PIL)上執行,因此可對合成陣列做無頭測試。``cv2.HoughLinesP`` 屬於 +OpenCV 核心;OpenCV + NumPy 透過 ``je_open_cv`` 引入。不匯入 ``PySide6``。 + +無頭 API +-------- + +.. code-block:: python + + from je_auto_control import find_lines, find_grid, find_separators + + for seg in find_lines(min_length=80, orientation="vertical"): + print(seg["x1"], seg["y1"], seg["x2"], seg["y2"], seg["length"]) + + grid = find_grid(min_length=120) + cell = grid["cells"][0] # 第 0 列、第 0 欄的 {x, y, width, height} + click(cell["x"] + cell["width"] // 2, cell["y"] + cell["height"] // 2) + + dividers = find_separators(axis="horizontal") # 各格線的 [y0, y1, ...] + +``find_lines`` 為每段回傳 ``{x1, y1, x2, y2, angle, length, orientation}``,最長者優先;傳入非 ``any`` 的 +``orientation`` 只保留該類。``find_grid`` 將水平格線分群為列座標、垂直格線分群為欄,回傳 ``{rows, cols, cells}`` +(儲存格為相鄰格線之間的矩形)。``find_separators`` 回傳沿 ``axis`` 的長分隔線合併後座標。空白畫面不產生線條 / 儲存格。 + +執行器命令 +---------- + +``AC_find_lines``(``min_length`` / ``max_gap`` / ``orientation`` / ``region`` → ``{count, lines}``)、 +``AC_find_grid``(``min_length`` / ``tol`` / ``region`` → ``{rows, cols, cells}``)與 ``AC_find_separators`` +(``axis`` / ``min_length`` / ``tol`` / ``region`` → ``{count, axis, coordinates}``)。它們以 MCP 工具 +``ac_find_lines`` / ``ac_find_grid`` / ``ac_find_separators`` 以及 Script Builder 中 **Image** 分類下的命令提供。 diff --git a/docs/source/Zh/zh_index.rst b/docs/source/Zh/zh_index.rst index fd580f6f..5ef6cc1c 100644 --- a/docs/source/Zh/zh_index.rst +++ b/docs/source/Zh/zh_index.rst @@ -163,6 +163,7 @@ AutoControl 所有功能的完整使用指南。 doc/new_features/v138_features_doc doc/new_features/v139_features_doc doc/new_features/v140_features_doc + doc/new_features/v141_features_doc doc/ocr_backends/ocr_backends_doc doc/observability/observability_doc doc/operations_layer/operations_layer_doc diff --git a/je_auto_control/__init__.py b/je_auto_control/__init__.py index c21890fc..3db5d0e1 100644 --- a/je_auto_control/__init__.py +++ b/je_auto_control/__init__.py @@ -324,6 +324,10 @@ from je_auto_control.utils.text_regions import ( find_text_lines, find_text_regions, ) +# Line / grid / separator detection on raw pixels (Hough transform) +from je_auto_control.utils.edge_lines import ( + find_grid, find_lines, find_separators, +) # CI workflow annotations (GitHub Actions) from je_auto_control.utils.ci_annotations import ( emit_annotations, format_annotation, @@ -1162,6 +1166,9 @@ def start_autocontrol_gui(*args, **kwargs): "dominant_hue_regions", "find_text_regions", "find_text_lines", + "find_lines", + "find_grid", + "find_separators", "emit_annotations", "format_annotation", "ClipboardHistory", "default_clipboard_history", "analyze_heal_log", "heal_stats", "scan_secrets", diff --git a/je_auto_control/gui/script_builder/command_schema.py b/je_auto_control/gui/script_builder/command_schema.py index 7d0b26e1..d5bcdbd1 100644 --- a/je_auto_control/gui/script_builder/command_schema.py +++ b/je_auto_control/gui/script_builder/command_schema.py @@ -437,6 +437,40 @@ def _add_image_specs(specs: List[CommandSpec]) -> None: ), description="Locate horizontal text lines without OCR.", )) + specs.append(CommandSpec( + "AC_find_lines", "Image", "Find Lines (Hough)", + fields=( + FieldSpec("min_length", FieldType.INT, optional=True, default=80), + FieldSpec("max_gap", FieldType.INT, optional=True, default=10), + FieldSpec("orientation", FieldType.ENUM, optional=True, default="any", + choices=("any", "horizontal", "vertical", "diagonal")), + FieldSpec("region", FieldType.STRING, optional=True, + placeholder=_REGION_PLACEHOLDER), + ), + description="Detect straight line segments on raw pixels.", + )) + specs.append(CommandSpec( + "AC_find_grid", "Image", "Find Table Grid", + fields=( + FieldSpec("min_length", FieldType.INT, optional=True, default=120), + FieldSpec("tol", FieldType.INT, optional=True, default=10), + FieldSpec("region", FieldType.STRING, optional=True, + placeholder=_REGION_PLACEHOLDER), + ), + description="Recover a table's rows / columns / cells from its lines.", + )) + specs.append(CommandSpec( + "AC_find_separators", "Image", "Find Separator Lines", + fields=( + FieldSpec("axis", FieldType.ENUM, optional=True, default="horizontal", + choices=("horizontal", "vertical")), + FieldSpec("min_length", FieldType.INT, optional=True, default=120), + FieldSpec("tol", FieldType.INT, optional=True, default=10), + FieldSpec("region", FieldType.STRING, optional=True, + placeholder=_REGION_PLACEHOLDER), + ), + description="Coordinates of long divider lines along an axis.", + )) specs.append(CommandSpec( "AC_fuse_elements", "Image", "Fuse Element Boxes", fields=( diff --git a/je_auto_control/utils/edge_lines/__init__.py b/je_auto_control/utils/edge_lines/__init__.py new file mode 100644 index 00000000..1bfb56a6 --- /dev/null +++ b/je_auto_control/utils/edge_lines/__init__.py @@ -0,0 +1,6 @@ +"""Line / grid / separator detection on raw pixels (Hough transform).""" +from je_auto_control.utils.edge_lines.edge_lines import ( + find_grid, find_lines, find_separators, +) + +__all__ = ["find_grid", "find_lines", "find_separators"] diff --git a/je_auto_control/utils/edge_lines/edge_lines.py b/je_auto_control/utils/edge_lines/edge_lines.py new file mode 100644 index 00000000..0682cc31 --- /dev/null +++ b/je_auto_control/utils/edge_lines/edge_lines.py @@ -0,0 +1,113 @@ +"""Line, grid and separator detection on raw pixels (Hough) — tables and dividers. + +``grid_locator`` clusters *already-found* element boxes into a grid; it cannot find +the ruling lines of a table / spreadsheet or a UI divider from raw pixels, and +``shape_locator`` only finds closed rectangles. This detects straight line segments +via Canny + the probabilistic Hough transform, classifies them horizontal / vertical / +diagonal, recovers a table's row/column coordinates and cells, and returns the +positions of long divider lines — so a script can address "row 3, column 2" or split a +panel at its separators without templates. + +Runs on an injectable ``haystack`` (ndarray / path / PIL), so it is headless-testable +on synthetic arrays. ``cv2.HoughLinesP`` is base OpenCV; OpenCV + NumPy come in via +``je_open_cv``. Imports no ``PySide6``. +""" +import math +from typing import Any, Dict, List, Optional, Sequence + +from je_auto_control.utils.visual_match.visual_match import _haystack_gray + +ImageSource = Any +_CANNY_LOW = 50 +_CANNY_HIGH = 150 + + +def _orientation(angle: float) -> str: + a = abs(angle) % 180 + if a < 10 or a > 170: + return "horizontal" + if 80 < a < 100: + return "vertical" + return "diagonal" + + +def _segments(gray, min_length: int, max_gap: int): + import cv2 + import numpy as np + edges = cv2.Canny(gray, _CANNY_LOW, _CANNY_HIGH) + return cv2.HoughLinesP(edges, 1, np.pi / 180, threshold=50, + minLineLength=int(min_length), maxLineGap=int(max_gap)) + + +def find_lines(haystack: Optional[ImageSource] = None, *, + region: Optional[Sequence[int]] = None, min_length: int = 80, + max_gap: int = 10, orientation: str = "any" + ) -> List[Dict[str, Any]]: + """Return straight line segments on screen, longest first. + + Each is ``{x1, y1, x2, y2, angle, length, orientation}`` where ``orientation`` is + ``horizontal`` / ``vertical`` / ``diagonal``. Pass ``orientation`` other than + ``any`` to keep only that kind. ``min_length`` / ``max_gap`` tune the Hough probe. + """ + segments = _segments(_haystack_gray(haystack, region), int(min_length), + int(max_gap)) + out: List[Dict[str, Any]] = [] + if segments is None: + return out + for x1, y1, x2, y2 in segments[:, 0]: + angle = math.degrees(math.atan2(int(y2) - int(y1), int(x2) - int(x1))) + kind = _orientation(angle) + if orientation not in ("any", kind): + continue + out.append({"x1": int(x1), "y1": int(y1), "x2": int(x2), "y2": int(y2), + "angle": round(angle, 1), + "length": round(math.hypot(x2 - x1, y2 - y1), 1), + "orientation": kind}) + out.sort(key=lambda seg: seg["length"], reverse=True) + return out + + +def _cluster(values: Sequence[int], tol: int) -> List[int]: + """Group near-equal coordinates and return each cluster's mean (sorted).""" + groups: List[List[int]] = [] + for value in sorted(values): + if groups and value - groups[-1][-1] <= tol: + groups[-1].append(value) + else: + groups.append([value]) + return [int(round(sum(group) / len(group))) for group in groups] + + +def find_separators(haystack: Optional[ImageSource] = None, *, + region: Optional[Sequence[int]] = None, + axis: str = "horizontal", min_length: int = 120, + tol: int = 10) -> List[int]: + """Return the coordinates of long divider lines along ``axis``. + + ``axis="horizontal"`` returns the ``y`` of each horizontal rule (top-to-bottom); + ``axis="vertical"`` returns the ``x`` of each vertical rule. Near-equal lines are + merged within ``tol`` pixels. + """ + lines = find_lines(haystack, region=region, min_length=int(min_length), + orientation=axis) + coords = [seg["y1"] if axis == "horizontal" else seg["x1"] for seg in lines] + return _cluster(coords, int(tol)) + + +def find_grid(haystack: Optional[ImageSource] = None, *, + region: Optional[Sequence[int]] = None, min_length: int = 120, + tol: int = 10) -> Dict[str, Any]: + """Recover a table's grid: ``{rows: [y…], cols: [x…], cells: [{x,y,width,height}]}``. + + Horizontal rules give the row coordinates, vertical rules the columns; the cells + are the rectangles between consecutive rules. ``min_length`` filters short edges. + """ + lines = find_lines(haystack, region=region, min_length=int(min_length)) + rows = _cluster([seg["y1"] for seg in lines + if seg["orientation"] == "horizontal"], int(tol)) + cols = _cluster([seg["x1"] for seg in lines + if seg["orientation"] == "vertical"], int(tol)) + cells = [{"x": cols[i], "y": rows[j], "width": cols[i + 1] - cols[i], + "height": rows[j + 1] - rows[j]} + for j in range(len(rows) - 1) for i in range(len(cols) - 1)] + return {"rows": rows, "cols": cols, "cells": cells} diff --git a/je_auto_control/utils/executor/action_executor.py b/je_auto_control/utils/executor/action_executor.py index d25a0559..ad5a5402 100644 --- a/je_auto_control/utils/executor/action_executor.py +++ b/je_auto_control/utils/executor/action_executor.py @@ -3561,6 +3561,40 @@ def _find_text_lines(y_tolerance: Any = 8, region: Any = None) -> Dict[str, Any] return {"count": len(lines), "lines": lines} +def _find_lines(min_length: Any = 80, max_gap: Any = 10, orientation: str = "any", + region: Any = None) -> Dict[str, Any]: + """Adapter: detect straight line segments on screen (Hough).""" + import json + from je_auto_control.utils.edge_lines import find_lines + if isinstance(region, str): + region = json.loads(region) if region.strip() else None + lines = find_lines(region=region, min_length=int(min_length), + max_gap=int(max_gap), orientation=str(orientation)) + return {"count": len(lines), "lines": lines} + + +def _find_grid(min_length: Any = 120, tol: Any = 10, + region: Any = None) -> Dict[str, Any]: + """Adapter: recover a table grid (rows / cols / cells) from screen lines.""" + import json + from je_auto_control.utils.edge_lines import find_grid + if isinstance(region, str): + region = json.loads(region) if region.strip() else None + return find_grid(region=region, min_length=int(min_length), tol=int(tol)) + + +def _find_separators(axis: str = "horizontal", min_length: Any = 120, tol: Any = 10, + region: Any = None) -> Dict[str, Any]: + """Adapter: coordinates of long divider lines along an axis.""" + import json + from je_auto_control.utils.edge_lines import find_separators + if isinstance(region, str): + region = json.loads(region) if region.strip() else None + coords = find_separators(region=region, axis=str(axis), + min_length=int(min_length), tol=int(tol)) + return {"count": len(coords), "axis": str(axis), "coordinates": coords} + + def _with_modifiers(modifiers: Any, actions: Any) -> Dict[str, Any]: """Adapter: run nested actions while modifier keys are held down.""" import json @@ -5297,6 +5331,9 @@ def __init__(self): "AC_dominant_hue_regions": _dominant_hue_regions, "AC_find_text_regions": _find_text_regions, "AC_find_text_lines": _find_text_lines, + "AC_find_lines": _find_lines, + "AC_find_grid": _find_grid, + "AC_find_separators": _find_separators, "AC_tile_rect": _tile_rect, "AC_grid_rects": _grid_rects, "AC_cascade_rects": _cascade_rects, diff --git a/je_auto_control/utils/mcp_server/tools/_factories.py b/je_auto_control/utils/mcp_server/tools/_factories.py index b7477f98..0fba6982 100644 --- a/je_auto_control/utils/mcp_server/tools/_factories.py +++ b/je_auto_control/utils/mcp_server/tools/_factories.py @@ -2970,6 +2970,53 @@ def text_regions_tools() -> List[MCPTool]: ] +def edge_lines_tools() -> List[MCPTool]: + return [ + MCPTool( + name="ac_find_lines", + description=("Detect straight line segments on screen (Hough): " + "{count, lines:[{x1,y1,x2,y2,angle,length,orientation}]} " + "longest first. 'orientation' horizontal/vertical/diagonal/" + "any filters; 'min_length'/'max_gap' tune the probe."), + input_schema=schema({ + "min_length": {"type": "integer"}, + "max_gap": {"type": "integer"}, + "orientation": {"type": "string"}, + "region": {"type": "array", "items": {"type": "integer"}}}, + required=[]), + handler=h.find_lines, + annotations=READ_ONLY, + ), + MCPTool( + name="ac_find_grid", + description=("Recover a table's grid from screen lines: {rows:[y...], " + "cols:[x...], cells:[{x,y,width,height}]}. Address 'row 3, " + "col 2' without a template. 'min_length' filters edges."), + input_schema=schema({ + "min_length": {"type": "integer"}, + "tol": {"type": "integer"}, + "region": {"type": "array", "items": {"type": "integer"}}}, + required=[]), + handler=h.find_grid, + annotations=READ_ONLY, + ), + MCPTool( + name="ac_find_separators", + description=("Coordinates of long divider lines along 'axis' " + "(horizontal -> y of each rule, vertical -> x). Returns " + "{count, axis, coordinates}. Split a panel at its dividers."), + input_schema=schema({ + "axis": {"type": "string"}, + "min_length": {"type": "integer"}, + "tol": {"type": "integer"}, + "region": {"type": "array", "items": {"type": "integer"}}}, + required=[]), + handler=h.find_separators, + annotations=READ_ONLY, + ), + ] + + def ssim_tools() -> List[MCPTool]: return [ MCPTool( @@ -6474,7 +6521,8 @@ def media_assert_tools() -> List[MCPTool]: color_region_tools, ssim_tools, feature_match_tools, shape_locator_tools, window_layout_tools, window_arrange_tools, preprocess_tools, monitor_layout_tools, actionability_tools, element_parse_tools, - hsv_segment_tools, text_regions_tools, plugin_sdk_tools, governance_tools, + hsv_segment_tools, text_regions_tools, edge_lines_tools, plugin_sdk_tools, + governance_tools, credential_lease_tools, egress_tools, approval_testing_tools, trajectory_eval_tools, compliance_tools, agent_trace_tools, video_report_tools, fuzzy_tools, artifact_store_tools, image_dedup_tools, diff --git a/je_auto_control/utils/mcp_server/tools/_handlers.py b/je_auto_control/utils/mcp_server/tools/_handlers.py index 880ae7d0..5a261b73 100644 --- a/je_auto_control/utils/mcp_server/tools/_handlers.py +++ b/je_auto_control/utils/mcp_server/tools/_handlers.py @@ -2211,6 +2211,21 @@ def find_text_lines(y_tolerance=8, region=None): return _find_text_lines(y_tolerance, region) +def find_lines(min_length=80, max_gap=10, orientation="any", region=None): + from je_auto_control.utils.executor.action_executor import _find_lines + return _find_lines(min_length, max_gap, orientation, region) + + +def find_grid(min_length=120, tol=10, region=None): + from je_auto_control.utils.executor.action_executor import _find_grid + return _find_grid(min_length, tol, region) + + +def find_separators(axis="horizontal", min_length=120, tol=10, region=None): + from je_auto_control.utils.executor.action_executor import _find_separators + return _find_separators(axis, min_length, tol, region) + + def detect_drift(reference, current, threshold=0.25, bins=10): from je_auto_control.utils.executor.action_executor import _detect_drift return _detect_drift(reference, current, threshold, bins) diff --git a/test/unit_test/headless/test_edge_lines_batch.py b/test/unit_test/headless/test_edge_lines_batch.py new file mode 100644 index 00000000..729a0cdb --- /dev/null +++ b/test/unit_test/headless/test_edge_lines_batch.py @@ -0,0 +1,73 @@ +"""Headless tests for Hough line / grid / separator detection. No Qt.""" +import pytest + +import je_auto_control as ac + +np = pytest.importorskip("numpy") +cv2 = pytest.importorskip("cv2") + +from je_auto_control.utils.edge_lines import ( # noqa: E402 + find_grid, find_lines, find_separators, +) + + +def _grid_image(): + img = np.full((240, 300), 255, dtype=np.uint8) + for x in (50, 150, 250): + cv2.line(img, (x, 40), (x, 200), 0, 2) + for y in (40, 120, 200): + cv2.line(img, (50, y), (250, y), 0, 2) + return img + + +def test_find_lines_classifies_orientation(): + lines = find_lines(_grid_image(), min_length=80) + assert lines + assert {line["orientation"] for line in lines} == {"horizontal", "vertical"} + assert all(line["length"] >= 80 for line in lines) + + +def test_find_lines_orientation_filter(): + horizontals = find_lines(_grid_image(), min_length=80, orientation="horizontal") + assert horizontals and all(h["orientation"] == "horizontal" for h in horizontals) + + +def test_find_grid_recovers_rows_cols_cells(): + grid = find_grid(_grid_image(), min_length=100) + assert grid["rows"] == [40, 120, 200] + assert grid["cols"] == [50, 150, 250] + assert len(grid["cells"]) == 4 # 2x2 cells from a 3x3 grid + first = grid["cells"][0] + assert first == {"x": 50, "y": 40, "width": 100, "height": 80} + + +def test_find_separators_both_axes(): + assert find_separators(_grid_image(), axis="horizontal", min_length=100) == [ + 40, 120, 200] + assert find_separators(_grid_image(), axis="vertical", min_length=100) == [ + 50, 150, 250] + + +def test_blank_image_has_no_lines(): + blank = np.full((100, 100), 255, dtype=np.uint8) + assert find_lines(blank) == [] + assert find_grid(blank)["cells"] == [] + assert find_separators(blank) == [] + + +# --- wiring --------------------------------------------------------------- + +def test_wiring(): + known = set(ac.executor.known_commands()) + assert {"AC_find_lines", "AC_find_grid", "AC_find_separators"} <= known + from je_auto_control.utils.mcp_server.tools import build_default_tool_registry + names = {t.name for t in build_default_tool_registry()} + assert {"ac_find_lines", "ac_find_grid", "ac_find_separators"} <= names + from je_auto_control.gui.script_builder.command_schema import _build_specs + specs = {s.command for s in _build_specs()} + assert {"AC_find_lines", "AC_find_grid", "AC_find_separators"} <= specs + + +def test_facade_exports(): + for attr in ("find_lines", "find_grid", "find_separators"): + assert hasattr(ac, attr) and attr in ac.__all__