Skip to content

Commit fd5345d

Browse files
JE-ChenJE-Chen
committed
Add colour-histogram fingerprint and change detection
1 parent 1037919 commit fd5345d

15 files changed

Lines changed: 399 additions & 1 deletion

File tree

‎README/WHATS_NEW_zh-CN.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,11 @@
11
# 本次更新 — AutoControl
22

3+
## 本次更新 (2026-06-23) — 色彩直方图指纹与变化检测
4+
5+
判断画面在光照 / 缩放下是否仍是「同一个」。完整参考:[`docs/source/Zh/doc/new_features/v145_features_doc.rst`](../docs/source/Zh/doc/new_features/v145_features_doc.rst)。
6+
7+
- **`image_histogram` / `compare_histograms` / `histogram_changed`**(`AC_image_histogram`、`AC_histogram_changed`):`image_dedup` 的感知哈希是空间性的(对颜色/主题脆弱)、`color_stats` 只有单一颜色。归一化色彩直方图是耐光照/缩放的「同一画面、还是调色板变了?」信号(主题切换、重载、旋转横幅)。`image_histogram` 返回逐通道直方图(`hsv`/`rgb`/`gray`);`compare_histograms` 提供 correlation/chisqr/intersection/bhattacharyya;`histogram_changed` 比较参考与实际屏幕。可注入图像 → 无头可测;OpenCV 核心(`cv2.calcHist`/`compareHist`)。
8+
39
## 本次更新 (2026-06-23) — 丰富剪贴板(HTML / CF_HTML)
410

511
把*格式化*的 HTML 复制粘贴到 Word / Outlook。完整参考:[`docs/source/Zh/doc/new_features/v144_features_doc.rst`](../docs/source/Zh/doc/new_features/v144_features_doc.rst)。

‎README/WHATS_NEW_zh-TW.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,11 @@
11
# 本次更新 — AutoControl
22

3+
## 本次更新 (2026-06-23) — 色彩直方圖指紋與變化偵測
4+
5+
判斷畫面在光照 / 縮放下是否仍是「同一個」。完整參考:[`docs/source/Zh/doc/new_features/v145_features_doc.rst`](../docs/source/Zh/doc/new_features/v145_features_doc.rst)。
6+
7+
- **`image_histogram` / `compare_histograms` / `histogram_changed`**(`AC_image_histogram`、`AC_histogram_changed`):`image_dedup` 的感知雜湊是空間性的(對顏色/主題脆弱)、`color_stats` 只有單一顏色。正規化色彩直方圖是耐光照/縮放的「同一畫面、還是調色盤變了?」訊號(主題切換、重載、旋轉橫幅)。`image_histogram` 回傳逐通道直方圖(`hsv`/`rgb`/`gray`);`compare_histograms` 提供 correlation/chisqr/intersection/bhattacharyya;`histogram_changed` 比較參考與實際螢幕。可注入影像 → 無頭可測;OpenCV 核心(`cv2.calcHist`/`compareHist`)。
8+
39
## 本次更新 (2026-06-23) — 豐富剪貼簿(HTML / CF_HTML)
410

511
把*格式化*的 HTML 複製貼上到 Word / Outlook。完整參考:[`docs/source/Zh/doc/new_features/v144_features_doc.rst`](../docs/source/Zh/doc/new_features/v144_features_doc.rst)。

‎WHATS_NEW.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,11 @@
11
# What's New — AutoControl
22

3+
## What's new (2026-06-23) — Colour-Histogram Fingerprint & Change Detection
4+
5+
Tell whether the view is "the same" despite lighting / scale. Full reference: [`docs/source/Eng/doc/new_features/v145_features_doc.rst`](docs/source/Eng/doc/new_features/v145_features_doc.rst).
6+
7+
- **`image_histogram` / `compare_histograms` / `histogram_changed`** (`AC_image_histogram`, `AC_histogram_changed`): `image_dedup`'s perceptual hash is spatial (brittle to colour/theme) and `color_stats` is one colour. A normalized colour histogram is the illumination/scale-robust "same view, or palette shifted?" signal (theme switch, reload, rotated banner). `image_histogram` returns a per-channel histogram (`hsv`/`rgb`/`gray`); `compare_histograms` does correlation/chisqr/intersection/bhattacharyya; `histogram_changed` compares a reference vs the live screen. Injectable image → headless-testable; base OpenCV (`cv2.calcHist`/`compareHist`).
8+
39
## What's new (2026-06-23) — Rich Clipboard (HTML / CF_HTML)
410

511
Copy and paste *formatted* HTML into Word / Outlook. Full reference: [`docs/source/Eng/doc/new_features/v144_features_doc.rst`](docs/source/Eng/doc/new_features/v144_features_doc.rst).

‎docs/source/Eng/doc/new_features/v145_features_doc.rst‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
Colour-Histogram Fingerprint & Change Detection
2+
===============================================
3+
4+
``image_dedup`` fingerprints with a perceptual aHash / dHash — a *spatial* 64-bit
5+
hash that is brittle to colour and theme shifts — and ``color_stats`` reports a single
6+
average / dominant colour. A normalized colour *histogram* is the standard
7+
illumination- and scale-robust signal for "is this the same view, or has the palette
8+
shifted?": a theme switch, a content reload, a rotated banner — which neither hashing
9+
nor one dominant colour captures.
10+
11+
Every function runs on an injectable image (ndarray / path / PIL, RGB), so it is
12+
headless-testable on synthetic arrays. ``cv2.calcHist`` / ``cv2.compareHist`` are base
13+
OpenCV; OpenCV + NumPy come in via ``je_open_cv``. Imports no ``PySide6``.
14+
15+
Headless API
16+
------------
17+
18+
.. code-block:: python
19+
20+
from je_auto_control import (image_histogram, compare_histograms,
21+
histogram_changed)
22+
23+
baseline = image_histogram("golden.png") # 3 * bins floats (HSV)
24+
if histogram_changed("golden.png"): # current = live screen
25+
print("the view changed")
26+
27+
score = compare_histograms(baseline, image_histogram()) # 1.0 == identical
28+
29+
``image_histogram`` returns a per-channel normalized histogram as a flat list
30+
(``space`` = ``hsv`` / ``rgb`` / ``gray``; each channel adds ``bins`` values).
31+
``compare_histograms`` supports ``correlation`` / ``chisqr`` / ``intersection`` /
32+
``bhattacharyya`` (for correlation / intersection higher is more similar; for the
33+
distance methods higher is more different). ``histogram_changed`` compares a
34+
``reference`` against ``current`` (default: the screen) and returns a bool, flipping
35+
the threshold comparison automatically for similarity vs distance methods.
36+
37+
Executor commands
38+
-----------------
39+
40+
``AC_image_histogram`` (``source`` / ``bins`` / ``space`` / ``region`` →
41+
``{bins, space, histogram}``) and ``AC_histogram_changed`` (``reference`` /
42+
``current`` / ``method`` / ``threshold`` / ``space`` / ``region`` →
43+
``{changed, score}``). They are exposed as the MCP tools ``ac_image_histogram`` /
44+
``ac_histogram_changed`` and as Script Builder commands under **Image**.

‎docs/source/Eng/eng_index.rst‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -167,6 +167,7 @@ Comprehensive guides for all AutoControl features.
167167
doc/new_features/v142_features_doc
168168
doc/new_features/v143_features_doc
169169
doc/new_features/v144_features_doc
170+
doc/new_features/v145_features_doc
170171
doc/ocr_backends/ocr_backends_doc
171172
doc/observability/observability_doc
172173
doc/operations_layer/operations_layer_doc

‎docs/source/Zh/doc/new_features/v145_features_doc.rst‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
色彩直方圖指紋與變化偵測
2+
========================
3+
4+
``image_dedup`` 以感知 aHash / dHash 做指紋——那是*空間性*的 64 位元雜湊,對顏色與主題變化很脆弱——而
5+
``color_stats`` 只回傳單一平均 / 主要顏色。正規化的色彩*直方圖*是判斷「這是不是同一個畫面、調色盤是否改變」的標準
6+
耐光照、耐縮放訊號:主題切換、內容重載、旋轉的橫幅——這些雜湊與單一主色都捕捉不到。
7+
8+
每個函式都在可注入的影像(ndarray / 路徑 / PIL,RGB)上執行,因此可對合成陣列做無頭測試。``cv2.calcHist`` /
9+
``cv2.compareHist`` 屬於 OpenCV 核心;OpenCV + NumPy 透過 ``je_open_cv`` 引入。不匯入 ``PySide6``。
10+
11+
無頭 API
12+
--------
13+
14+
.. code-block:: python
15+
16+
from je_auto_control import (image_histogram, compare_histograms,
17+
histogram_changed)
18+
19+
baseline = image_histogram("golden.png") # 3 * bins 個浮點數(HSV)
20+
if histogram_changed("golden.png"): # current = 實際螢幕
21+
print("the view changed")
22+
23+
score = compare_histograms(baseline, image_histogram()) # 1.0 == 完全相同
24+
25+
``image_histogram`` 回傳逐通道正規化直方圖的平面清單(``space`` = ``hsv`` / ``rgb`` / ``gray``;每通道貢獻
26+
``bins`` 個值)。``compare_histograms`` 支援 ``correlation`` / ``chisqr`` / ``intersection`` /
27+
``bhattacharyya``(correlation / intersection 越高越相似;距離方法越高越不同)。``histogram_changed`` 比較
28+
``reference`` 與 ``current``(預設為螢幕)並回傳布林值,會依相似 vs 距離方法自動翻轉門檻比較方向。
29+
30+
執行器命令
31+
----------
32+
33+
``AC_image_histogram``(``source`` / ``bins`` / ``space`` / ``region`` → ``{bins, space, histogram}``)與
34+
``AC_histogram_changed``(``reference`` / ``current`` / ``method`` / ``threshold`` / ``space`` / ``region`` →
35+
``{changed, score}``)。它們以 MCP 工具 ``ac_image_histogram`` / ``ac_histogram_changed`` 以及 Script Builder 中
36+
**Image** 分類下的命令提供。

‎docs/source/Zh/zh_index.rst‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -167,6 +167,7 @@ AutoControl 所有功能的完整使用指南。
167167
doc/new_features/v142_features_doc
168168
doc/new_features/v143_features_doc
169169
doc/new_features/v144_features_doc
170+
doc/new_features/v145_features_doc
170171
doc/ocr_backends/ocr_backends_doc
171172
doc/observability/observability_doc
172173
doc/operations_layer/operations_layer_doc

‎je_auto_control/__init__.py‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -339,6 +339,10 @@
339339
from je_auto_control.utils.rich_clipboard import (
340340
build_cf_html, get_clipboard_html, parse_cf_html, set_clipboard_html,
341341
)
342+
# Colour-histogram fingerprint & change detection (illumination-robust)
343+
from je_auto_control.utils.img_histogram import (
344+
compare_histograms, histogram_changed, image_histogram,
345+
)
342346
# CI workflow annotations (GitHub Actions)
343347
from je_auto_control.utils.ci_annotations import (
344348
emit_annotations, format_annotation,
@@ -1195,6 +1199,9 @@ def start_autocontrol_gui(*args, **kwargs):
11951199
"parse_cf_html",
11961200
"get_clipboard_html",
11971201
"set_clipboard_html",
1202+
"image_histogram",
1203+
"compare_histograms",
1204+
"histogram_changed",
11981205
"emit_annotations", "format_annotation",
11991206
"ClipboardHistory", "default_clipboard_history",
12001207
"analyze_heal_log", "heal_stats", "scan_secrets",

‎je_auto_control/gui/script_builder/command_schema.py‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -502,6 +502,36 @@ def _add_image_specs(specs: List[CommandSpec]) -> None:
502502
),
503503
description="Refine candidate boxes (within / filter / reading / nth / …).",
504504
))
505+
specs.append(CommandSpec(
506+
"AC_image_histogram", "Image", "Image Histogram",
507+
fields=(
508+
FieldSpec("source", FieldType.FILE_PATH, optional=True),
509+
FieldSpec("bins", FieldType.INT, optional=True, default=32),
510+
FieldSpec("space", FieldType.ENUM, optional=True, default="hsv",
511+
choices=("hsv", "rgb", "gray")),
512+
FieldSpec("region", FieldType.STRING, optional=True,
513+
placeholder=_REGION_PLACEHOLDER),
514+
),
515+
description="Colour-histogram fingerprint of an image / the screen.",
516+
))
517+
specs.append(CommandSpec(
518+
"AC_histogram_changed", "Image", "Histogram Changed?",
519+
fields=(
520+
FieldSpec("reference", FieldType.FILE_PATH),
521+
FieldSpec("current", FieldType.FILE_PATH, optional=True),
522+
FieldSpec("method", FieldType.ENUM, optional=True,
523+
default="correlation",
524+
choices=("correlation", "chisqr", "intersection",
525+
"bhattacharyya")),
526+
FieldSpec("threshold", FieldType.FLOAT, optional=True, default=0.9,
527+
min_value=0.0, max_value=1.0),
528+
FieldSpec("space", FieldType.ENUM, optional=True, default="hsv",
529+
choices=("hsv", "rgb", "gray")),
530+
FieldSpec("region", FieldType.STRING, optional=True,
531+
placeholder=_REGION_PLACEHOLDER),
532+
),
533+
description="Detect a palette/view change vs a reference (illumination-robust).",
534+
))
505535

506536

507537
def _add_ocr_specs(specs: List[CommandSpec]) -> None:

‎je_auto_control/utils/executor/action_executor.py‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3676,6 +3676,37 @@ def _get_clipboard_html() -> Dict[str, Any]:
36763676
return {"found": html is not None, "html": html}
36773677

36783678

3679+
def _image_histogram(source: Any = None, bins: Any = 32, space: str = "hsv",
3680+
region: Any = None) -> Dict[str, Any]:
3681+
"""Adapter: per-channel colour histogram of an image / the screen."""
3682+
import json
3683+
from je_auto_control.utils.img_histogram import image_histogram
3684+
if isinstance(region, str):
3685+
region = json.loads(region) if region.strip() else None
3686+
hist = image_histogram(source, region=region, bins=int(bins), space=str(space))
3687+
return {"bins": int(bins), "space": str(space), "histogram": hist}
3688+
3689+
3690+
def _histogram_changed(reference: str, current: Any = None, method: str =
3691+
"correlation", threshold: Any = 0.9, space: str = "hsv",
3692+
region: Any = None) -> Dict[str, Any]:
3693+
"""Adapter: whether the screen / current image differs from a reference."""
3694+
import json
3695+
from je_auto_control.utils.img_histogram import (compare_histograms,
3696+
histogram_changed,
3697+
image_histogram)
3698+
if isinstance(region, str):
3699+
region = json.loads(region) if region.strip() else None
3700+
changed = histogram_changed(reference, current, region=region,
3701+
method=str(method), threshold=float(threshold),
3702+
space=str(space))
3703+
ref_hist = image_histogram(reference, space=str(space))
3704+
cur_hist = (image_histogram(current, space=str(space)) if current is not None
3705+
else image_histogram(region=region, space=str(space)))
3706+
return {"changed": changed,
3707+
"score": compare_histograms(ref_hist, cur_hist, method=str(method))}
3708+
3709+
36793710
def _with_modifiers(modifiers: Any, actions: Any) -> Dict[str, Any]:
36803711
"""Adapter: run nested actions while modifier keys are held down."""
36813712
import json
@@ -5419,6 +5450,8 @@ def __init__(self):
54195450
"AC_locate_chain": _locate_chain,
54205451
"AC_set_clipboard_html": _set_clipboard_html,
54215452
"AC_get_clipboard_html": _get_clipboard_html,
5453+
"AC_image_histogram": _image_histogram,
5454+
"AC_histogram_changed": _histogram_changed,
54225455
"AC_tile_rect": _tile_rect,
54235456
"AC_grid_rects": _grid_rects,
54245457
"AC_cascade_rects": _cascade_rects,

0 commit comments

Comments
 (0)