Skip to content

Commit 8c4306b

Browse files
committed
Add clipboard_formats: classify + diff the clipboard's available formats
The clipboard holds the same content in several formats at once, and nothing reported which formats were present or detected when that set changed. Add classify_format/classify_formats (map standard CF_* ids + registered names to friendly categories), diff_formats (pure monitor primitive returning added/removed/changed), and a Win32 EnumClipboardFormats enumeration. Classifier and diff are pure and headless-tested; only the live enumeration is Win32.
1 parent d5353f4 commit 8c4306b

11 files changed

Lines changed: 426 additions & 0 deletions

File tree

WHATS_NEW.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,11 @@
11
# What's New — AutoControl
22

3+
## What's new (2026-06-24) — Clipboard Format Inspection (classify / diff available formats)
4+
5+
See which formats are on the clipboard, and detect when its shape changes. Full reference: [`docs/source/Eng/doc/new_features/v186_features_doc.rst`](docs/source/Eng/doc/new_features/v186_features_doc.rst).
6+
7+
- **`classify_format` / `classify_formats` / `diff_formats` / `list_clipboard_formats` / `clipboard_formats`** (`AC_clipboard_formats`, `AC_classify_formats`, `AC_diff_formats`): the clipboard usually holds the same content in several formats at once (a Word copy = text + HTML + RTF; a file copy = CF_HDROP; a screenshot = CF_DIB). This enumerates the live clipboard (`EnumClipboardFormats`) without consuming anything and classifies each format into a friendly category (text/image/files/html/rtf/csv/audio/…); `diff_formats` is a pure monitor primitive returning `{added, removed, changed}` between two snapshots. The classifier and diff are pure (registered names take priority over dynamic ids); only the live enumeration is Win32. No `PySide6`.
8+
39
## What's new (2026-06-24) — Rich Clipboard Formats (RTF and CSV/TSV)
410

511
Put styled text and tables on the clipboard for cross-app paste into Word and Excel. Full reference: [`docs/source/Eng/doc/new_features/v185_features_doc.rst`](docs/source/Eng/doc/new_features/v185_features_doc.rst).
Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
Clipboard Format Inspection (classify / diff available formats)
2+
===============================================================
3+
4+
The clipboard usually holds the *same* content in several formats at once — a
5+
copy from Word offers ``CF_UNICODETEXT`` + ``HTML Format`` + ``Rich Text Format``,
6+
a file copy offers ``CF_HDROP``, a screenshot offers ``CF_DIB``. Knowing *which
7+
formats are present* (without consuming any of them) tells an automation what it
8+
can paste, and comparing two snapshots detects when the clipboard's shape
9+
changed. ``clipboard_formats`` adds:
10+
11+
* :func:`classify_format` / :func:`classify_formats` — map standard ``CF_*`` ids
12+
and registered format names to friendly categories (text / image / files /
13+
html / rtf / csv / audio / …),
14+
* :func:`diff_formats` — a pure monitor primitive: ``{added, removed, changed}``
15+
between two snapshots,
16+
* :func:`list_clipboard_formats` / :func:`clipboard_formats` — enumerate the live
17+
clipboard (``EnumClipboardFormats``) and classify it.
18+
19+
The classifier and diff are pure functions (unit-testable on any platform); only
20+
the live enumeration is Win32 (raising ``RuntimeError`` elsewhere). Imports no
21+
``PySide6``.
22+
23+
Headless API
24+
------------
25+
26+
.. code-block:: python
27+
28+
from je_auto_control import (classify_formats, diff_formats,
29+
clipboard_formats)
30+
31+
classify_formats([13, {"id": 49383, "name": "HTML Format"}])
32+
# {"categories": ["html", "text"], "has_text": True, "has_image": False, ...}
33+
34+
diff_formats([13, 1], [13, 15]) # {"added": [files], "removed": [text], ...}
35+
36+
clipboard_formats() # live clipboard summary (Windows)
37+
38+
A descriptor is an id (``13``), an ``{"id": ..., "name": ...}`` dict, or an
39+
``(id, name)`` tuple. A registered ``name`` takes priority over the id, since
40+
registered formats have dynamic ids (``>= 0xC000``). Unrecognised formats are
41+
``"other"``.
42+
43+
Executor commands
44+
-----------------
45+
46+
``AC_clipboard_formats`` (live, Windows), ``AC_classify_formats`` (``formats``)
47+
and ``AC_diff_formats`` (``before`` / ``after``) — the latter two are pure and
48+
run anywhere. They are exposed as read-only ``ac_*`` MCP tools and as Script
49+
Builder commands under **Data**.
Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
剪貼簿格式檢視(分類 / 比較可用格式)
2+
====================================
3+
4+
剪貼簿通常同時以多種格式保存*相同*內容——從 Word 複製會提供 ``CF_UNICODETEXT`` +
5+
``HTML Format`` + ``Rich Text Format``,複製檔案會提供 ``CF_HDROP``,截圖會提供 ``CF_DIB``。
6+
知道*目前有哪些格式*(且不消耗任何一個)能讓自動化判斷可以貼上什麼,比較兩份快照則能偵測
7+
剪貼簿的形態何時改變。``clipboard_formats`` 加入:
8+
9+
* :func:`classify_format` / :func:`classify_formats` ——把標準 ``CF_*`` id 與已註冊格式名稱
10+
對應到友善類別(text / image / files / html / rtf / csv / audio……),
11+
* :func:`diff_formats` ——純粹的監看原語:兩份快照之間的 ``{added, removed, changed}``,
12+
* :func:`list_clipboard_formats` / :func:`clipboard_formats` ——列舉存活的剪貼簿
13+
(``EnumClipboardFormats``)並加以分類。
14+
15+
分類器與比較器為純函式(可在任何平台單元測試);只有存活列舉為 Win32(其他平台拋出
16+
``RuntimeError``)。不匯入 ``PySide6``。
17+
18+
無頭 API
19+
--------
20+
21+
.. code-block:: python
22+
23+
from je_auto_control import (classify_formats, diff_formats,
24+
clipboard_formats)
25+
26+
classify_formats([13, {"id": 49383, "name": "HTML Format"}])
27+
# {"categories": ["html", "text"], "has_text": True, "has_image": False, ...}
28+
29+
diff_formats([13, 1], [13, 15]) # {"added": [files], "removed": [text], ...}
30+
31+
clipboard_formats() # 存活剪貼簿摘要(Windows)
32+
33+
描述子可為 id(``13``)、``{"id": ..., "name": ...}`` 字典,或 ``(id, name)`` 元組。已註冊的
34+
``name`` 優先於 id,因為已註冊格式的 id 是動態的(``>= 0xC000``)。未辨識的格式為 ``"other"``。
35+
36+
執行器指令
37+
----------
38+
39+
``AC_clipboard_formats``(存活,Windows)、``AC_classify_formats``(``formats``)與
40+
``AC_diff_formats``(``before`` / ``after``)——後兩者為純函式,可在任何平台執行。皆以唯讀
41+
``ac_*`` MCP 工具及 Script Builder 指令(位於 **Data** 分類下)形式提供。

je_auto_control/__init__.py

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -71,6 +71,11 @@
7171
build_rtf, csv_to_rows, get_clipboard_csv, get_clipboard_rtf, rows_to_csv,
7272
rtf_to_text, set_clipboard_csv, set_clipboard_rtf,
7373
)
74+
# Clipboard format inspection (classify / diff available formats)
75+
from je_auto_control.utils.clipboard_formats import (
76+
classify_format, classify_formats, clipboard_formats, diff_formats,
77+
list_clipboard_formats,
78+
)
7479
# VLM element locator (headless)
7580
from je_auto_control.utils.vision import (
7681
VLMNotAvailableError, click_by_description, locate_by_description,
@@ -1642,6 +1647,8 @@ def start_autocontrol_gui(*args, **kwargs):
16421647
"build_rtf", "rtf_to_text", "rows_to_csv", "csv_to_rows",
16431648
"set_clipboard_rtf", "get_clipboard_rtf",
16441649
"set_clipboard_csv", "get_clipboard_csv",
1650+
"classify_format", "classify_formats", "diff_formats",
1651+
"list_clipboard_formats", "clipboard_formats",
16451652
# VLM locator
16461653
"VLMNotAvailableError", "locate_by_description", "click_by_description",
16471654
"verify_description",

je_auto_control/gui/script_builder/command_schema.py

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1622,6 +1622,24 @@ def _add_misc_specs(specs: List[CommandSpec]) -> None:
16221622
default=","),),
16231623
description="Read the clipboard's Csv content as rows (Windows).",
16241624
))
1625+
specs.append(CommandSpec(
1626+
"AC_clipboard_formats", "Data", "List Clipboard Formats",
1627+
description="Enumerate + classify the clipboard's formats (Windows).",
1628+
))
1629+
specs.append(CommandSpec(
1630+
"AC_classify_formats", "Data", "Classify Clipboard Formats",
1631+
fields=(FieldSpec("formats", FieldType.STRING,
1632+
placeholder='[1, 13, {"id": 49161, "name": "Csv"}]'),),
1633+
description="Classify a provided list of clipboard formats (pure).",
1634+
))
1635+
specs.append(CommandSpec(
1636+
"AC_diff_formats", "Data", "Diff Clipboard Formats",
1637+
fields=(
1638+
FieldSpec("before", FieldType.STRING, placeholder="[1, 13]"),
1639+
FieldSpec("after", FieldType.STRING, placeholder="[1, 13, 15]"),
1640+
),
1641+
description="Diff two clipboard-format snapshots (pure).",
1642+
))
16251643
specs.append(CommandSpec(
16261644
"AC_watchdog_add", "Flow", "Watchdog: Add Popup Rule",
16271645
fields=(
Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
"""Inspect and classify the clipboard's available formats (pure classify/diff + Win32 enum)."""
2+
from je_auto_control.utils.clipboard_formats.clipboard_formats import (
3+
classify_format, classify_formats, clipboard_formats, diff_formats,
4+
list_clipboard_formats,
5+
)
6+
7+
__all__ = [
8+
"classify_format", "classify_formats", "diff_formats",
9+
"list_clipboard_formats", "clipboard_formats",
10+
]
Lines changed: 140 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,140 @@
1+
"""Inspect and classify the clipboard's available formats.
2+
3+
The clipboard usually holds the *same* content in several formats at once — a
4+
copy from Word offers ``CF_UNICODETEXT`` + ``HTML Format`` + ``Rich Text Format``,
5+
a file copy offers ``CF_HDROP``, a screenshot offers ``CF_DIB``. Knowing *which
6+
formats are present* (without consuming any of them) tells an automation what it
7+
can paste and lets it detect when the clipboard's shape changed.
8+
9+
``clipboard_formats`` enumerates the live clipboard (``EnumClipboardFormats``) and
10+
classifies each format into a friendly category (text / image / files / html /
11+
rtf / csv / audio / …). The classifier and the snapshot diff are pure functions —
12+
unit-testable on any platform — and only the live enumeration is Win32 (raising
13+
``RuntimeError`` elsewhere, like the base ``clipboard`` module). Imports no
14+
``PySide6``.
15+
"""
16+
from typing import Any, Dict, List, Optional, Sequence, Tuple, Union
17+
18+
# Standard CF_* clipboard format ids → category.
19+
_CF_CATEGORY = {
20+
1: "text", 7: "text", 13: "text", # CF_TEXT / OEMTEXT / UNICODETEXT
21+
2: "image", 8: "image", 17: "image", # CF_BITMAP / DIB / DIBV5
22+
3: "image", 6: "image", 14: "image", # METAFILEPICT / TIFF / ENHMETAFILE
23+
9: "palette", # CF_PALETTE
24+
11: "audio", 12: "audio", # CF_RIFF / CF_WAVE
25+
15: "files", # CF_HDROP
26+
16: "locale", # CF_LOCALE
27+
}
28+
# Registered (named) clipboard formats → category (matched case-insensitively).
29+
_NAME_CATEGORY = {
30+
"html format": "html",
31+
"rich text format": "rtf",
32+
"rich text format without objects": "rtf",
33+
"csv": "csv",
34+
"png": "image", "jfif": "image", "gif": "image", "image/png": "image",
35+
"filenamew": "files", "filename": "files", "filegroupdescriptorw": "files",
36+
"filegroupdescriptor": "files", "shell idlist array": "files",
37+
}
38+
_Format = Union[int, Dict[str, Any], Tuple[int, str]]
39+
40+
41+
def _coerce(item: _Format) -> Tuple[int, str]:
42+
"""Normalise a format descriptor (int / ``{id,name}`` / ``(id,name)``)."""
43+
if isinstance(item, dict):
44+
return int(item.get("id", 0)), str(item.get("name") or "")
45+
if isinstance(item, (tuple, list)):
46+
return int(item[0]), str(item[1] if len(item) > 1 else "")
47+
return int(item), ""
48+
49+
50+
def classify_format(format_id: int, name: Optional[str] = None) -> str:
51+
"""Return the friendly category for one clipboard format.
52+
53+
A registered ``name`` (e.g. ``"HTML Format"``) takes priority; otherwise the
54+
standard ``CF_*`` id is mapped. Anything unrecognised is ``"other"``.
55+
"""
56+
if name:
57+
category = _NAME_CATEGORY.get(name.strip().lower())
58+
if category is not None:
59+
return category
60+
return _CF_CATEGORY.get(int(format_id), "other")
61+
62+
63+
def classify_formats(formats: Sequence[_Format]) -> Dict[str, Any]:
64+
"""Classify a list of clipboard formats into a summary.
65+
66+
Returns ``{categories, formats:[{id,name,category}], has_text, has_image,
67+
has_files}``.
68+
"""
69+
items: List[Dict[str, Any]] = []
70+
for entry in formats:
71+
format_id, name = _coerce(entry)
72+
items.append({"id": format_id, "name": name,
73+
"category": classify_format(format_id, name)})
74+
categories = sorted({item["category"] for item in items})
75+
return {"categories": categories, "formats": items,
76+
"has_text": "text" in categories,
77+
"has_image": "image" in categories,
78+
"has_files": "files" in categories}
79+
80+
81+
def diff_formats(before: Sequence[_Format],
82+
after: Sequence[_Format]) -> Dict[str, Any]:
83+
"""Diff two clipboard-format snapshots into ``{added, removed, changed}``.
84+
85+
``added`` / ``removed`` are classified format dicts; ``changed`` is True when
86+
either is non-empty — a pure monitor primitive over two enumerations.
87+
"""
88+
def keyed(formats: Sequence[_Format]) -> Dict[Tuple[int, str], Dict[str, Any]]:
89+
out: Dict[Tuple[int, str], Dict[str, Any]] = {}
90+
for entry in formats:
91+
format_id, name = _coerce(entry)
92+
out[(format_id, name)] = {"id": format_id, "name": name,
93+
"category": classify_format(format_id, name)}
94+
return out
95+
96+
before_map, after_map = keyed(before), keyed(after)
97+
added = [after_map[k] for k in after_map if k not in before_map]
98+
removed = [before_map[k] for k in before_map if k not in after_map]
99+
return {"added": added, "removed": removed,
100+
"changed": bool(added or removed)}
101+
102+
103+
# --- Win32 live enumeration ------------------------------------------------
104+
105+
def _format_name(user32, format_id: int) -> str:
106+
import ctypes
107+
buffer = ctypes.create_unicode_buffer(256)
108+
if user32.GetClipboardFormatNameW(format_id, buffer, 256):
109+
return buffer.value
110+
return ""
111+
112+
113+
def list_clipboard_formats() -> List[Dict[str, Any]]:
114+
"""Return the formats currently on the clipboard as ``[{id, name}]`` (Windows)."""
115+
import sys
116+
if not sys.platform.startswith("win"):
117+
raise RuntimeError("list_clipboard_formats is only supported on Windows")
118+
import ctypes
119+
user32 = ctypes.windll.user32
120+
if not user32.OpenClipboard(None):
121+
raise RuntimeError("OpenClipboard failed")
122+
try:
123+
formats: List[Dict[str, Any]] = []
124+
format_id = user32.EnumClipboardFormats(0)
125+
while format_id:
126+
formats.append({"id": int(format_id),
127+
"name": _format_name(user32, format_id)})
128+
format_id = user32.EnumClipboardFormats(format_id)
129+
return formats
130+
finally:
131+
user32.CloseClipboard()
132+
133+
134+
def clipboard_formats() -> Dict[str, Any]:
135+
"""Enumerate and classify the live clipboard's formats (Windows).
136+
137+
Returns the :func:`classify_formats` summary for whatever is on the
138+
clipboard right now, without consuming any format.
139+
"""
140+
return classify_formats(list_clipboard_formats())

je_auto_control/utils/executor/action_executor.py

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4216,6 +4216,32 @@ def _get_clipboard_csv(delimiter: str = ",") -> Dict[str, Any]:
42164216
return {"found": rows is not None, "rows": rows or []}
42174217

42184218

4219+
def _clipboard_formats() -> Dict[str, Any]:
4220+
"""Adapter: enumerate and classify the live clipboard's formats (Windows)."""
4221+
from je_auto_control.utils.clipboard_formats import clipboard_formats
4222+
return clipboard_formats()
4223+
4224+
4225+
def _classify_formats(formats: Any) -> Dict[str, Any]:
4226+
"""Adapter: classify a provided list of clipboard formats (pure)."""
4227+
import json
4228+
from je_auto_control.utils.clipboard_formats import classify_formats
4229+
if isinstance(formats, str):
4230+
formats = json.loads(formats)
4231+
return classify_formats(formats)
4232+
4233+
4234+
def _diff_formats(before: Any, after: Any) -> Dict[str, Any]:
4235+
"""Adapter: diff two clipboard-format snapshots (pure)."""
4236+
import json
4237+
from je_auto_control.utils.clipboard_formats import diff_formats
4238+
if isinstance(before, str):
4239+
before = json.loads(before)
4240+
if isinstance(after, str):
4241+
after = json.loads(after)
4242+
return diff_formats(before, after)
4243+
4244+
42194245
def _image_histogram(source: Any = None, bins: Any = 32, space: str = "hsv",
42204246
region: Any = None) -> Dict[str, Any]:
42214247
"""Adapter: per-channel colour histogram of an image / the screen."""
@@ -6433,6 +6459,9 @@ def __init__(self):
64336459
"AC_get_clipboard_rtf": _get_clipboard_rtf,
64346460
"AC_set_clipboard_csv": _set_clipboard_csv,
64356461
"AC_get_clipboard_csv": _get_clipboard_csv,
6462+
"AC_clipboard_formats": _clipboard_formats,
6463+
"AC_classify_formats": _classify_formats,
6464+
"AC_diff_formats": _diff_formats,
64366465
"AC_image_histogram": _image_histogram,
64376466
"AC_histogram_changed": _histogram_changed,
64386467
"AC_changed_regions": _changed_regions,

je_auto_control/utils/mcp_server/tools/_factories.py

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3292,6 +3292,37 @@ def clipboard_files_tools() -> List[MCPTool]:
32923292
handler=h.get_clipboard_csv,
32933293
annotations=READ_ONLY,
32943294
),
3295+
MCPTool(
3296+
name="ac_clipboard_formats",
3297+
description=("Enumerate and classify the formats currently on the "
3298+
"clipboard without consuming them (Windows). Returns "
3299+
"{categories, formats:[{id,name,category}], has_text, "
3300+
"has_image, has_files}."),
3301+
input_schema=schema({}, required=[]),
3302+
handler=h.clipboard_formats,
3303+
annotations=READ_ONLY,
3304+
),
3305+
MCPTool(
3306+
name="ac_classify_formats",
3307+
description=("Classify a provided list of clipboard formats (pure). "
3308+
"'formats' is a list of ids or {id,name}. Returns the "
3309+
"same summary as ac_clipboard_formats."),
3310+
input_schema=schema({"formats": {"type": "array"}},
3311+
required=["formats"]),
3312+
handler=h.classify_formats,
3313+
annotations=READ_ONLY,
3314+
),
3315+
MCPTool(
3316+
name="ac_diff_formats",
3317+
description=("Diff two clipboard-format snapshots (pure): "
3318+
"{added, removed, changed}. 'before'/'after' are lists "
3319+
"of ids or {id,name}."),
3320+
input_schema=schema({"before": {"type": "array"},
3321+
"after": {"type": "array"}},
3322+
required=["before", "after"]),
3323+
handler=h.diff_formats,
3324+
annotations=READ_ONLY,
3325+
),
32953326
]
32963327

32973328

0 commit comments

Comments
 (0)