Merge pull request #295 from Integration-Automation/feat/link-header-batch

JE-Chen · web-flow · commit b14d6a91f666 · 2026-06-22T02:04:06.000+08:00
Add RFC 8288 Link header parsing and pagination
diff --git a/README.md b/README.md
@@ -13,6 +13,7 @@
 
 ## Table of Contents
 
+- [What's new (2026-06-22) — RFC 8288 Link Header & Pagination](#whats-new-2026-06-22--rfc-8288-link-header--pagination)
 - [What's new (2026-06-22) — Referential Integrity Checks](#whats-new-2026-06-22--referential-integrity-checks)
 - [What's new (2026-06-22) — URI-Scheme Value References](#whats-new-2026-06-22--uri-scheme-value-references)
 - [What's new (2026-06-21) — W3C Baggage Propagation](#whats-new-2026-06-21--w3c-baggage-propagation)
@@ -139,6 +140,12 @@
 
 ---
 
+## What's new (2026-06-22) — RFC 8288 Link Header & Pagination
+
+Parse `Link` headers and follow `rel="next"`. Full reference: [`docs/source/Eng/doc/new_features/v87_features_doc.rst`](docs/source/Eng/doc/new_features/v87_features_doc.rst).
+
+- **`parse_link_header` / `next_url` / `links_by_rel` / `paginate`** (`AC_parse_link_header`, `AC_next_url`): paginated REST APIs return `Link: <...>; rel="next"` but nothing parsed it. This parses the header (quoted values with commas, multiple links), indexes by relation, and `paginate` walks `rel="next"` over an injected `fetch` (transport/cassette) up to `max_pages`. Pure-stdlib, deterministic.
+
 ## What's new (2026-06-22) — Referential Integrity Checks
 
 Foreign-key, unique, accepted-values and row-count checks across tables. Full reference: [`docs/source/Eng/doc/new_features/v86_features_doc.rst`](docs/source/Eng/doc/new_features/v86_features_doc.rst).
diff --git a/README/README_zh-CN.md b/README/README_zh-CN.md
@@ -12,6 +12,7 @@
 
 ## 目录
 
+- [本次更新 (2026-06-22) — RFC 8288 Link 标头与分页](#本次更新-2026-06-22--rfc-8288-link-标头与分页)
 - [本次更新 (2026-06-22) — 参照完整性检查](#本次更新-2026-06-22--参照完整性检查)
 - [本次更新 (2026-06-22) — URI-Scheme 值引用](#本次更新-2026-06-22--uri-scheme-值引用)
 - [本次更新 (2026-06-21) — W3C Baggage 传播](#本次更新-2026-06-21--w3c-baggage-传播)
@@ -138,6 +139,12 @@
 
 ---
 
+## 本次更新 (2026-06-22) — RFC 8288 Link 标头与分页
+
+解析 `Link` 标头并跟随 `rel="next"`。完整参考:[`docs/source/Zh/doc/new_features/v87_features_doc.rst`](../docs/source/Zh/doc/new_features/v87_features_doc.rst)。
+
+- **`parse_link_header` / `next_url` / `links_by_rel` / `paginate`**(`AC_parse_link_header`、`AC_next_url`):分页的 REST API 返回 `Link: <...>; rel="next"`,但没有东西解析它。本功能解析该标头(含逗号的引号值、多个链接)、按关系索引,`paginate` 通过注入的 `fetch`(传输/卡带)跟随 `rel="next"`,上限为 `max_pages`。纯标准库、确定。
+
 ## 本次更新 (2026-06-22) — 参照完整性检查
 
 跨数据表的外键、唯一键、accepted-values 与行数检查。完整参考:[`docs/source/Zh/doc/new_features/v86_features_doc.rst`](../docs/source/Zh/doc/new_features/v86_features_doc.rst)。
diff --git a/README/README_zh-TW.md b/README/README_zh-TW.md
@@ -12,6 +12,7 @@
 
 ## 目錄
 
+- [本次更新 (2026-06-22) — RFC 8288 Link 標頭與分頁](#本次更新-2026-06-22--rfc-8288-link-標頭與分頁)
 - [本次更新 (2026-06-22) — 參照完整性檢查](#本次更新-2026-06-22--參照完整性檢查)
 - [本次更新 (2026-06-22) — URI-Scheme 值參照](#本次更新-2026-06-22--uri-scheme-值參照)
 - [本次更新 (2026-06-21) — W3C Baggage 傳播](#本次更新-2026-06-21--w3c-baggage-傳播)
@@ -138,6 +139,12 @@
 
 ---
 
+## 本次更新 (2026-06-22) — RFC 8288 Link 標頭與分頁
+
+解析 `Link` 標頭並跟隨 `rel="next"`。完整參考:[`docs/source/Zh/doc/new_features/v87_features_doc.rst`](../docs/source/Zh/doc/new_features/v87_features_doc.rst)。
+
+- **`parse_link_header` / `next_url` / `links_by_rel` / `paginate`**(`AC_parse_link_header`、`AC_next_url`):分頁的 REST API 回傳 `Link: <...>; rel="next"`,但沒有東西解析它。本功能解析該標頭(含逗號的引號值、多個連結)、依關係索引,`paginate` 透過注入的 `fetch`(傳輸/卡帶)跟隨 `rel="next"`,上限為 `max_pages`。純標準函式庫、具決定性。
+
 ## 本次更新 (2026-06-22) — 參照完整性檢查
 
 跨資料表的外鍵、唯一鍵、accepted-values 與筆數檢查。完整參考:[`docs/source/Zh/doc/new_features/v86_features_doc.rst`](../docs/source/Zh/doc/new_features/v86_features_doc.rst)。
diff --git a/docs/source/Eng/doc/new_features/v87_features_doc.rst b/docs/source/Eng/doc/new_features/v87_features_doc.rst
@@ -0,0 +1,41 @@
+RFC 8288 Link Header & Pagination
+=================================
+
+Paginated REST APIs return ``Link: <...>; rel="next"`` headers, but nothing
+parsed them, so multi-page fetches needed manual glue. This parses the header
+(handling quoted parameter values and multiple links), indexes links by their
+relation, and walks ``rel="next"`` over an injected transport.
+
+Pure standard library (``re``); imports no ``PySide6``. The parser is pure and
+``paginate`` takes an injected ``fetch`` callable, so pagination is CI-testable
+without a live server.
+
+Headless API
+------------
+
+.. code-block:: python
+
+    from je_auto_control import parse_link_header, next_url, links_by_rel, paginate
+
+    header = '<https://api/x?page=2>; rel="next", <https://api/x?page=9>; rel="last"'
+    links = parse_link_header(header)            # [Link(uri=..., rel="next"), ...]
+    nxt = next_url(header)                        # "https://api/x?page=2"
+    last = links_by_rel(header)["last"].uri
+
+    # Walk every page over an injected fetch (transport / cassette):
+    pages = paginate(start_url, fetch, max_pages=50)
+
+``parse_link_header`` returns a list of ``Link`` (``uri``, ``rel``, and all
+``params``), tolerating quoted values that contain commas and multiple links in
+one header. ``links_by_rel`` indexes by each (space-separated) relation,
+``next_url`` is the ``rel="next"`` convenience, and ``paginate`` fetches a URL
+and follows ``next`` links via the supplied ``fetch`` callable up to
+``max_pages``.
+
+Executor commands
+-----------------
+
+``AC_parse_link_header`` parses a header ``value`` into ``{links}``;
+``AC_next_url`` returns ``{url}`` for ``rel="next"``. Both are exposed as MCP
+tools (``ac_parse_link_header`` / ``ac_next_url``) and as Script Builder
+commands under **Data**.
diff --git a/docs/source/Eng/eng_index.rst b/docs/source/Eng/eng_index.rst
@@ -109,6 +109,7 @@ Comprehensive guides for all AutoControl features.
    doc/new_features/v84_features_doc
    doc/new_features/v85_features_doc
    doc/new_features/v86_features_doc
+   doc/new_features/v87_features_doc
    doc/ocr_backends/ocr_backends_doc
    doc/observability/observability_doc
    doc/operations_layer/operations_layer_doc
diff --git a/docs/source/Zh/doc/new_features/v87_features_doc.rst b/docs/source/Zh/doc/new_features/v87_features_doc.rst
@@ -0,0 +1,34 @@
+RFC 8288 Link 標頭與分頁
+=======================
+
+分頁的 REST API 會回傳 ``Link: <...>; rel="next"`` 標頭,但沒有任何東西解析它,因此多頁抓取需要手動黏合。
+本功能解析該標頭(處理含引號的參數值與多個連結)、依關係索引連結,並透過注入的傳輸走訪 ``rel="next"``。
+
+純標準函式庫(``re``);不匯入 ``PySide6``。解析器為純函式,``paginate`` 接受注入的 ``fetch`` callable,
+因此分頁可在無線上伺服器的情況下於 CI 測試。
+
+無頭 API
+--------
+
+.. code-block:: python
+
+    from je_auto_control import parse_link_header, next_url, links_by_rel, paginate
+
+    header = '<https://api/x?page=2>; rel="next", <https://api/x?page=9>; rel="last"'
+    links = parse_link_header(header)            # [Link(uri=..., rel="next"), ...]
+    nxt = next_url(header)                        # "https://api/x?page=2"
+    last = links_by_rel(header)["last"].uri
+
+    # 透過注入的 fetch(傳輸 / 卡帶)走訪每一頁:
+    pages = paginate(start_url, fetch, max_pages=50)
+
+``parse_link_header`` 回傳 ``Link`` 清單(``uri``、``rel`` 與所有 ``params``),容許含逗號的引號值與單一
+標頭中的多個連結。``links_by_rel`` 依每個(以空白分隔的)關係索引,``next_url`` 是 ``rel="next"`` 的便利
+函式,``paginate`` 抓取一個 URL 並透過提供的 ``fetch`` callable 跟隨 ``next`` 連結,上限為 ``max_pages``。
+
+執行器命令
+----------
+
+``AC_parse_link_header`` 把標頭 ``value`` 解析成 ``{links}``;``AC_next_url`` 回傳 ``rel="next"`` 的
+``{url}``。兩者皆以 MCP 工具(``ac_parse_link_header`` / ``ac_next_url``)以及 Script Builder 中 **Data**
+分類下的命令提供。
diff --git a/docs/source/Zh/zh_index.rst b/docs/source/Zh/zh_index.rst
@@ -109,6 +109,7 @@ AutoControl 所有功能的完整使用指南。
    doc/new_features/v84_features_doc
    doc/new_features/v85_features_doc
    doc/new_features/v86_features_doc
+   doc/new_features/v87_features_doc
    doc/ocr_backends/ocr_backends_doc
    doc/observability/observability_doc
    doc/operations_layer/operations_layer_doc
diff --git a/je_auto_control/__init__.py b/je_auto_control/__init__.py
@@ -412,6 +412,10 @@
 from je_auto_control.utils.sse_client import (
     SSEEvent, SSEParser, parse_event_stream,
 )
+# RFC 8288 Link header parsing + rel="next" pagination
+from je_auto_control.utils.link_header import (
+    Link, links_by_rel, next_url, paginate, parse_link_header,
+)
 # W3C Trace Context propagation (traceparent / tracestate)
 from je_auto_control.utils.trace_context import (
     SpanContext, TraceContextError, child_context, extract_context,
@@ -933,6 +937,7 @@ def start_autocontrol_gui(*args, **kwargs):
     "HttpProblemError", "ProblemDetails", "is_problem", "parse_problem",
     "raise_for_problem",
     "SSEEvent", "SSEParser", "parse_event_stream",
+    "Link", "links_by_rel", "next_url", "paginate", "parse_link_header",
     "SpanContext", "TraceContextError", "child_context", "extract_context",
     "format_traceparent", "format_tracestate", "inject_context",
     "new_root_context", "new_span_id", "new_trace_id", "parse_traceparent",
diff --git a/je_auto_control/gui/script_builder/command_schema.py b/je_auto_control/gui/script_builder/command_schema.py
@@ -1710,6 +1710,22 @@ def _add_resilience_specs(specs: List[CommandSpec]) -> None:
         ),
         description="Parse a text/event-stream blob into events.",
     ))
+    specs.append(CommandSpec(
+        "AC_parse_link_header", "Data", "Link Header: Parse",
+        fields=(
+            FieldSpec("value", FieldType.STRING,
+                      placeholder='<https://api/x?page=2>; rel="next"'),
+        ),
+        description="Parse an RFC 8288 Link header into links.",
+    ))
+    specs.append(CommandSpec(
+        "AC_next_url", "Data", "Link Header: Next URL",
+        fields=(
+            FieldSpec("value", FieldType.STRING,
+                      placeholder='<https://api/x?page=2>; rel="next"'),
+        ),
+        description="Return the rel=next URL from a Link header.",
+    ))
     specs.append(CommandSpec(
         "AC_resolve_config", "Data", "Layered Config: Resolve",
         fields=(
diff --git a/je_auto_control/utils/executor/action_executor.py b/je_auto_control/utils/executor/action_executor.py
@@ -3015,6 +3015,18 @@ def _resolve_refs(obj: Any) -> Dict[str, Any]:
     return {"resolved": resolve_refs_in(obj)}
 
 
+def _parse_link_header(value: str) -> Dict[str, Any]:
+    """Adapter: parse an RFC 8288 Link header into {links}."""
+    from je_auto_control.utils.link_header import parse_link_header
+    return {"links": [link.to_dict() for link in parse_link_header(value)]}
+
+
+def _next_url(value: str) -> Dict[str, Any]:
+    """Adapter: return the rel=next URL from a Link header."""
+    from je_auto_control.utils.link_header import next_url
+    return {"url": next_url(value)}
+
+
 def _baggage_parse(header: str) -> Dict[str, Any]:
     """Adapter: parse a W3C baggage header into {items}."""
     from je_auto_control.utils.baggage import parse_baggage
@@ -4274,6 +4286,8 @@ def __init__(self):
             "AC_baggage_format": _baggage_format,
             "AC_resolve_ref": _resolve_ref,
             "AC_resolve_refs": _resolve_refs,
+            "AC_parse_link_header": _parse_link_header,
+            "AC_next_url": _next_url,
             "AC_profile_rows": _profile_rows,
             "AC_infer_schema": _infer_schema,
             "AC_parse_problem": _parse_problem,
diff --git a/je_auto_control/utils/link_header/__init__.py b/je_auto_control/utils/link_header/__init__.py
@@ -0,0 +1,8 @@
+"""RFC 8288 Link header parsing and pagination for AutoControl."""
+from je_auto_control.utils.link_header.link_header import (
+    Link, links_by_rel, next_url, paginate, parse_link_header,
+)
+
+__all__ = [
+    "Link", "links_by_rel", "next_url", "paginate", "parse_link_header",
+]
diff --git a/je_auto_control/utils/link_header/link_header.py b/je_auto_control/utils/link_header/link_header.py
@@ -0,0 +1,104 @@
+"""Parse RFC 8288 ``Link`` headers and follow ``rel="next"`` pagination.
+
+Paginated REST APIs return ``Link: <...>; rel="next"`` headers, but nothing
+parsed them, so multi-page fetches needed manual glue. This parses the header
+(handling quoted parameter values and multiple links), indexes links by their
+relation, and walks ``rel="next"`` over an injected transport.
+
+Pure standard library (``re``); imports no ``PySide6``. The parser is pure and
+``paginate`` takes an injected ``fetch`` callable, so pagination is CI-testable
+without a live server.
+"""
+import re
+from dataclasses import dataclass, field
+from typing import Any, Callable, Dict, List, Mapping, Optional, Union
+
+_LINK_RE = re.compile(r"<([^>]*)>\s*(;[^<]*)?")
+
+Links = Union[str, List["Link"]]
+Fetch = Callable[[str], Mapping[str, Any]]
+
+
+@dataclass(frozen=True)
+class Link:
+    """One RFC 8288 web link."""
+
+    uri: str
+    rel: Optional[str] = None
+    params: Dict[str, str] = field(default_factory=dict)
+
+    def to_dict(self) -> Dict[str, Any]:
+        """Return a JSON-friendly view of the link."""
+        return {"uri": self.uri, "rel": self.rel, "params": dict(self.params)}
+
+
+def _strip_quotes(value: str) -> str:
+    if len(value) >= 2 and value[0] == '"' and value[-1] == '"':
+        return value[1:-1]
+    return value
+
+
+def _parse_params(chunk: Optional[str]) -> Dict[str, str]:
+    params: Dict[str, str] = {}
+    for part in (chunk or "").split(";"):
+        cleaned = part.strip().rstrip(",").strip()
+        if not cleaned:
+            continue
+        key, sep, value = cleaned.partition("=")
+        key = key.strip().lower()
+        if key and sep:
+            params[key] = _strip_quotes(value.strip().rstrip(",").strip())
+    return params
+
+
+def parse_link_header(value: Optional[str]) -> List[Link]:
+    """Parse a ``Link`` header value into a list of :class:`Link`."""
+    links: List[Link] = []
+    for match in _LINK_RE.finditer(value or ""):
+        params = _parse_params(match.group(2))
+        links.append(Link(uri=match.group(1).strip(),
+                          rel=params.get("rel"), params=params))
+    return links
+
+
+def _as_links(links: Links) -> List[Link]:
+    return parse_link_header(links) if isinstance(links, str) else list(links)
+
+
+def links_by_rel(links: Links) -> Dict[str, Link]:
+    """Index links by each (possibly space-separated) relation; last wins."""
+    indexed: Dict[str, Link] = {}
+    for link in _as_links(links):
+        for token in (link.rel or "").split():
+            indexed[token] = link
+    return indexed
+
+
+def next_url(links: Links) -> Optional[str]:
+    """Return the ``rel="next"`` URI, if any."""
+    link = links_by_rel(links).get("next")
+    return link.uri if link is not None else None
+
+
+def _link_header_of(headers: Optional[Mapping[str, Any]]) -> Optional[str]:
+    for key, value in (headers or {}).items():
+        if str(key).lower() == "link":
+            return str(value)
+    return None
+
+
+def paginate(url: str, fetch: Fetch, *,
+             max_pages: int = 100) -> List[Mapping[str, Any]]:
+    """Fetch ``url`` then follow ``rel="next"`` Link headers via ``fetch``.
+
+    ``fetch`` maps a URL to a response mapping with a ``headers`` dict. Stops at
+    ``max_pages`` or when no ``next`` link is present. Returns each response.
+    """
+    responses: List[Mapping[str, Any]] = []
+    current: Optional[str] = url
+    while current and len(responses) < max_pages:
+        response = fetch(current)
+        responses.append(response)
+        header = _link_header_of(response.get("headers"))
+        current = next_url(header) if header else None
+    return responses
diff --git a/je_auto_control/utils/mcp_server/tools/_factories.py b/je_auto_control/utils/mcp_server/tools/_factories.py
@@ -3343,6 +3343,28 @@ def rate_limit_tools() -> List[MCPTool]:
     ]
 
 
+def link_header_tools() -> List[MCPTool]:
+    return [
+        MCPTool(
+            name="ac_parse_link_header",
+            description=("Parse an RFC 8288 Link header 'value' into {links} "
+                         "(each {uri, rel, params}), handling quoted params and "
+                         "multiple links."),
+            input_schema=schema({"value": {"type": "string"}}, ["value"]),
+            handler=h.parse_link_header,
+            annotations=READ_ONLY,
+        ),
+        MCPTool(
+            name="ac_next_url",
+            description=("Return the rel=next URL from a Link header 'value' as "
+                         "{url} (null when absent)."),
+            input_schema=schema({"value": {"type": "string"}}, ["value"]),
+            handler=h.next_url,
+            annotations=READ_ONLY,
+        ),
+    ]
+
+
 def referential_tools() -> List[MCPTool]:
     rows = {"type": "array", "items": {"type": "object"}}
     return [
@@ -5166,7 +5188,7 @@ def media_assert_tools() -> List[MCPTool]:
     trace_context_tools, baggage_tools, secret_ref_tools,
     data_profile_tools, http_problem_tools, dotenv_tools,
     sse_client_tools, layered_config_tools, data_drift_tools,
-    dataset_diff_tools, referential_tools,
+    dataset_diff_tools, referential_tools, link_header_tools,
     saga_tools, decision_table_tools, locator_repair_tools,
     pii_text_tools, sarif_tools,
     screen_record_tools,
diff --git a/je_auto_control/utils/mcp_server/tools/_handlers.py b/je_auto_control/utils/mcp_server/tools/_handlers.py
@@ -1746,6 +1746,16 @@ def resolve_refs(obj):
     return _resolve_refs(obj)
 
 
+def parse_link_header(value):
+    from je_auto_control.utils.executor.action_executor import _parse_link_header
+    return _parse_link_header(value)
+
+
+def next_url(value):
+    from je_auto_control.utils.executor.action_executor import _next_url
+    return _next_url(value)
+
+
 def profile_rows(rows, columns=None):
     from je_auto_control.utils.executor.action_executor import _profile_rows
     return _profile_rows(rows, columns)
diff --git a/test/unit_test/headless/test_link_header_batch.py b/test/unit_test/headless/test_link_header_batch.py
