diff --git a/README.md b/README.md index bda086a2..204da324 100644 --- a/README.md +++ b/README.md @@ -13,6 +13,7 @@ ## Table of Contents +- [What's new (2026-06-21) — RFC 9457 Problem Details Parsing](#whats-new-2026-06-21--rfc-9457-problem-details-parsing) - [What's new (2026-06-21) — Data Profiling & Schema Inference](#whats-new-2026-06-21--data-profiling--schema-inference) - [What's new (2026-06-21) — W3C Trace Context Propagation](#whats-new-2026-06-21--w3c-trace-context-propagation) - [What's new (2026-06-21) — HTTP Record & Replay Cassette](#whats-new-2026-06-21--http-record--replay-cassette) @@ -130,6 +131,12 @@ --- +## What's new (2026-06-21) — RFC 9457 Problem Details Parsing + +Read standardized API errors out of HTTP responses. Full reference: [`docs/source/Eng/doc/new_features/v78_features_doc.rst`](docs/source/Eng/doc/new_features/v78_features_doc.rst). + +- **`parse_problem` / `is_problem` / `raise_for_problem` / `ProblemDetails`** (`AC_parse_problem`): `http_request` returned a non-2xx body unparsed, so flows and `assert_http` had no structured way to read a standardized API error. This parses the RFC 9457 `application/problem+json` document — registered `type`/`title`/`status`/`detail`/`instance` members plus vendor extensions — returning `None` for non-problem responses or raising `HttpProblemError`. Pure-stdlib, fully deterministic. + ## What's new (2026-06-21) — Data Profiling & Schema Inference Survey a row-set and propose a validation schema. Full reference: [`docs/source/Eng/doc/new_features/v77_features_doc.rst`](docs/source/Eng/doc/new_features/v77_features_doc.rst). diff --git a/README/README_zh-CN.md b/README/README_zh-CN.md index ea82e31c..da8051f7 100644 --- a/README/README_zh-CN.md +++ b/README/README_zh-CN.md @@ -12,6 +12,7 @@ ## 目录 +- [本次更新 (2026-06-21) — RFC 9457 Problem Details 解析](#本次更新-2026-06-21--rfc-9457-problem-details-解析) - [本次更新 (2026-06-21) — 数据剖析与结构推断](#本次更新-2026-06-21--数据剖析与结构推断) - [本次更新 (2026-06-21) — W3C Trace Context 传播](#本次更新-2026-06-21--w3c-trace-context-传播) - [本次更新 (2026-06-21) — HTTP 录制与重播卡带](#本次更新-2026-06-21--http-录制与重播卡带) @@ -129,6 +130,12 @@ --- +## 本次更新 (2026-06-21) — RFC 9457 Problem Details 解析 + +从 HTTP 响应读取标准化的 API 错误。完整参考:[`docs/source/Zh/doc/new_features/v78_features_doc.rst`](../docs/source/Zh/doc/new_features/v78_features_doc.rst)。 + +- **`parse_problem` / `is_problem` / `raise_for_problem` / `ProblemDetails`**(`AC_parse_problem`):`http_request` 返回的非 2xx 内文未经解析,因此流程与 `assert_http` 无法以结构化方式读取标准化的 API 错误。本功能解析 RFC 9457 `application/problem+json` 文档 —— 已注册的 `type`/`title`/`status`/`detail`/`instance` 成员加上 vendor 扩展字段 —— 对非 problem 响应返回 `None`,或抛出 `HttpProblemError`。纯标准库、完全确定。 + ## 本次更新 (2026-06-21) — 数据剖析与结构推断 扫描数据行集合并提出验证结构。完整参考:[`docs/source/Zh/doc/new_features/v77_features_doc.rst`](../docs/source/Zh/doc/new_features/v77_features_doc.rst)。 diff --git a/README/README_zh-TW.md b/README/README_zh-TW.md index 81dda30a..0fd1caad 100644 --- a/README/README_zh-TW.md +++ b/README/README_zh-TW.md @@ -12,6 +12,7 @@ ## 目錄 +- [本次更新 (2026-06-21) — RFC 9457 Problem Details 解析](#本次更新-2026-06-21--rfc-9457-problem-details-解析) - [本次更新 (2026-06-21) — 資料剖析與結構推斷](#本次更新-2026-06-21--資料剖析與結構推斷) - [本次更新 (2026-06-21) — W3C Trace Context 傳播](#本次更新-2026-06-21--w3c-trace-context-傳播) - [本次更新 (2026-06-21) — HTTP 錄製與重播卡帶](#本次更新-2026-06-21--http-錄製與重播卡帶) @@ -129,6 +130,12 @@ --- +## 本次更新 (2026-06-21) — RFC 9457 Problem Details 解析 + +從 HTTP 回應讀取標準化的 API 錯誤。完整參考:[`docs/source/Zh/doc/new_features/v78_features_doc.rst`](../docs/source/Zh/doc/new_features/v78_features_doc.rst)。 + +- **`parse_problem` / `is_problem` / `raise_for_problem` / `ProblemDetails`**(`AC_parse_problem`):`http_request` 回傳的非 2xx 內文未經解析,因此流程與 `assert_http` 無法以結構化方式讀取標準化的 API 錯誤。本功能解析 RFC 9457 `application/problem+json` 文件 —— 已註冊的 `type`/`title`/`status`/`detail`/`instance` 成員加上 vendor 擴充欄位 —— 對非 problem 回應回傳 `None`,或拋出 `HttpProblemError`。純標準函式庫、完全具決定性。 + ## 本次更新 (2026-06-21) — 資料剖析與結構推斷 掃描資料列集合並提出驗證結構。完整參考:[`docs/source/Zh/doc/new_features/v77_features_doc.rst`](../docs/source/Zh/doc/new_features/v77_features_doc.rst)。 diff --git a/docs/source/Eng/doc/new_features/v78_features_doc.rst b/docs/source/Eng/doc/new_features/v78_features_doc.rst new file mode 100644 index 00000000..3fcacf59 --- /dev/null +++ b/docs/source/Eng/doc/new_features/v78_features_doc.rst @@ -0,0 +1,44 @@ +RFC 9457 Problem Details Parsing +================================ + +``http_request`` returns a non-2xx body unparsed, so a flow — or +``assert_http`` — had no structured way to read a standardised API error. +This parses the RFC 9457 ``application/problem+json`` document: the registered +``type`` / ``title`` / ``status`` / ``detail`` / ``instance`` members plus any +vendor extensions. + +Pure standard library (``json``); imports no ``PySide6``. Every function is +pure (response dict in, dataclass out), so it is fully deterministic in CI. + +Headless API +------------ + +.. code-block:: python + + from je_auto_control import http_request, parse_problem, raise_for_problem + + response = http_request("https://api.example.com/orders/12") + problem = parse_problem(response) # None unless problem+json + if problem is not None: + log(problem.status, problem.title, problem.detail) + retry_after = problem.extensions.get("balance") + + # or convert a problem response into an exception: + raise_for_problem(response) # raises HttpProblemError + +``is_problem`` checks the ``Content-Type`` (case-insensitively). +``parse_problem`` returns a ``ProblemDetails`` (``type`` defaulting to +``about:blank``, an integer ``status`` when coercible, and all non-registered +keys collected into ``extensions``) or ``None`` when the response is not a +problem document; it falls back to parsing ``text`` when ``json`` is absent. +``ProblemDetails.summary`` gives a one-line description and ``to_dict`` flattens +the document with extensions merged back in. ``raise_for_problem`` raises +``HttpProblemError`` (carrying the ``ProblemDetails``) for a problem response +and does nothing otherwise. + +Executor command +---------------- + +``AC_parse_problem`` takes an ``http_request`` ``response`` and returns +``{problem}`` (the flattened document) or ``null``. It is exposed as the MCP +tool ``ac_parse_problem`` and as a Script Builder command under **Data**. diff --git a/docs/source/Eng/eng_index.rst b/docs/source/Eng/eng_index.rst index 2fcc1eb5..c3f869e3 100644 --- a/docs/source/Eng/eng_index.rst +++ b/docs/source/Eng/eng_index.rst @@ -100,6 +100,7 @@ Comprehensive guides for all AutoControl features. doc/new_features/v75_features_doc doc/new_features/v76_features_doc doc/new_features/v77_features_doc + doc/new_features/v78_features_doc doc/ocr_backends/ocr_backends_doc doc/observability/observability_doc doc/operations_layer/operations_layer_doc diff --git a/docs/source/Zh/doc/new_features/v78_features_doc.rst b/docs/source/Zh/doc/new_features/v78_features_doc.rst new file mode 100644 index 00000000..bfde1432 --- /dev/null +++ b/docs/source/Zh/doc/new_features/v78_features_doc.rst @@ -0,0 +1,37 @@ +RFC 9457 Problem Details 解析 +============================ + +``http_request`` 回傳的非 2xx 內文未經解析,因此流程 —— 或 ``assert_http`` —— 無法以結構化方式讀取 +標準化的 API 錯誤。本功能解析 RFC 9457 ``application/problem+json`` 文件:已註冊的 ``type`` / +``title`` / ``status`` / ``detail`` / ``instance`` 成員,加上任何 vendor 擴充欄位。 + +純標準函式庫(``json``);不匯入 ``PySide6``。每個函式皆為純函式(輸入 response dict、輸出 dataclass), +因此在 CI 中完全具決定性。 + +無頭 API +-------- + +.. code-block:: python + + from je_auto_control import http_request, parse_problem, raise_for_problem + + response = http_request("https://api.example.com/orders/12") + problem = parse_problem(response) # 非 problem+json 時為 None + if problem is not None: + log(problem.status, problem.title, problem.detail) + retry_after = problem.extensions.get("balance") + + # 或把 problem 回應轉成例外: + raise_for_problem(response) # 拋出 HttpProblemError + +``is_problem`` 檢查 ``Content-Type``(不分大小寫)。``parse_problem`` 回傳 ``ProblemDetails`` +(``type`` 預設 ``about:blank``,可轉換時 ``status`` 為整數,所有非註冊鍵收進 ``extensions``), +回應非 problem 文件時回傳 ``None``;當 ``json`` 缺席時會回退去解析 ``text``。``ProblemDetails.summary`` +給出一行描述,``to_dict`` 把文件攤平並併回擴充欄位。``raise_for_problem`` 對 problem 回應拋出 +``HttpProblemError``(帶著 ``ProblemDetails``),否則不做任何事。 + +執行器命令 +---------- + +``AC_parse_problem`` 接受 ``http_request`` 的 ``response``,回傳 ``{problem}``(攤平的文件)或 +``null``。它以 MCP 工具 ``ac_parse_problem`` 以及 Script Builder 中 **Data** 分類下的命令提供。 diff --git a/docs/source/Zh/zh_index.rst b/docs/source/Zh/zh_index.rst index 6b31a58b..9f0f7c2c 100644 --- a/docs/source/Zh/zh_index.rst +++ b/docs/source/Zh/zh_index.rst @@ -100,6 +100,7 @@ AutoControl 所有功能的完整使用指南。 doc/new_features/v75_features_doc doc/new_features/v76_features_doc doc/new_features/v77_features_doc + doc/new_features/v78_features_doc doc/ocr_backends/ocr_backends_doc doc/observability/observability_doc doc/operations_layer/operations_layer_doc diff --git a/je_auto_control/__init__.py b/je_auto_control/__init__.py index 2384822d..bd05bc40 100644 --- a/je_auto_control/__init__.py +++ b/je_auto_control/__init__.py @@ -379,6 +379,11 @@ ) # HTTP record/replay cassette (deterministic offline API tests) from je_auto_control.utils.http_cassette import Cassette, CassetteMissError +# RFC 9457 problem+json error parsing +from je_auto_control.utils.http_problem import ( + HttpProblemError, ProblemDetails, is_problem, parse_problem, + raise_for_problem, +) # W3C Trace Context propagation (traceparent / tracestate) from je_auto_control.utils.trace_context import ( SpanContext, TraceContextError, child_context, extract_context, @@ -886,6 +891,8 @@ def start_autocontrol_gui(*args, **kwargs): "Bulkhead", "BulkheadFullError", "next_delay", "parse_ratelimit", "parse_retry_after", "Cassette", "CassetteMissError", + "HttpProblemError", "ProblemDetails", "is_problem", "parse_problem", + "raise_for_problem", "SpanContext", "TraceContextError", "child_context", "extract_context", "format_traceparent", "format_tracestate", "inject_context", "new_root_context", "new_span_id", "new_trace_id", "parse_traceparent", diff --git a/je_auto_control/gui/script_builder/command_schema.py b/je_auto_control/gui/script_builder/command_schema.py index cc8ae4ec..6cdc4b4d 100644 --- a/je_auto_control/gui/script_builder/command_schema.py +++ b/je_auto_control/gui/script_builder/command_schema.py @@ -1644,6 +1644,16 @@ def _add_resilience_specs(specs: List[CommandSpec]) -> None: ), description="Infer a validate_rows-compatible schema from observed rows.", )) + specs.append(CommandSpec( + "AC_parse_problem", "Data", "HTTP Problem (RFC 9457): Parse", + fields=( + FieldSpec("response", FieldType.STRING, + placeholder='{"status": 400, "headers": ' + '{"Content-Type": "application/problem+json"}, ' + '"json": {"title": "Bad Request"}}'), + ), + description="Parse an application/problem+json error response.", + )) specs.append(CommandSpec( "AC_rate_limit", "Flow", "Rate Limit (Token Bucket)", fields=( diff --git a/je_auto_control/utils/executor/action_executor.py b/je_auto_control/utils/executor/action_executor.py index 4bc9354d..96953076 100644 --- a/je_auto_control/utils/executor/action_executor.py +++ b/je_auto_control/utils/executor/action_executor.py @@ -3022,6 +3022,16 @@ def _infer_schema(rows: Any, columns: Any = None) -> Dict[str, Any]: return {"schema": infer_schema(rows, columns)} +def _parse_problem(response: Any) -> Dict[str, Any]: + """Adapter: parse an RFC 9457 problem+json HTTP response.""" + import json + from je_auto_control.utils.http_problem import parse_problem + if isinstance(response, str): + response = json.loads(response) + problem = parse_problem(response) + return {"problem": problem.to_dict() if problem is not None else None} + + def _percentiles(samples: Any, qs: Any = None) -> Dict[str, Any]: """Adapter: exact percentiles of a numeric sample list (or JSON string).""" import json @@ -4102,6 +4112,7 @@ def __init__(self): "AC_trace_extract": _trace_extract, "AC_profile_rows": _profile_rows, "AC_infer_schema": _infer_schema, + "AC_parse_problem": _parse_problem, "AC_unified_diff": _unified_diff, "AC_apply_unified": _apply_unified, "AC_three_way_merge": _three_way_merge, diff --git a/je_auto_control/utils/http_problem/__init__.py b/je_auto_control/utils/http_problem/__init__.py new file mode 100644 index 00000000..dcaede8b --- /dev/null +++ b/je_auto_control/utils/http_problem/__init__.py @@ -0,0 +1,10 @@ +"""RFC 9457 problem+json parsing for AutoControl HTTP responses.""" +from je_auto_control.utils.http_problem.http_problem import ( + HttpProblemError, ProblemDetails, is_problem, parse_problem, + raise_for_problem, +) + +__all__ = [ + "HttpProblemError", "ProblemDetails", "is_problem", "parse_problem", + "raise_for_problem", +] diff --git a/je_auto_control/utils/http_problem/http_problem.py b/je_auto_control/utils/http_problem/http_problem.py new file mode 100644 index 00000000..995d70a7 --- /dev/null +++ b/je_auto_control/utils/http_problem/http_problem.py @@ -0,0 +1,106 @@ +"""Parse RFC 9457 ``application/problem+json`` error responses. + +``http_request`` returns a non-2xx body unparsed, so a flow (or ``assert_http``) +had no structured way to read a standardised API error. This reads the RFC 9457 +problem document — the registered ``type`` / ``title`` / ``status`` / ``detail`` +/ ``instance`` members plus any vendor extensions. + +Pure standard library (``json``); imports no ``PySide6``. Every function is pure +(response dict in, dataclass out) so it is fully deterministic in CI. +""" +import json +from dataclasses import dataclass, field +from typing import Any, Dict, Mapping, Optional + +from je_auto_control.utils.exception.exceptions import AutoControlException + +_PROBLEM_MEDIA_TYPE = "application/problem+json" +_REGISTERED = ("type", "title", "status", "detail", "instance") + + +class HttpProblemError(AutoControlException): + """Raised by :func:`raise_for_problem` when a response is a problem.""" + + def __init__(self, problem: "ProblemDetails") -> None: + super().__init__(problem.summary()) + self.problem = problem + + +@dataclass(frozen=True) +class ProblemDetails: + """An RFC 9457 problem document.""" + + type: str = "about:blank" + title: Optional[str] = None + status: Optional[int] = None + detail: Optional[str] = None + instance: Optional[str] = None + extensions: Dict[str, Any] = field(default_factory=dict) + + def summary(self) -> str: + """A short ``status title — detail`` description.""" + head = " ".join(str(part) for part in (self.status, self.title) if part) + return f"{head} — {self.detail}" if self.detail else (head or self.type) + + def to_dict(self) -> Dict[str, Any]: + """Return the document as a flat dict (extensions merged in).""" + out: Dict[str, Any] = {"type": self.type} + for name in ("title", "status", "detail", "instance"): + value = getattr(self, name) + if value is not None: + out[name] = value + out.update(self.extensions) + return out + + +def is_problem(headers: Optional[Mapping[str, Any]]) -> bool: + """Whether ``headers`` advertise an ``application/problem+json`` body.""" + for key, value in (headers or {}).items(): + if str(key).lower() == "content-type": + return _PROBLEM_MEDIA_TYPE in str(value).lower() + return False + + +def _coerce_status(value: Any) -> Optional[int]: + try: + return int(value) if value is not None else None + except (TypeError, ValueError): + return None + + +def _from_document(document: Mapping[str, Any]) -> ProblemDetails: + extensions = {key: value for key, value in document.items() + if key not in _REGISTERED} + return ProblemDetails( + type=str(document.get("type", "about:blank")), + title=document.get("title"), + status=_coerce_status(document.get("status")), + detail=document.get("detail"), + instance=document.get("instance"), + extensions=extensions) + + +def parse_problem(response: Mapping[str, Any]) -> Optional[ProblemDetails]: + """Parse a problem document from an ``http_request`` response, or ``None``. + + Returns ``None`` unless the response advertises ``application/problem+json`` + and carries a JSON object body. + """ + if not is_problem(response.get("headers")): + return None + document = response.get("json") + if document is None and response.get("text"): + try: + document = json.loads(response["text"]) + except (ValueError, TypeError): + return None + if not isinstance(document, dict): + return None + return _from_document(document) + + +def raise_for_problem(response: Mapping[str, Any]) -> None: + """Raise :class:`HttpProblemError` when ``response`` is a problem document.""" + problem = parse_problem(response) + if problem is not None: + raise HttpProblemError(problem) diff --git a/je_auto_control/utils/mcp_server/tools/_factories.py b/je_auto_control/utils/mcp_server/tools/_factories.py index 19788fec..5624fbb6 100644 --- a/je_auto_control/utils/mcp_server/tools/_factories.py +++ b/je_auto_control/utils/mcp_server/tools/_factories.py @@ -3343,6 +3343,21 @@ def rate_limit_tools() -> List[MCPTool]: ] +def http_problem_tools() -> List[MCPTool]: + return [ + MCPTool( + name="ac_parse_problem", + description=("Parse an RFC 9457 application/problem+json HTTP " + "'response' ({status, headers, json}). Returns " + "{problem} (type/title/status/detail/instance + " + "extensions) or null when not a problem document."), + input_schema=schema({"response": {"type": "object"}}, ["response"]), + handler=h.parse_problem, + annotations=READ_ONLY, + ), + ] + + def data_profile_tools() -> List[MCPTool]: return [ MCPTool( @@ -4932,7 +4947,7 @@ def media_assert_tools() -> List[MCPTool]: search_index_tools, stats_tools, recurrence_tools, text_diff_tools, feature_flag_tools, provenance_tools, json_contract_tools, chaos_tools, slo_tools, percentiles_tools, bulkhead_tools, http_cassette_tools, - trace_context_tools, data_profile_tools, + trace_context_tools, data_profile_tools, http_problem_tools, saga_tools, decision_table_tools, locator_repair_tools, pii_text_tools, sarif_tools, screen_record_tools, diff --git a/je_auto_control/utils/mcp_server/tools/_handlers.py b/je_auto_control/utils/mcp_server/tools/_handlers.py index 414b5f20..431ddf50 100644 --- a/je_auto_control/utils/mcp_server/tools/_handlers.py +++ b/je_auto_control/utils/mcp_server/tools/_handlers.py @@ -1736,6 +1736,11 @@ def infer_schema(rows, columns=None): return _infer_schema(rows, columns) +def parse_problem(response): + from je_auto_control.utils.executor.action_executor import _parse_problem + return _parse_problem(response) + + def build_provenance(paths, builder_id="je_auto_control"): from je_auto_control.utils.provenance import build_provenance, subject_for subjects = [subject_for(path) for path in paths] diff --git a/test/unit_test/headless/test_http_problem_batch.py b/test/unit_test/headless/test_http_problem_batch.py new file mode 100644 index 00000000..44d7eef7 --- /dev/null +++ b/test/unit_test/headless/test_http_problem_batch.py @@ -0,0 +1,95 @@ +"""Headless tests for RFC 9457 problem+json parsing. Pure stdlib, no Qt.""" +import json + +import pytest + +import je_auto_control as ac +from je_auto_control.utils.http_problem import ( + HttpProblemError, ProblemDetails, is_problem, parse_problem, + raise_for_problem, +) + +_PROBLEM = { + "status": 403, + "headers": {"Content-Type": "application/problem+json; charset=utf-8"}, + "json": { + "type": "https://example.com/probs/forbidden", + "title": "Forbidden", + "status": 403, + "detail": "Account is suspended", + "instance": "/orders/12", + "balance": 30, # vendor extension + }, +} +_PLAIN = {"status": 200, "headers": {"Content-Type": "application/json"}, + "json": {"ok": True}} + + +def test_is_problem_detects_media_type(): + assert is_problem(_PROBLEM["headers"]) is True + assert is_problem(_PLAIN["headers"]) is False + assert is_problem({"content-type": "application/problem+json"}) is True + assert is_problem(None) is False + + +def test_parse_registered_members_and_extensions(): + problem = parse_problem(_PROBLEM) + assert isinstance(problem, ProblemDetails) + assert problem.type.endswith("/forbidden") + assert problem.title == "Forbidden" and problem.status == 403 + assert problem.detail == "Account is suspended" + assert problem.extensions == {"balance": 30} + assert "balance" in problem.to_dict() + + +def test_non_problem_returns_none(): + assert parse_problem(_PLAIN) is None + + +def test_parse_from_text_when_json_absent(): + response = {"status": 400, + "headers": {"Content-Type": "application/problem+json"}, + "text": json.dumps({"title": "Bad", "status": 400})} + problem = parse_problem(response) + assert problem.title == "Bad" and problem.status == 400 + + +def test_defaults_and_bad_status_coercion(): + response = {"headers": {"content-type": "application/problem+json"}, + "json": {"status": "not-a-number"}} + problem = parse_problem(response) + assert problem.type == "about:blank" and problem.status is None + + +def test_raise_for_problem(): + with pytest.raises(HttpProblemError) as info: + raise_for_problem(_PROBLEM) + assert info.value.problem.status == 403 + raise_for_problem(_PLAIN) # no raise for a non-problem + + +# --- wiring --------------------------------------------------------------- + +def test_executor_round_trip(): + rec = ac.execute_action([[ + "AC_parse_problem", {"response": json.dumps(_PROBLEM)}]]) + problem = next(v for v in rec.values() if isinstance(v, dict))["problem"] + assert problem["title"] == "Forbidden" and problem["balance"] == 30 + rec2 = ac.execute_action([[ + "AC_parse_problem", {"response": json.dumps(_PLAIN)}]]) + assert next(v for v in rec2.values() + if isinstance(v, dict))["problem"] is None + + +def test_wiring(): + assert "AC_parse_problem" in ac.executor.known_commands() + from je_auto_control.utils.mcp_server.tools import build_default_tool_registry + assert "ac_parse_problem" in {t.name for t in build_default_tool_registry()} + from je_auto_control.gui.script_builder.command_schema import _build_specs + assert "AC_parse_problem" in {s.command for s in _build_specs()} + + +def test_facade_exports(): + for attr in ("HttpProblemError", "ProblemDetails", "is_problem", + "parse_problem", "raise_for_problem"): + assert hasattr(ac, attr) and attr in ac.__all__