diff --git a/README.md b/README.md index 91fa4335..61116fa1 100644 --- a/README.md +++ b/README.md @@ -13,6 +13,7 @@ ## Table of Contents +- [What's new (2026-06-22) — Conditional HTTP Requests & Cache Validators](#whats-new-2026-06-22--conditional-http-requests--cache-validators) - [What's new (2026-06-22) — Cookie Jar (HTTP Session Carry)](#whats-new-2026-06-22--cookie-jar-http-session-carry) - [What's new (2026-06-22) — HTTP Content Negotiation & Decompression](#whats-new-2026-06-22--http-content-negotiation--decompression) - [What's new (2026-06-22) — multipart/form-data Build & Parse](#whats-new-2026-06-22--multipartform-data-build--parse) @@ -144,6 +145,12 @@ --- +## What's new (2026-06-22) — Conditional HTTP Requests & Cache Validators + +Skip re-downloading unchanged resources (ETag / 304). Full reference: [`docs/source/Eng/doc/new_features/v92_features_doc.rst`](docs/source/Eng/doc/new_features/v92_features_doc.rst). + +- **`store_validators` / `conditioned_call` / `is_fresh` / `parse_cache_control` / `is_not_modified`** (`AC_parse_cache_control`, `AC_store_validators`): `http_request` never sent `If-None-Match`/`If-Modified-Since` nor read `Cache-Control`, so every poll re-downloaded. This extracts validators, parses `Cache-Control` (max-age/no-store/…), decides freshness by an explicit age, conditions the next request, and detects `304 Not Modified`. Pure-stdlib, deterministic. + ## What's new (2026-06-22) — Cookie Jar (HTTP Session Carry) Carry a session across HTTP calls. Full reference: [`docs/source/Eng/doc/new_features/v91_features_doc.rst`](docs/source/Eng/doc/new_features/v91_features_doc.rst). diff --git a/README/README_zh-CN.md b/README/README_zh-CN.md index 7317be85..cd522447 100644 --- a/README/README_zh-CN.md +++ b/README/README_zh-CN.md @@ -12,6 +12,7 @@ ## 目录 +- [本次更新 (2026-06-22) — 条件式 HTTP 请求与缓存验证子](#本次更新-2026-06-22--条件式-http-请求与缓存验证子) - [本次更新 (2026-06-22) — Cookie Jar(HTTP 会话携带)](#本次更新-2026-06-22--cookie-jarhttp-会话携带) - [本次更新 (2026-06-22) — HTTP 内容协商与解压](#本次更新-2026-06-22--http-内容协商与解压) - [本次更新 (2026-06-22) — multipart/form-data 构建与解析](#本次更新-2026-06-22--multipartform-data-构建与解析) @@ -143,6 +144,12 @@ --- +## 本次更新 (2026-06-22) — 条件式 HTTP 请求与缓存验证子 + +跳过重新下载未变更的资源(ETag / 304)。完整参考:[`docs/source/Zh/doc/new_features/v92_features_doc.rst`](../docs/source/Zh/doc/new_features/v92_features_doc.rst)。 + +- **`store_validators` / `conditioned_call` / `is_fresh` / `parse_cache_control` / `is_not_modified`**(`AC_parse_cache_control`、`AC_store_validators`):`http_request` 从不发 `If-None-Match`/`If-Modified-Since` 也不读 `Cache-Control`,因此每次轮询都重新下载。本功能提取验证子、解析 `Cache-Control`(max-age/no-store/…)、以明确 age 判定新鲜度、为下一个请求加上条件标头,并检测 `304 Not Modified`。纯标准库、确定。 + ## 本次更新 (2026-06-22) — Cookie Jar(HTTP 会话携带) 跨 HTTP 调用携带会话。完整参考:[`docs/source/Zh/doc/new_features/v91_features_doc.rst`](../docs/source/Zh/doc/new_features/v91_features_doc.rst)。 diff --git a/README/README_zh-TW.md b/README/README_zh-TW.md index 63783f46..82db1fa4 100644 --- a/README/README_zh-TW.md +++ b/README/README_zh-TW.md @@ -12,6 +12,7 @@ ## 目錄 +- [本次更新 (2026-06-22) — 條件式 HTTP 請求與快取驗證子](#本次更新-2026-06-22--條件式-http-請求與快取驗證子) - [本次更新 (2026-06-22) — Cookie Jar(HTTP 工作階段攜帶)](#本次更新-2026-06-22--cookie-jarhttp-工作階段攜帶) - [本次更新 (2026-06-22) — HTTP 內容協商與解壓縮](#本次更新-2026-06-22--http-內容協商與解壓縮) - [本次更新 (2026-06-22) — multipart/form-data 建立與解析](#本次更新-2026-06-22--multipartform-data-建立與解析) @@ -143,6 +144,12 @@ --- +## 本次更新 (2026-06-22) — 條件式 HTTP 請求與快取驗證子 + +略過重新下載未變更的資源(ETag / 304)。完整參考:[`docs/source/Zh/doc/new_features/v92_features_doc.rst`](../docs/source/Zh/doc/new_features/v92_features_doc.rst)。 + +- **`store_validators` / `conditioned_call` / `is_fresh` / `parse_cache_control` / `is_not_modified`**(`AC_parse_cache_control`、`AC_store_validators`):`http_request` 從不送 `If-None-Match`/`If-Modified-Since` 也不讀 `Cache-Control`,因此每次輪詢都重新下載。本功能擷取驗證子、解析 `Cache-Control`(max-age/no-store/…)、以明確 age 判定新鮮度、為下一個請求加上條件標頭,並偵測 `304 Not Modified`。純標準函式庫、具決定性。 + ## 本次更新 (2026-06-22) — Cookie Jar(HTTP 工作階段攜帶) 跨 HTTP 呼叫攜帶工作階段。完整參考:[`docs/source/Zh/doc/new_features/v91_features_doc.rst`](../docs/source/Zh/doc/new_features/v91_features_doc.rst)。 diff --git a/docs/source/Eng/doc/new_features/v92_features_doc.rst b/docs/source/Eng/doc/new_features/v92_features_doc.rst new file mode 100644 index 00000000..7dcdec38 --- /dev/null +++ b/docs/source/Eng/doc/new_features/v92_features_doc.rst @@ -0,0 +1,47 @@ +Conditional HTTP Requests & Cache Validators +============================================ + +``http_request`` never sends ``If-None-Match`` / ``If-Modified-Since`` nor reads +``Cache-Control``, so every poll re-downloads an unchanged resource. This +extracts caching validators from a response, parses ``Cache-Control``, decides +freshness, and conditions the next request so the server can answer ``304 Not +Modified``. + +Pure standard library; imports no ``PySide6``. Freshness takes an explicit age +(no wall clock), so the logic is fully deterministic in CI. + +Headless API +------------ + +.. code-block:: python + + from je_auto_control import ( + store_validators, conditioned_call, is_fresh, is_not_modified, + parse_cache_control, build_call, + ) + + response = http_request(url) + validators = store_validators(response) + if is_fresh(validators, age_seconds=now - stored_at): + use_cached() + else: + revalidation = conditioned_call(build_call(url), validators) + fresh = perform(revalidation) + if is_not_modified(fresh): + use_cached() # 304 → the stored body is still valid + +``store_validators`` pulls ``etag`` / ``last_modified`` / ``date`` and the parsed +``cache_control`` from a response. ``parse_cache_control`` turns the header into +a directive dict (``max-age`` as an int, flags as ``True``). ``conditioned_call`` +adds ``If-None-Match`` / ``If-Modified-Since`` to a ``build_call`` dict. +``is_fresh`` reports whether a cached entry is still fresh for a given age +(``no-store`` / ``no-cache`` are never fresh). ``is_not_modified`` detects a +``304`` response. + +Executor commands +----------------- + +``AC_parse_cache_control`` returns ``{directives}`` for ``headers``; +``AC_store_validators`` returns ``{validators}`` for a ``response``. Both are +exposed as MCP tools (``ac_parse_cache_control`` / ``ac_store_validators``) and +as Script Builder commands under **Data**. diff --git a/docs/source/Eng/eng_index.rst b/docs/source/Eng/eng_index.rst index 1b860133..8e82f7eb 100644 --- a/docs/source/Eng/eng_index.rst +++ b/docs/source/Eng/eng_index.rst @@ -114,6 +114,7 @@ Comprehensive guides for all AutoControl features. doc/new_features/v89_features_doc doc/new_features/v90_features_doc doc/new_features/v91_features_doc + doc/new_features/v92_features_doc doc/ocr_backends/ocr_backends_doc doc/observability/observability_doc doc/operations_layer/operations_layer_doc diff --git a/docs/source/Zh/doc/new_features/v92_features_doc.rst b/docs/source/Zh/doc/new_features/v92_features_doc.rst new file mode 100644 index 00000000..cbf83592 --- /dev/null +++ b/docs/source/Zh/doc/new_features/v92_features_doc.rst @@ -0,0 +1,41 @@ +條件式 HTTP 請求與快取驗證子 +======================== + +``http_request`` 從不送出 ``If-None-Match`` / ``If-Modified-Since`` 也不讀取 ``Cache-Control``,因此每次 +輪詢都會重新下載未變更的資源。本功能從回應擷取快取驗證子、解析 ``Cache-Control``、判定新鮮度,並為下一個 +請求加上條件標頭,讓伺服器可回應 ``304 Not Modified``。 + +純標準函式庫;不匯入 ``PySide6``。新鮮度判定接受明確的 age(不使用 wall clock),因此邏輯在 CI 中完全 +具決定性。 + +無頭 API +-------- + +.. code-block:: python + + from je_auto_control import ( + store_validators, conditioned_call, is_fresh, is_not_modified, + parse_cache_control, build_call, + ) + + response = http_request(url) + validators = store_validators(response) + if is_fresh(validators, age_seconds=now - stored_at): + use_cached() + else: + revalidation = conditioned_call(build_call(url), validators) + fresh = perform(revalidation) + if is_not_modified(fresh): + use_cached() # 304 → 已儲存的內文仍有效 + +``store_validators`` 從回應取出 ``etag`` / ``last_modified`` / ``date`` 與解析後的 ``cache_control``。 +``parse_cache_control`` 把標頭轉成 directive dict(``max-age`` 為 int,旗標為 ``True``)。``conditioned_call`` +把 ``If-None-Match`` / ``If-Modified-Since`` 加到 ``build_call`` dict。``is_fresh`` 在給定 age 下回報快取項 +是否仍新鮮(``no-store`` / ``no-cache`` 永不新鮮)。``is_not_modified`` 偵測 ``304`` 回應。 + +執行器命令 +---------- + +``AC_parse_cache_control`` 對 ``headers`` 回傳 ``{directives}``;``AC_store_validators`` 對 ``response`` +回傳 ``{validators}``。兩者皆以 MCP 工具(``ac_parse_cache_control`` / ``ac_store_validators``)以及 +Script Builder 中 **Data** 分類下的命令提供。 diff --git a/docs/source/Zh/zh_index.rst b/docs/source/Zh/zh_index.rst index a1c7831e..aaf476cc 100644 --- a/docs/source/Zh/zh_index.rst +++ b/docs/source/Zh/zh_index.rst @@ -114,6 +114,7 @@ AutoControl 所有功能的完整使用指南。 doc/new_features/v89_features_doc doc/new_features/v90_features_doc doc/new_features/v91_features_doc + doc/new_features/v92_features_doc doc/ocr_backends/ocr_backends_doc doc/observability/observability_doc doc/operations_layer/operations_layer_doc diff --git a/je_auto_control/__init__.py b/je_auto_control/__init__.py index 9b853385..75ab94e5 100644 --- a/je_auto_control/__init__.py +++ b/je_auto_control/__init__.py @@ -431,6 +431,11 @@ ) # RFC 6265 cookie jar (carry a session across HTTP calls) from je_auto_control.utils.cookie_jar import CookieJar, parse_set_cookie +# Conditional HTTP requests + RFC 9111 cache validators (ETag / 304) +from je_auto_control.utils.http_conditional import ( + conditioned_call, is_fresh, is_not_modified, parse_cache_control, + store_validators, +) # W3C Trace Context propagation (traceparent / tracestate) from je_auto_control.utils.trace_context import ( SpanContext, TraceContextError, child_context, extract_context, @@ -958,6 +963,8 @@ def start_autocontrol_gui(*args, **kwargs): "build_accept", "build_accept_encoding", "decode_body", "negotiated_call", "parse_quality_values", "CookieJar", "parse_set_cookie", + "conditioned_call", "is_fresh", "is_not_modified", "parse_cache_control", + "store_validators", "SpanContext", "TraceContextError", "child_context", "extract_context", "format_traceparent", "format_tracestate", "inject_context", "new_root_context", "new_span_id", "new_trace_id", "parse_traceparent", diff --git a/je_auto_control/gui/script_builder/command_schema.py b/je_auto_control/gui/script_builder/command_schema.py index 873a8b84..33845730 100644 --- a/je_auto_control/gui/script_builder/command_schema.py +++ b/je_auto_control/gui/script_builder/command_schema.py @@ -1800,6 +1800,22 @@ def _add_resilience_specs(specs: List[CommandSpec]) -> None: ), description="Parse one Set-Cookie header into name/value/attributes.", )) + specs.append(CommandSpec( + "AC_parse_cache_control", "Data", "HTTP Conditional: Cache-Control", + fields=( + FieldSpec("headers", FieldType.STRING, + placeholder='{"Cache-Control": "max-age=60, public"}'), + ), + description="Parse a Cache-Control header into directives.", + )) + specs.append(CommandSpec( + "AC_store_validators", "Data", "HTTP Conditional: Store Validators", + fields=( + FieldSpec("response", FieldType.STRING, + placeholder='{"headers": {"ETag": "\\"abc\\""}}'), + ), + description="Extract ETag / Last-Modified / Cache-Control validators.", + )) specs.append(CommandSpec( "AC_resolve_config", "Data", "Layered Config: Resolve", fields=( diff --git a/je_auto_control/utils/executor/action_executor.py b/je_auto_control/utils/executor/action_executor.py index e6cb35f4..1160dd60 100644 --- a/je_auto_control/utils/executor/action_executor.py +++ b/je_auto_control/utils/executor/action_executor.py @@ -3030,6 +3030,24 @@ def _redact_secret_text(text: str, mask: str = "***") -> Dict[str, Any]: return {"text": redact_secret_text(text, mask=mask)} +def _parse_cache_control(headers: Any) -> Dict[str, Any]: + """Adapter: parse a Cache-Control header into {directives}.""" + import json + from je_auto_control.utils.http_conditional import parse_cache_control + if isinstance(headers, str): + headers = json.loads(headers) + return {"directives": parse_cache_control(headers)} + + +def _store_validators(response: Any) -> Dict[str, Any]: + """Adapter: extract cache validators from an HTTP response.""" + import json + from je_auto_control.utils.http_conditional import store_validators + if isinstance(response, str): + response = json.loads(response) + return {"validators": store_validators(response)} + + def _cookie_header(set_cookies: Any) -> Dict[str, Any]: """Adapter: build a Cookie header from one/many Set-Cookie strings.""" import json @@ -4372,6 +4390,8 @@ def __init__(self): "AC_parse_quality_values": _parse_quality_values, "AC_cookie_header": _cookie_header, "AC_parse_set_cookie": _parse_set_cookie, + "AC_parse_cache_control": _parse_cache_control, + "AC_store_validators": _store_validators, "AC_profile_rows": _profile_rows, "AC_infer_schema": _infer_schema, "AC_parse_problem": _parse_problem, diff --git a/je_auto_control/utils/http_conditional/__init__.py b/je_auto_control/utils/http_conditional/__init__.py new file mode 100644 index 00000000..14c8b40c --- /dev/null +++ b/je_auto_control/utils/http_conditional/__init__.py @@ -0,0 +1,10 @@ +"""Conditional HTTP requests and cache validators for AutoControl.""" +from je_auto_control.utils.http_conditional.http_conditional import ( + conditioned_call, is_fresh, is_not_modified, parse_cache_control, + store_validators, +) + +__all__ = [ + "conditioned_call", "is_fresh", "is_not_modified", "parse_cache_control", + "store_validators", +] diff --git a/je_auto_control/utils/http_conditional/http_conditional.py b/je_auto_control/utils/http_conditional/http_conditional.py new file mode 100644 index 00000000..8dd6acd1 --- /dev/null +++ b/je_auto_control/utils/http_conditional/http_conditional.py @@ -0,0 +1,77 @@ +"""Conditional HTTP requests and RFC 9111 cache validators. + +``http_request`` never sends ``If-None-Match`` / ``If-Modified-Since`` nor reads +``Cache-Control``, so every poll re-downloads an unchanged resource. This +extracts caching validators from a response, parses ``Cache-Control``, decides +freshness, and conditions the next request so the server can answer ``304 Not +Modified``. + +Pure standard library; imports no ``PySide6``. Freshness takes an explicit age +(no wall clock), so the logic is fully deterministic in CI. +""" +from typing import Any, Dict, Mapping, Optional + + +def _header(headers: Optional[Mapping[str, Any]], name: str) -> str: + for key, value in (headers or {}).items(): + if str(key).lower() == name: + return str(value) + return "" + + +def parse_cache_control(headers: Optional[Mapping[str, Any]]) -> Dict[str, Any]: + """Parse a ``Cache-Control`` header into a directive dict.""" + directives: Dict[str, Any] = {} + for part in _header(headers, "cache-control").split(","): + cleaned = part.strip() + if not cleaned: + continue + key, sep, value = cleaned.partition("=") + name = key.strip().lower() + if not sep: + directives[name] = True + continue + token = value.strip().strip('"') + try: + directives[name] = int(token) + except ValueError: + directives[name] = token + return directives + + +def store_validators(response: Mapping[str, Any]) -> Dict[str, Any]: + """Extract cache validators from an ``http_request`` response.""" + headers = response.get("headers") or {} + return {"etag": _header(headers, "etag") or None, + "last_modified": _header(headers, "last-modified") or None, + "date": _header(headers, "date") or None, + "cache_control": parse_cache_control(headers)} + + +def conditioned_call(call: Mapping[str, Any], + validators: Mapping[str, Any]) -> Dict[str, Any]: + """Return a copy of a ``build_call`` dict with conditional headers added.""" + headers = dict(call.get("headers") or {}) + if validators.get("etag"): + headers["If-None-Match"] = validators["etag"] + if validators.get("last_modified"): + headers["If-Modified-Since"] = validators["last_modified"] + out = dict(call) + out["headers"] = headers + return out + + +def is_fresh(validators: Mapping[str, Any], age_seconds: float) -> bool: + """Whether a cached entry is still fresh ``age_seconds`` after storing.""" + cache_control = validators.get("cache_control") or {} + if cache_control.get("no-store") or cache_control.get("no-cache"): + return False + max_age = cache_control.get("max-age") + if isinstance(max_age, int): + return age_seconds < max_age + return False + + +def is_not_modified(response: Mapping[str, Any]) -> bool: + """Whether ``response`` is a ``304 Not Modified``.""" + return int(response.get("status", 0)) == 304 diff --git a/je_auto_control/utils/mcp_server/tools/_factories.py b/je_auto_control/utils/mcp_server/tools/_factories.py index 3bb6c98c..37ccd3a1 100644 --- a/je_auto_control/utils/mcp_server/tools/_factories.py +++ b/je_auto_control/utils/mcp_server/tools/_factories.py @@ -3343,6 +3343,28 @@ def rate_limit_tools() -> List[MCPTool]: ] +def http_conditional_tools() -> List[MCPTool]: + return [ + MCPTool( + name="ac_parse_cache_control", + description=("Parse the 'headers' Cache-Control into {directives} " + "(max-age as int, flags as true)."), + input_schema=schema({"headers": {"type": "object"}}, ["headers"]), + handler=h.parse_cache_control, + annotations=READ_ONLY, + ), + MCPTool( + name="ac_store_validators", + description=("Extract cache validators (etag, last_modified, date, " + "cache_control) from an HTTP 'response'. Returns " + "{validators}."), + input_schema=schema({"response": {"type": "object"}}, ["response"]), + handler=h.store_validators, + annotations=READ_ONLY, + ), + ] + + def cookie_jar_tools() -> List[MCPTool]: return [ MCPTool( @@ -5296,7 +5318,7 @@ def media_assert_tools() -> List[MCPTool]: data_profile_tools, http_problem_tools, dotenv_tools, sse_client_tools, layered_config_tools, data_drift_tools, dataset_diff_tools, referential_tools, link_header_tools, multipart_tools, - http_content_tools, cookie_jar_tools, + http_content_tools, cookie_jar_tools, http_conditional_tools, saga_tools, decision_table_tools, locator_repair_tools, pii_text_tools, sarif_tools, screen_record_tools, diff --git a/je_auto_control/utils/mcp_server/tools/_handlers.py b/je_auto_control/utils/mcp_server/tools/_handlers.py index 57611d42..3ac864b8 100644 --- a/je_auto_control/utils/mcp_server/tools/_handlers.py +++ b/je_auto_control/utils/mcp_server/tools/_handlers.py @@ -1787,6 +1787,17 @@ def parse_set_cookie(header): return _parse_set_cookie(header) +def parse_cache_control(headers): + from je_auto_control.utils.executor.action_executor import ( + _parse_cache_control) + return _parse_cache_control(headers) + + +def store_validators(response): + from je_auto_control.utils.executor.action_executor import _store_validators + return _store_validators(response) + + def redact_config(obj, mask="***"): from je_auto_control.utils.executor.action_executor import _redact_config return _redact_config(obj, mask) diff --git a/test/unit_test/headless/test_http_conditional_batch.py b/test/unit_test/headless/test_http_conditional_batch.py new file mode 100644 index 00000000..5547a753 --- /dev/null +++ b/test/unit_test/headless/test_http_conditional_batch.py @@ -0,0 +1,81 @@ +"""Headless tests for conditional HTTP requests + cache validators. No Qt.""" +import json + +import je_auto_control as ac +from je_auto_control.utils.http_conditional import ( + conditioned_call, is_fresh, is_not_modified, parse_cache_control, + store_validators, +) + +_RESPONSE = { + "status": 200, + "headers": {"ETag": '"v1"', "Last-Modified": "Wed, 21 Oct 2026 07:28:00 GMT", + "Cache-Control": "max-age=60, public", "Date": "now"}, +} + + +def test_parse_cache_control(): + directives = parse_cache_control({"Cache-Control": "max-age=60, no-cache"}) + assert directives["max-age"] == 60 and directives["no-cache"] is True + + +def test_store_validators(): + validators = store_validators(_RESPONSE) + assert validators["etag"] == '"v1"' + assert validators["last_modified"].endswith("GMT") + assert validators["cache_control"]["max-age"] == 60 + + +def test_conditioned_call_adds_headers(): + validators = store_validators(_RESPONSE) + call = conditioned_call({"url": "https://api/x", "headers": {}}, validators) + assert call["headers"]["If-None-Match"] == '"v1"' + assert "If-Modified-Since" in call["headers"] + + +def test_is_fresh_by_age(): + validators = store_validators(_RESPONSE) + assert is_fresh(validators, age_seconds=30) is True + assert is_fresh(validators, age_seconds=120) is False + + +def test_no_store_is_never_fresh(): + validators = {"cache_control": {"no-store": True, "max-age": 999}} + assert is_fresh(validators, age_seconds=1) is False + + +def test_is_not_modified(): + assert is_not_modified({"status": 304}) is True + assert is_not_modified({"status": 200}) is False + + +# --- wiring --------------------------------------------------------------- + +def test_executor_round_trip(): + rec = ac.execute_action([[ + "AC_parse_cache_control", + {"headers": json.dumps({"Cache-Control": "max-age=5"})}]]) + out = next(v for v in rec.values() if isinstance(v, dict))["directives"] + assert out["max-age"] == 5 + rec2 = ac.execute_action([[ + "AC_store_validators", {"response": json.dumps(_RESPONSE)}]]) + validators = next(v for v in rec2.values() + if isinstance(v, dict))["validators"] + assert validators["etag"] == '"v1"' + + +def test_wiring(): + known = ac.executor.known_commands() + assert {"AC_parse_cache_control", "AC_store_validators"} <= set(known) + from je_auto_control.utils.mcp_server.tools import build_default_tool_registry + names = {t.name for t in build_default_tool_registry()} + assert {"ac_parse_cache_control", "ac_store_validators"} <= names + from je_auto_control.gui.script_builder.command_schema import _build_specs + specs = {s.command for s in _build_specs()} + assert {"AC_parse_cache_control", "AC_store_validators"} <= specs + + +def test_facade_exports(): + for attr in ("parse_cache_control", "store_validators", "conditioned_call", + "is_fresh", "is_not_modified"): + assert hasattr(ac, attr) and attr in ac.__all__