diff --git a/README.md b/README.md index df75980d..2fc3f3af 100644 --- a/README.md +++ b/README.md @@ -13,6 +13,7 @@ ## Table of Contents +- [What's new (2026-06-21) — JSON Schema Validation](#whats-new-2026-06-21--json-schema-validation) - [What's new (2026-06-20) — SARIF 2.1.0 Findings Export](#whats-new-2026-06-20--sarif-210-findings-export) - [What's new (2026-06-20) — Text PII Detection & Redaction](#whats-new-2026-06-20--text-pii-detection--redaction) - [What's new (2026-06-20) — Self-Healing Locator Write-Back](#whats-new-2026-06-20--self-healing-locator-write-back) @@ -109,6 +110,12 @@ --- +## What's new (2026-06-21) — JSON Schema Validation + +Validate nested JSON against a real schema. Full reference: [`docs/source/Eng/doc/new_features/v57_features_doc.rst`](docs/source/Eng/doc/new_features/v57_features_doc.rst). + +- **`validate_json` / `is_valid` / `assert_schema`** (`AC_validate_json`, `ac_validate_json`): the framework only *generated* JSON Schema and `data_quality` is a flat per-column checker — neither could validate a nested API request/response body. This adds the consumer: a JSON Schema (Draft 2020-12 subset) validator that reports **every** violation as `{path, keyword, message}` (e.g. `$.age maximum`). Covers `type` (incl. integral-float `integer`), `enum`/`const`, numeric/string bounds, array & object keywords, `allOf`/`anyOf`/`oneOf`/`not`, boolean schemas and local `$ref`. Pure-stdlib `re`; pairs with `json_query` and the `http_request` helper. + ## What's new (2026-06-20) — SARIF 2.1.0 Findings Export Unify scanner findings for GitHub code scanning. Full reference: [`docs/source/Eng/doc/new_features/v56_features_doc.rst`](docs/source/Eng/doc/new_features/v56_features_doc.rst). diff --git a/README/README_zh-CN.md b/README/README_zh-CN.md index 7b34e713..03368989 100644 --- a/README/README_zh-CN.md +++ b/README/README_zh-CN.md @@ -12,6 +12,7 @@ ## 目录 +- [本次更新 (2026-06-21) — JSON Schema 验证](#本次更新-2026-06-21--json-schema-验证) - [本次更新 (2026-06-20) — SARIF 2.1.0 发现项目导出](#本次更新-2026-06-20--sarif-210-发现项目导出) - [本次更新 (2026-06-20) — 文本 PII 检测与遮蔽](#本次更新-2026-06-20--文本-pii-检测与遮蔽) - [本次更新 (2026-06-20) — 自我修复定位器回写](#本次更新-2026-06-20--自我修复定位器回写) @@ -108,6 +109,12 @@ --- +## 本次更新 (2026-06-21) — JSON Schema 验证 + +以真正的 schema 验证嵌套 JSON。完整参考:[`docs/source/Zh/doc/new_features/v57_features_doc.rst`](../docs/source/Zh/doc/new_features/v57_features_doc.rst)。 + +- **`validate_json` / `is_valid` / `assert_schema`**(`AC_validate_json`、`ac_validate_json`):框架过去只会*产生* JSON Schema,而 `data_quality` 是扁平的逐列检查器 —— 两者都无法验证嵌套的 API 请求/响应内容。本功能补上消费端:一个 JSON Schema(Draft 2020-12 子集)验证器,将**每一个**违规以 `{path, keyword, message}` 报告(例如 `$.age maximum`)。涵盖 `type`(含整数值浮点数的 `integer`)、`enum`/`const`、数字/字符串界限、数组与对象关键字、`allOf`/`anyOf`/`oneOf`/`not`、布尔 schema 与本地 `$ref`。纯标准库 `re`;与 `json_query` 及 `http_request` 辅助函数搭配。 + ## 本次更新 (2026-06-20) — SARIF 2.1.0 发现项目导出 统一扫描结果供 GitHub 代码扫描。完整参考:[`docs/source/Zh/doc/new_features/v56_features_doc.rst`](../docs/source/Zh/doc/new_features/v56_features_doc.rst)。 diff --git a/README/README_zh-TW.md b/README/README_zh-TW.md index 58aa8143..fc9d5705 100644 --- a/README/README_zh-TW.md +++ b/README/README_zh-TW.md @@ -12,6 +12,7 @@ ## 目錄 +- [本次更新 (2026-06-21) — JSON Schema 驗證](#本次更新-2026-06-21--json-schema-驗證) - [本次更新 (2026-06-20) — SARIF 2.1.0 發現項目匯出](#本次更新-2026-06-20--sarif-210-發現項目匯出) - [本次更新 (2026-06-20) — 文字 PII 偵測與遮蔽](#本次更新-2026-06-20--文字-pii-偵測與遮蔽) - [本次更新 (2026-06-20) — 自我修復定位器回寫](#本次更新-2026-06-20--自我修復定位器回寫) @@ -108,6 +109,12 @@ --- +## 本次更新 (2026-06-21) — JSON Schema 驗證 + +以真正的 schema 驗證巢狀 JSON。完整參考:[`docs/source/Zh/doc/new_features/v57_features_doc.rst`](../docs/source/Zh/doc/new_features/v57_features_doc.rst)。 + +- **`validate_json` / `is_valid` / `assert_schema`**(`AC_validate_json`、`ac_validate_json`):框架過去只會*產生* JSON Schema,而 `data_quality` 是扁平的逐欄檢查器 —— 兩者都無法驗證巢狀的 API 請求/回應內容。本功能補上消費端:一個 JSON Schema(Draft 2020-12 子集)驗證器,將**每一個**違規以 `{path, keyword, message}` 回報(例如 `$.age maximum`)。涵蓋 `type`(含整數值浮點數的 `integer`)、`enum`/`const`、數字/字串界限、陣列與物件關鍵字、`allOf`/`anyOf`/`oneOf`/`not`、布林 schema 與本地 `$ref`。純標準函式庫 `re`;與 `json_query` 及 `http_request` 輔助函式搭配。 + ## 本次更新 (2026-06-20) — SARIF 2.1.0 發現項目匯出 統一掃描結果供 GitHub 程式碼掃描。完整參考:[`docs/source/Zh/doc/new_features/v56_features_doc.rst`](../docs/source/Zh/doc/new_features/v56_features_doc.rst)。 diff --git a/docs/source/Eng/doc/new_features/v57_features_doc.rst b/docs/source/Eng/doc/new_features/v57_features_doc.rst new file mode 100644 index 00000000..5863131a --- /dev/null +++ b/docs/source/Eng/doc/new_features/v57_features_doc.rst @@ -0,0 +1,73 @@ +JSON Schema Validation +====================== + +The framework only ever *generated* JSON Schema (action-lint, the tool-use +schema), and ``data_quality.validate_rows`` is a flat, per-column table checker — +neither can validate a nested API request/response body against a real schema. +This adds the missing consumer: validate an in-memory value against a JSON +Schema (Draft 2020-12 subset) and report **every** violation with a readable +path. + +``validate_json`` returns a ``SchemaValidationResult`` (``ok`` plus an +``errors`` list of ``{path, keyword, message}``); ``is_valid`` is the boolean +shortcut; ``assert_schema`` raises ``AutoControlAssertionException`` with a +joined summary. Pure standard library (``re``); imports no ``PySide6``. + +Supported keywords +------------------ + +* ``type`` — including ``integer`` matching integral floats (``5.0``) but never + booleans; ``enum`` / ``const`` (keeping ``True`` and ``1`` distinct). +* numbers — ``minimum`` / ``maximum`` / ``exclusiveMinimum`` / + ``exclusiveMaximum`` / ``multipleOf``. +* strings — ``minLength`` / ``maxLength`` / ``pattern``. +* arrays — ``minItems`` / ``maxItems`` / ``uniqueItems`` / ``items`` / + ``prefixItems`` / ``contains``. +* objects — ``required`` / ``minProperties`` / ``maxProperties`` / + ``properties`` / ``patternProperties`` / ``additionalProperties``. +* combinators — ``allOf`` / ``anyOf`` / ``oneOf`` / ``not``; boolean schemas + (``True`` / ``False``); local ``$ref`` (``#/$defs/...`` JSON Pointer). + +Remote ``$ref`` and ``format`` assertions are intentionally out of scope. + +Headless API +------------ + +.. code-block:: python + + from je_auto_control import validate_json, is_valid, assert_schema + + schema = { + "type": "object", + "required": ["name", "age"], + "additionalProperties": False, + "properties": { + "name": {"type": "string", "minLength": 1}, + "age": {"type": "integer", "minimum": 0, "maximum": 130}, + "roles": {"type": "array", "items": {"enum": ["admin", "user"]}, + "uniqueItems": True}, + }, + } + + result = validate_json({"name": "", "age": 200, "extra": 1}, schema) + # result.ok is False + for error in result.errors: + print(error["path"], error["keyword"], "-", error["message"]) + # $.name minLength - shorter than 1 + # $.age maximum - greater than maximum 130 + # $.extra additionalProperties - additional property not allowed + + if is_valid(payload, schema): + ... # boolean shortcut + + assert_schema(payload, schema) # raises on the first invalid value + +This pairs naturally with the existing ``json_query`` (extract a value) and the +``http_request`` helper (validate an API response body before acting on it). + +Executor command +---------------- + +``AC_validate_json`` takes ``data`` and ``schema`` (each a list/object, or a +JSON string) and returns ``{ok, errors}``. The same operation is exposed as the +MCP tool ``ac_validate_json`` and as a Script Builder command under **Data**. diff --git a/docs/source/Eng/eng_index.rst b/docs/source/Eng/eng_index.rst index 5cb65505..42311254 100644 --- a/docs/source/Eng/eng_index.rst +++ b/docs/source/Eng/eng_index.rst @@ -79,6 +79,7 @@ Comprehensive guides for all AutoControl features. doc/new_features/v54_features_doc doc/new_features/v55_features_doc doc/new_features/v56_features_doc + doc/new_features/v57_features_doc doc/ocr_backends/ocr_backends_doc doc/observability/observability_doc doc/operations_layer/operations_layer_doc diff --git a/docs/source/Zh/doc/new_features/v57_features_doc.rst b/docs/source/Zh/doc/new_features/v57_features_doc.rst new file mode 100644 index 00000000..16066e44 --- /dev/null +++ b/docs/source/Zh/doc/new_features/v57_features_doc.rst @@ -0,0 +1,71 @@ +JSON Schema 驗證 +================ + +框架過去只會*產生* JSON Schema(action-lint、tool-use schema),而 +``data_quality.validate_rows`` 是扁平、逐欄的表格檢查器 —— 兩者都無法以真正的 schema 驗證 +巢狀的 API 請求/回應內容。這補上了缺少的消費端:以 JSON Schema(Draft 2020-12 子集)驗證 +一個記憶體中的值,並以可讀的路徑回報**每一個**違規。 + +``validate_json`` 回傳 ``SchemaValidationResult``(``ok`` 加上一個由 +``{path, keyword, message}`` 組成的 ``errors`` 清單);``is_valid`` 是布林捷徑; +``assert_schema`` 會以彙整後的摘要拋出 ``AutoControlAssertionException``。純標準函式庫 +(``re``);不匯入 ``PySide6``。 + +支援的關鍵字 +------------ + +* ``type`` —— 包含 ``integer`` 會匹配整數值的浮點數(``5.0``)但永不匹配布林值; + ``enum`` / ``const``(讓 ``True`` 與 ``1`` 保持相異)。 +* 數字 —— ``minimum`` / ``maximum`` / ``exclusiveMinimum`` / ``exclusiveMaximum`` / + ``multipleOf``。 +* 字串 —— ``minLength`` / ``maxLength`` / ``pattern``。 +* 陣列 —— ``minItems`` / ``maxItems`` / ``uniqueItems`` / ``items`` / + ``prefixItems`` / ``contains``。 +* 物件 —— ``required`` / ``minProperties`` / ``maxProperties`` / ``properties`` / + ``patternProperties`` / ``additionalProperties``。 +* 組合器 —— ``allOf`` / ``anyOf`` / ``oneOf`` / ``not``;布林 schema(``True`` / + ``False``);本地 ``$ref``(``#/$defs/...`` JSON Pointer)。 + +遠端 ``$ref`` 與 ``format`` 斷言刻意排除在範圍之外。 + +無頭 API +-------- + +.. code-block:: python + + from je_auto_control import validate_json, is_valid, assert_schema + + schema = { + "type": "object", + "required": ["name", "age"], + "additionalProperties": False, + "properties": { + "name": {"type": "string", "minLength": 1}, + "age": {"type": "integer", "minimum": 0, "maximum": 130}, + "roles": {"type": "array", "items": {"enum": ["admin", "user"]}, + "uniqueItems": True}, + }, + } + + result = validate_json({"name": "", "age": 200, "extra": 1}, schema) + # result.ok 為 False + for error in result.errors: + print(error["path"], error["keyword"], "-", error["message"]) + # $.name minLength - shorter than 1 + # $.age maximum - greater than maximum 130 + # $.extra additionalProperties - additional property not allowed + + if is_valid(payload, schema): + ... # 布林捷徑 + + assert_schema(payload, schema) # 遇到第一個無效值即拋出 + +這與既有的 ``json_query``(擷取一個值)以及 ``http_request`` 輔助函式(在處理前先驗證 API +回應內容)自然搭配。 + +執行器命令 +---------- + +``AC_validate_json`` 接受 ``data`` 與 ``schema``(各為 list/object 或 JSON 字串),回傳 +``{ok, errors}``。同一操作亦以 MCP 工具 ``ac_validate_json`` 以及 Script Builder 中 +**Data** 分類下的命令提供。 diff --git a/docs/source/Zh/zh_index.rst b/docs/source/Zh/zh_index.rst index cca026f2..92cd092a 100644 --- a/docs/source/Zh/zh_index.rst +++ b/docs/source/Zh/zh_index.rst @@ -79,6 +79,7 @@ AutoControl 所有功能的完整使用指南。 doc/new_features/v54_features_doc doc/new_features/v55_features_doc doc/new_features/v56_features_doc + doc/new_features/v57_features_doc doc/ocr_backends/ocr_backends_doc doc/observability/observability_doc doc/operations_layer/operations_layer_doc diff --git a/je_auto_control/__init__.py b/je_auto_control/__init__.py index 5c3adaca..c00c65b7 100644 --- a/je_auto_control/__init__.py +++ b/je_auto_control/__init__.py @@ -298,6 +298,10 @@ from je_auto_control.utils.sarif import ( from_audit_findings, from_lint_issues, make_finding, to_sarif, write_sarif, ) +# JSON Schema (Draft 2020-12 subset) validation of parsed JSON +from je_auto_control.utils.json_schema import ( + SchemaValidationResult, assert_schema, is_valid, validate_json, +) # Background popup/interrupt watchdog (unattended automation) from je_auto_control.utils.watchdog import ( PopupWatchdog, WatchdogRule, default_popup_watchdog, @@ -767,6 +771,7 @@ def start_autocontrol_gui(*args, **kwargs): "PIIFinding", "detect_pii", "redact_pii_text", "from_audit_findings", "from_lint_issues", "make_finding", "to_sarif", "write_sarif", + "SchemaValidationResult", "assert_schema", "is_valid", "validate_json", # MCP server "AuditLogger", "HttpMCPServer", "MCPContent", "MCPPrompt", "MCPPromptArgument", "MCPResource", "MCPServer", "MCPTool", diff --git a/je_auto_control/gui/script_builder/command_schema.py b/je_auto_control/gui/script_builder/command_schema.py index 5fc1bd43..c7b91b42 100644 --- a/je_auto_control/gui/script_builder/command_schema.py +++ b/je_auto_control/gui/script_builder/command_schema.py @@ -1145,6 +1145,16 @@ def _add_misc_specs(specs: List[CommandSpec]) -> None: ), description="Extract a {key: jsonpath} mapping into a flat object.", )) + specs.append(CommandSpec( + "AC_validate_json", "Data", "JSON Schema: Validate", + fields=( + FieldSpec("data", FieldType.STRING, + placeholder='{"name": "Jo", "age": 30}'), + FieldSpec("schema", FieldType.STRING, + placeholder='{"type": "object", "required": ["name"]}'), + ), + description="Validate JSON against a JSON Schema; returns {ok, errors}.", + )) specs.append(CommandSpec( "AC_run_saga", "Flow", "Run Saga (Compensating Rollback)", fields=( diff --git a/je_auto_control/utils/executor/action_executor.py b/je_auto_control/utils/executor/action_executor.py index 65f2e984..8b513316 100644 --- a/je_auto_control/utils/executor/action_executor.py +++ b/je_auto_control/utils/executor/action_executor.py @@ -3316,6 +3316,17 @@ def _json_extract(data: Any, mapping: Any) -> Dict[str, Any]: return {"result": json_extract(data, mapping)} +def _validate_json(data: Any, schema: Any) -> Dict[str, Any]: + """Adapter: validate data against a JSON Schema (each JSON string or object).""" + import json + from je_auto_control.utils.json_schema import validate_json + if isinstance(data, str): + data = json.loads(data) + if isinstance(schema, str): + schema = json.loads(schema) + return validate_json(data, schema).to_dict() + + def _run_saga(steps: Any) -> Dict[str, Any]: """Adapter: run a saga (steps with compensating rollback) from a spec.""" import json @@ -3686,6 +3697,7 @@ def __init__(self): "AC_notify_webhook": _notify_webhook, "AC_json_query": _json_query, "AC_json_extract": _json_extract, + "AC_validate_json": _validate_json, "AC_run_saga": _run_saga, "AC_decision_table": _decision_table, "AC_repair_record": _repair_record, diff --git a/je_auto_control/utils/json_schema/__init__.py b/je_auto_control/utils/json_schema/__init__.py new file mode 100644 index 00000000..f26e7cea --- /dev/null +++ b/je_auto_control/utils/json_schema/__init__.py @@ -0,0 +1,8 @@ +"""JSON Schema (Draft 2020-12 subset) validation over parsed JSON.""" +from je_auto_control.utils.json_schema.json_schema import ( + SchemaValidationResult, assert_schema, is_valid, validate_json, +) + +__all__ = [ + "SchemaValidationResult", "assert_schema", "is_valid", "validate_json", +] diff --git a/je_auto_control/utils/json_schema/json_schema.py b/je_auto_control/utils/json_schema/json_schema.py new file mode 100644 index 00000000..a61914aa --- /dev/null +++ b/je_auto_control/utils/json_schema/json_schema.py @@ -0,0 +1,346 @@ +"""A focused JSON Schema (Draft 2020-12) validator over parsed JSON. + +The framework only ever *generates* JSON Schema (action-lint, tool-use schema) +and ``data_quality.validate_rows`` is a flat, tabular column checker — neither +can validate a nested API request/response body against a real schema. This +adds the missing consumer: validate an in-memory value against a JSON Schema +subset and report every violation with a readable path. + +Supported keywords: ``type`` (incl. ``integer`` matching integral floats), +``enum``/``const``, the numeric bounds (``minimum``/``maximum``/ +``exclusiveMinimum``/``exclusiveMaximum``/``multipleOf``), the string bounds +(``minLength``/``maxLength``/``pattern``), the array keywords +(``minItems``/``maxItems``/``uniqueItems``/``items``/``prefixItems``/ +``contains``), the object keywords (``required``/``minProperties``/ +``maxProperties``/``properties``/``patternProperties``/ +``additionalProperties``), the combinators (``allOf``/``anyOf``/``oneOf``/ +``not``), boolean schemas (``True``/``False``) and local ``$ref`` +(``#/$defs/...`` JSON Pointer). Remote ``$ref`` and format assertions are out +of scope. + +Pure standard library (``re``); imports no ``PySide6``. +""" +import json +import re +from dataclasses import dataclass, field +from typing import Any, Callable, Dict, List + +from je_auto_control.utils.exception.exceptions import ( + AutoControlAssertionException, AutoControlJsonException) + +Schema = Any # a dict, or a bool (a boolean schema) + + +@dataclass(frozen=True) +class SchemaValidationResult: + """Outcome of validating one value against a schema.""" + + ok: bool + errors: List[Dict[str, str]] = field(default_factory=list) + + def to_dict(self) -> Dict[str, Any]: + """Return a plain-dict view for JSON/executor/MCP responses.""" + return {"ok": self.ok, "errors": list(self.errors)} + + +def _err(path: str, keyword: str, message: str) -> Dict[str, str]: + return {"path": path, "keyword": keyword, "message": message} + + +def _is_number(value: Any) -> bool: + return isinstance(value, (int, float)) and not isinstance(value, bool) + + +def _is_integer(value: Any) -> bool: + if isinstance(value, bool): + return False + if isinstance(value, int): + return True + return isinstance(value, float) and value.is_integer() + + +_TYPE_CHECKS: Dict[str, Callable[[Any], bool]] = { + "null": lambda v: v is None, + "boolean": lambda v: isinstance(v, bool), + "object": lambda v: isinstance(v, dict), + "array": lambda v: isinstance(v, list), + "string": lambda v: isinstance(v, str), + "number": _is_number, + "integer": _is_integer, +} + + +def _json_equal(left: Any, right: Any) -> bool: + """Equality that keeps ``True``/``1`` and ``False``/``0`` distinct.""" + if isinstance(left, bool) or isinstance(right, bool): + return left is right + return left == right + + +def _is_multiple(value: float, factor: float) -> bool: + if factor == 0: + return False + quotient = value / factor + return abs(quotient - round(quotient)) < 1e-9 + + +def _has_duplicates(items: List[Any]) -> bool: + seen = set() + for item in items: + key = json.dumps(item, sort_keys=True, default=str) + if key in seen: + return True + seen.add(key) + return False + + +# --- keyword checkers (each returns a list of error dicts) ---------------- + +def _check_type(instance: Any, schema: Dict, path: str, _root: Schema) -> List[Dict]: + if "type" not in schema: + return [] + types = schema["type"] + names = [types] if isinstance(types, str) else types + if any(_TYPE_CHECKS.get(name, lambda _v: True)(instance) for name in names): + return [] + return [_err(path, "type", f"expected type {schema['type']}")] + + +def _check_enum_const(instance: Any, schema: Dict, path: str, _root: Schema) -> List[Dict]: + errors: List[Dict] = [] + if "const" in schema and not _json_equal(instance, schema["const"]): + errors.append(_err(path, "const", f"must equal {schema['const']!r}")) + if "enum" in schema and not any( + _json_equal(instance, option) for option in schema["enum"]): + errors.append(_err(path, "enum", f"must be one of {schema['enum']}")) + return errors + + +_NUMBER_BOUNDS = ( + ("minimum", lambda v, b: v >= b, "less than minimum"), + ("maximum", lambda v, b: v <= b, "greater than maximum"), + ("exclusiveMinimum", lambda v, b: v > b, "not greater than exclusiveMinimum"), + ("exclusiveMaximum", lambda v, b: v < b, "not less than exclusiveMaximum"), +) + + +def _check_number(instance: Any, schema: Dict, path: str, _root: Schema) -> List[Dict]: + if not _is_number(instance): + return [] + errors: List[Dict] = [] + for key, ok, message in _NUMBER_BOUNDS: + if key in schema and not ok(instance, schema[key]): + errors.append(_err(path, key, f"{message} {schema[key]}")) + if "multipleOf" in schema and not _is_multiple(instance, schema["multipleOf"]): + errors.append( + _err(path, "multipleOf", f"not a multiple of {schema['multipleOf']}")) + return errors + + +def _check_string(instance: Any, schema: Dict, path: str, _root: Schema) -> List[Dict]: + if not isinstance(instance, str): + return [] + errors: List[Dict] = [] + if "minLength" in schema and len(instance) < schema["minLength"]: + errors.append(_err(path, "minLength", f"shorter than {schema['minLength']}")) + if "maxLength" in schema and len(instance) > schema["maxLength"]: + errors.append(_err(path, "maxLength", f"longer than {schema['maxLength']}")) + if "pattern" in schema and not re.search(schema["pattern"], instance): + errors.append(_err(path, "pattern", f"does not match /{schema['pattern']}/")) + return errors + + +def _array_size_errors(instance: List, schema: Dict, path: str) -> List[Dict]: + errors: List[Dict] = [] + if "minItems" in schema and len(instance) < schema["minItems"]: + errors.append(_err(path, "minItems", f"fewer than {schema['minItems']} items")) + if "maxItems" in schema and len(instance) > schema["maxItems"]: + errors.append(_err(path, "maxItems", f"more than {schema['maxItems']} items")) + if schema.get("uniqueItems") and _has_duplicates(instance): + errors.append(_err(path, "uniqueItems", "items are not unique")) + return errors + + +def _array_item_errors(instance: List, schema: Dict, path: str, root: Schema) -> List[Dict]: + errors: List[Dict] = [] + prefix = schema.get("prefixItems") + start = 0 + if isinstance(prefix, list): + for index, subschema in enumerate(prefix[:len(instance)]): + errors.extend(_validate(instance[index], subschema, f"{path}[{index}]", root)) + start = len(prefix) + items = schema.get("items") + if isinstance(items, (dict, bool)): + for index in range(start, len(instance)): + errors.extend(_validate(instance[index], items, f"{path}[{index}]", root)) + return errors + + +def _array_contains_errors(instance: List, schema: Dict, path: str, root: Schema) -> List[Dict]: + if "contains" not in schema: + return [] + subschema = schema["contains"] + if any(not _validate(item, subschema, path, root) for item in instance): + return [] + return [_err(path, "contains", "no items match the 'contains' schema")] + + +def _check_array(instance: Any, schema: Dict, path: str, root: Schema) -> List[Dict]: + if not isinstance(instance, list): + return [] + errors = _array_size_errors(instance, schema, path) + errors.extend(_array_item_errors(instance, schema, path, root)) + errors.extend(_array_contains_errors(instance, schema, path, root)) + return errors + + +def _object_size_required_errors(instance: Dict, schema: Dict, path: str) -> List[Dict]: + errors: List[Dict] = [] + for key in schema.get("required", []): + if key not in instance: + errors.append(_err(path, "required", f"missing required property '{key}'")) + if "minProperties" in schema and len(instance) < schema["minProperties"]: + errors.append( + _err(path, "minProperties", f"fewer than {schema['minProperties']} properties")) + if "maxProperties" in schema and len(instance) > schema["maxProperties"]: + errors.append( + _err(path, "maxProperties", f"more than {schema['maxProperties']} properties")) + return errors + + +def _additional_property_errors(value: Any, additional: Any, path: str, root: Schema) -> List[Dict]: + if additional is None or additional is True: + return [] + if additional is False: + return [_err(path, "additionalProperties", "additional property not allowed")] + return _validate(value, additional, path, root) + + +def _one_property_errors(key: str, value: Any, schema: Dict, path: str, root: Schema) -> List[Dict]: + child = f"{path}.{key}" + matched: List[Schema] = [] + if key in schema.get("properties", {}): + matched.append(schema["properties"][key]) + matched.extend( + sub for pattern, sub in schema.get("patternProperties", {}).items() + if re.search(pattern, key)) + if matched: + errors: List[Dict] = [] + for subschema in matched: + errors.extend(_validate(value, subschema, child, root)) + return errors + return _additional_property_errors( + value, schema.get("additionalProperties"), child, root) + + +def _check_object(instance: Any, schema: Dict, path: str, root: Schema) -> List[Dict]: + if not isinstance(instance, dict): + return [] + errors = _object_size_required_errors(instance, schema, path) + for key, value in instance.items(): + errors.extend(_one_property_errors(key, value, schema, path, root)) + return errors + + +def _check_all_of(instance: Any, schema: Dict, path: str, root: Schema) -> List[Dict]: + errors: List[Dict] = [] + for subschema in schema.get("allOf", []): + errors.extend(_validate(instance, subschema, path, root)) + return errors + + +def _check_any_of(instance: Any, schema: Dict, path: str, root: Schema) -> List[Dict]: + options = schema.get("anyOf") + if not options: + return [] + if any(not _validate(instance, sub, path, root) for sub in options): + return [] + return [_err(path, "anyOf", "does not match any schema in anyOf")] + + +def _check_one_of(instance: Any, schema: Dict, path: str, root: Schema) -> List[Dict]: + options = schema.get("oneOf") + if not options: + return [] + matches = sum(1 for sub in options if not _validate(instance, sub, path, root)) + if matches == 1: + return [] + return [_err(path, "oneOf", f"matched {matches} schemas in oneOf, expected 1")] + + +def _check_not(instance: Any, schema: Dict, path: str, root: Schema) -> List[Dict]: + if "not" not in schema: + return [] + if _validate(instance, schema["not"], path, root): + return [] + return [_err(path, "not", "must not match the 'not' schema")] + + +def _check_combinators(instance: Any, schema: Dict, path: str, root: Schema) -> List[Dict]: + errors = _check_all_of(instance, schema, path, root) + errors.extend(_check_any_of(instance, schema, path, root)) + errors.extend(_check_one_of(instance, schema, path, root)) + errors.extend(_check_not(instance, schema, path, root)) + return errors + + +_CHECKERS = ( + _check_type, _check_enum_const, _check_number, _check_string, + _check_array, _check_object, _check_combinators, +) + + +def _ref_step(node: Any, token: str, ref: str) -> Any: + if isinstance(node, dict) and token in node: + return node[token] + if isinstance(node, list) and token.isdigit() and int(token) < len(node): + return node[int(token)] + raise AutoControlJsonException(f"cannot resolve $ref {ref!r}") + + +def _resolve_ref(ref: str, root: Schema) -> Schema: + if not ref.startswith("#"): + raise AutoControlJsonException(f"only local $ref is supported, got {ref!r}") + pointer = ref[1:].lstrip("/") + if not pointer: + return root + node = root + for raw in pointer.split("/"): + token = raw.replace("~1", "/").replace("~0", "~") + node = _ref_step(node, token, ref) + return node + + +def _validate(instance: Any, schema: Schema, path: str, root: Schema) -> List[Dict]: + if schema is True: + return [] + if schema is False: + return [_err(path, "schema", "no value is allowed here")] + if not isinstance(schema, dict): + return [] + if "$ref" in schema: + return _validate(instance, _resolve_ref(schema["$ref"], root), path, root) + errors: List[Dict] = [] + for checker in _CHECKERS: + errors.extend(checker(instance, schema, path, root)) + return errors + + +def validate_json(instance: Any, schema: Schema) -> SchemaValidationResult: + """Validate ``instance`` against ``schema``; collect every violation.""" + errors = _validate(instance, schema, "$", schema) + return SchemaValidationResult(ok=not errors, errors=errors) + + +def is_valid(instance: Any, schema: Schema) -> bool: + """Return ``True`` when ``instance`` satisfies ``schema``.""" + return not _validate(instance, schema, "$", schema) + + +def assert_schema(instance: Any, schema: Schema) -> None: + """Raise ``AutoControlAssertionException`` if ``instance`` is invalid.""" + result = validate_json(instance, schema) + if not result.ok: + summary = "; ".join(f"{e['path']}: {e['message']}" for e in result.errors) + raise AutoControlAssertionException( + f"JSON Schema validation failed: {summary}") diff --git a/je_auto_control/utils/mcp_server/tools/_factories.py b/je_auto_control/utils/mcp_server/tools/_factories.py index 64bdd3b9..ce03e453 100644 --- a/je_auto_control/utils/mcp_server/tools/_factories.py +++ b/je_auto_control/utils/mcp_server/tools/_factories.py @@ -3226,6 +3226,24 @@ def jsonpath_tools() -> List[MCPTool]: ] +def json_schema_tools() -> List[MCPTool]: + return [ + MCPTool( + name="ac_validate_json", + description=("Validate parsed JSON 'data' against a JSON Schema " + "(Draft 2020-12 subset: type/enum/const, numeric & " + "string bounds, array/object keywords, allOf/anyOf/" + "oneOf/not, local $ref). Returns {ok, errors:[{path, " + "keyword, message}]}."), + input_schema=schema( + {"data": {"type": "object"}, "schema": {"type": "object"}}, + ["data", "schema"]), + handler=h.validate_json, + annotations=READ_ONLY, + ), + ] + + def saga_tools() -> List[MCPTool]: return [ MCPTool( @@ -4420,7 +4438,8 @@ def media_assert_tools() -> List[MCPTool]: video_report_tools, fuzzy_tools, artifact_store_tools, image_dedup_tools, locale_tools, voice_tools, coordinate_space_tools, loop_guard_tools, process_mining_tools, asset_tools, events_tools, notify_channel_tools, - jsonpath_tools, saga_tools, decision_table_tools, locator_repair_tools, + jsonpath_tools, json_schema_tools, saga_tools, decision_table_tools, + locator_repair_tools, pii_text_tools, sarif_tools, screen_record_tools, process_and_shell_tools, remote_desktop_tools, gamepad_tools, diff --git a/je_auto_control/utils/mcp_server/tools/_handlers.py b/je_auto_control/utils/mcp_server/tools/_handlers.py index 6d067b23..f73fe9ef 100644 --- a/je_auto_control/utils/mcp_server/tools/_handlers.py +++ b/je_auto_control/utils/mcp_server/tools/_handlers.py @@ -1556,6 +1556,11 @@ def json_extract(data, mapping): return {"result": _x(data, mapping)} +def validate_json(data, schema): + from je_auto_control.utils.json_schema import validate_json as _v + return _v(data, schema).to_dict() + + def run_saga(steps): from je_auto_control.utils.saga import run_saga as _run result = _run(steps) diff --git a/test/unit_test/headless/test_json_schema_batch.py b/test/unit_test/headless/test_json_schema_batch.py new file mode 100644 index 00000000..35b858d6 --- /dev/null +++ b/test/unit_test/headless/test_json_schema_batch.py @@ -0,0 +1,148 @@ +"""Headless tests for the JSON Schema validator. Pure stdlib, no Qt imports.""" +import json + +import pytest + +import je_auto_control as ac +from je_auto_control.utils.exception.exceptions import ( + AutoControlAssertionException, AutoControlJsonException) +from je_auto_control.utils.json_schema import ( + SchemaValidationResult, assert_schema, is_valid, validate_json) + +PERSON = { + "type": "object", + "required": ["name", "age"], + "additionalProperties": False, + "properties": { + "name": {"type": "string", "minLength": 1}, + "age": {"type": "integer", "minimum": 0, "maximum": 130}, + "roles": {"type": "array", "items": {"enum": ["admin", "user"]}, + "uniqueItems": True}, + "email": {"type": "string", "pattern": "@"}, + }, +} + + +def test_valid_object_passes(): + value = {"name": "Jo", "age": 30, "roles": ["admin"], "email": "a@b.com"} + result = validate_json(value, PERSON) + assert isinstance(result, SchemaValidationResult) + assert result.ok is True + assert result.errors == [] + assert is_valid(value, PERSON) is True + + +def test_invalid_object_collects_all_errors(): + bad = {"name": "", "age": 200, "roles": ["admin", "admin"], + "email": "nope", "extra": 1} + result = validate_json(bad, PERSON) + assert result.ok is False + keywords = {(e["path"], e["keyword"]) for e in result.errors} + assert ("$.name", "minLength") in keywords + assert ("$.age", "maximum") in keywords + assert ("$.roles", "uniqueItems") in keywords + assert ("$.email", "pattern") in keywords + assert ("$.extra", "additionalProperties") in keywords + + +def test_required_property_missing(): + result = validate_json({"name": "Jo"}, PERSON) + assert ("$", "required") in {(e["path"], e["keyword"]) for e in result.errors} + + +def test_integer_accepts_integral_float_but_not_bool(): + assert is_valid(5.0, {"type": "integer"}) is True + assert is_valid(5.5, {"type": "integer"}) is False + assert is_valid(True, {"type": "integer"}) is False + assert is_valid(3, {"type": "number"}) is True + + +def test_const_keeps_bool_and_int_distinct(): + assert is_valid(True, {"const": 1}) is False + assert is_valid(1, {"const": 1}) is True + assert is_valid("x", {"enum": ["x", "y"]}) is True + assert is_valid("z", {"enum": ["x", "y"]}) is False + + +def test_numeric_bounds_and_multiple_of(): + schema = {"type": "number", "exclusiveMinimum": 0, "multipleOf": 0.5} + assert is_valid(1.5, schema) is True + assert is_valid(0, schema) is False + assert is_valid(1.3, schema) is False + + +def test_array_items_and_prefix_items(): + assert validate_json([1, "x"], {"prefixItems": [{"type": "integer"}, + {"type": "string"}]}).errors == [] + bad = validate_json([1, 2], {"items": {"type": "string"}}) + assert bad.ok is False + assert {e["path"] for e in bad.errors} == {"$[0]", "$[1]"} + + +def test_contains_and_size(): + schema = {"type": "array", "minItems": 1, "contains": {"type": "integer"}} + assert is_valid([1, "a"], schema) is True + assert is_valid(["a", "b"], schema) is False + + +def test_combinators(): + assert is_valid(5, {"oneOf": [{"type": "integer"}, {"type": "string"}]}) is True + assert is_valid(5, {"anyOf": [{"type": "integer"}, {"type": "string"}]}) is True + assert is_valid("s", {"not": {"type": "integer"}}) is True + both = {"allOf": [{"type": "integer"}, {"minimum": 10}]} + assert is_valid(5, both) is False + + +def test_local_ref_resolution(): + schema = {"$defs": {"pos": {"type": "integer", "minimum": 1}}, + "properties": {"n": {"$ref": "#/$defs/pos"}}} + assert is_valid({"n": 3}, schema) is True + bad = validate_json({"n": -1}, schema) + assert bad.errors[0]["path"] == "$.n" + assert bad.errors[0]["keyword"] == "minimum" + + +def test_unresolvable_ref_raises(): + with pytest.raises(AutoControlJsonException): + validate_json({"n": 1}, {"properties": {"n": {"$ref": "#/$defs/missing"}}}) + + +def test_boolean_schema(): + assert is_valid(123, True) is True + assert is_valid(123, False) is False + + +def test_assert_schema_raises_on_invalid(): + with pytest.raises(AutoControlAssertionException): + assert_schema({"age": 5}, PERSON) + assert assert_schema({"name": "Jo", "age": 5}, PERSON) is None + + +# --- wiring --------------------------------------------------------------- + +def test_executor_round_trip(): + rec = ac.execute_action([[ + "AC_validate_json", + {"data": json.dumps({"name": "", "age": 1}), + "schema": json.dumps(PERSON)}, + ]]) + payload = next(v for v in rec.values() if isinstance(v, dict)) + assert payload["ok"] is False + assert any(e["keyword"] == "minLength" for e in payload["errors"]) + + +def test_wiring(): + assert "AC_validate_json" in ac.executor.known_commands() + from je_auto_control.utils.mcp_server.tools import build_default_tool_registry + names = {t.name for t in build_default_tool_registry()} + assert "ac_validate_json" in names + from je_auto_control.gui.script_builder.command_schema import _build_specs + cmds = {s.command for s in _build_specs()} + assert "AC_validate_json" in cmds + + +def test_facade_exports(): + for attr in ("validate_json", "is_valid", "assert_schema", + "SchemaValidationResult"): + assert hasattr(ac, attr) + assert attr in ac.__all__