Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@

## Table of Contents

- [What's new (2026-06-22) — URI-Scheme Value References](#whats-new-2026-06-22--uri-scheme-value-references)
- [What's new (2026-06-21) — W3C Baggage Propagation](#whats-new-2026-06-21--w3c-baggage-propagation)
- [What's new (2026-06-21) — Dataset Diff (Row-Set Change Report)](#whats-new-2026-06-21--dataset-diff-row-set-change-report)
- [What's new (2026-06-21) — Distribution Drift Detection](#whats-new-2026-06-21--distribution-drift-detection)
Expand Down Expand Up @@ -137,6 +138,12 @@

---

## What's new (2026-06-22) — URI-Scheme Value References

Store pointers, not secrets, in config. Full reference: [`docs/source/Eng/doc/new_features/v85_features_doc.rst`](docs/source/Eng/doc/new_features/v85_features_doc.rst).

- **`resolve_ref` / `resolve_refs_in` / `is_ref` / `RefResolver`** (`AC_resolve_ref`, `AC_resolve_refs`): `interpolate` hardcoded only `${secrets.NAME}` and `AssetStore` refs were vault-name-only — there was no general read-time indirection. This resolves `env://VAR`, `file://path` (with an optional `base_dir` traversal guard), and `secret://name` (injectable resolver or the governance broker), and walks nested structures resolving every reference. Env reader / secret resolver / base dir are injectable. Pure-stdlib, deterministic.

## What's new (2026-06-21) — W3C Baggage Propagation

Carry cross-cutting key-value context across HTTP. Full reference: [`docs/source/Eng/doc/new_features/v84_features_doc.rst`](docs/source/Eng/doc/new_features/v84_features_doc.rst).
Expand Down
7 changes: 7 additions & 0 deletions README/README_zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@

## 目录

- [本次更新 (2026-06-22) — URI-Scheme 值引用](#本次更新-2026-06-22--uri-scheme-值引用)
- [本次更新 (2026-06-21) — W3C Baggage 传播](#本次更新-2026-06-21--w3c-baggage-传播)
- [本次更新 (2026-06-21) — 数据集差异(数据行变更报告)](#本次更新-2026-06-21--数据集差异数据行变更报告)
- [本次更新 (2026-06-21) — 分布漂移检测](#本次更新-2026-06-21--分布漂移检测)
Expand Down Expand Up @@ -136,6 +137,12 @@

---

## 本次更新 (2026-06-22) — URI-Scheme 值引用

在配置中存储指针而非机密。完整参考:[`docs/source/Zh/doc/new_features/v85_features_doc.rst`](../docs/source/Zh/doc/new_features/v85_features_doc.rst)。

- **`resolve_ref` / `resolve_refs_in` / `is_ref` / `RefResolver`**(`AC_resolve_ref`、`AC_resolve_refs`):`interpolate` 只写死 `${secrets.NAME}`,`AssetStore` 引用仅限 vault 名称 —— 没有通用的读取时间接。本功能解析 `env://VAR`、`file://path`(可选 `base_dir` 防穿越保护)与 `secret://name`(可注入解析器或 governance broker),并遍历嵌套结构解析每个引用。env 读取器 / secret 解析器 / 基目录均可注入。纯标准库、确定。

## 本次更新 (2026-06-21) — W3C Baggage 传播

跨 HTTP 携带横切键值上下文。完整参考:[`docs/source/Zh/doc/new_features/v84_features_doc.rst`](../docs/source/Zh/doc/new_features/v84_features_doc.rst)。
Expand Down
7 changes: 7 additions & 0 deletions README/README_zh-TW.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@

## 目錄

- [本次更新 (2026-06-22) — URI-Scheme 值參照](#本次更新-2026-06-22--uri-scheme-值參照)
- [本次更新 (2026-06-21) — W3C Baggage 傳播](#本次更新-2026-06-21--w3c-baggage-傳播)
- [本次更新 (2026-06-21) — 資料集差異(資料列變更報告)](#本次更新-2026-06-21--資料集差異資料列變更報告)
- [本次更新 (2026-06-21) — 分布漂移偵測](#本次更新-2026-06-21--分布漂移偵測)
Expand Down Expand Up @@ -136,6 +137,12 @@

---

## 本次更新 (2026-06-22) — URI-Scheme 值參照

在設定中儲存指標而非機密。完整參考:[`docs/source/Zh/doc/new_features/v85_features_doc.rst`](../docs/source/Zh/doc/new_features/v85_features_doc.rst)。

- **`resolve_ref` / `resolve_refs_in` / `is_ref` / `RefResolver`**(`AC_resolve_ref`、`AC_resolve_refs`):`interpolate` 只寫死 `${secrets.NAME}`,`AssetStore` 參照僅限 vault 名稱 —— 沒有通用的讀取時間接。本功能解析 `env://VAR`、`file://path`(可選 `base_dir` 防穿越保護)與 `secret://name`(可注入解析器或 governance broker),並走訪巢狀結構解析每個參照。env 讀取器 / secret 解析器 / 基底目錄皆可注入。純標準函式庫、具決定性。

## 本次更新 (2026-06-21) — W3C Baggage 傳播

跨 HTTP 攜帶橫切鍵值脈絡。完整參考:[`docs/source/Zh/doc/new_features/v84_features_doc.rst`](../docs/source/Zh/doc/new_features/v84_features_doc.rst)。
Expand Down
44 changes: 44 additions & 0 deletions docs/source/Eng/doc/new_features/v85_features_doc.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
URI-Scheme Value References
===========================

``script_vars.interpolate`` hardcodes a single indirection (``${secrets.NAME}``
→ vault) and ``AssetStore`` credential references are vault-name-only. There was
no general, pluggable read-time indirection — the modern config pattern of
storing a *pointer* (``env://TOKEN``, ``file://./token``, ``secret://api-key``)
rather than the value. This adds that resolver.

Pure standard library (``os`` / ``re``); imports no ``PySide6``. The env reader,
secret resolver, and base directory are injectable, so resolution is safe and
deterministic in CI.

Headless API
------------

.. code-block:: python

from je_auto_control import resolve_ref, resolve_refs_in, RefResolver

token = resolve_ref("env://API_TOKEN")
key = resolve_ref("file://./secrets/key.pem")

config = resolve_refs_in({
"token": "env://API_TOKEN",
"db": {"password": "secret://db-password"},
})

``resolve_ref`` resolves one reference: ``env://`` reads an environment variable
(from an injectable mapping, falling back to ``os.environ``), ``file://`` reads a
file (with an optional ``base_dir`` realpath guard against traversal), and
``secret://`` delegates to an injectable resolver or the governance credential
broker. ``resolve_refs_in`` walks a nested dict/list and resolves every
reference in place, leaving non-reference values untouched. ``is_ref`` tests a
value, and ``RefResolver`` bundles the injectable backends for repeated use.
Unresolvable or unknown-scheme references raise ``SecretRefError``.

Executor commands
-----------------

``AC_resolve_ref`` resolves a single ``ref`` into ``{value}``;
``AC_resolve_refs`` resolves every reference inside ``obj`` and returns
``{resolved}``. Both are exposed as MCP tools (``ac_resolve_ref`` /
``ac_resolve_refs``) and as Script Builder commands under **Security**.
1 change: 1 addition & 0 deletions docs/source/Eng/eng_index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,7 @@ Comprehensive guides for all AutoControl features.
doc/new_features/v82_features_doc
doc/new_features/v83_features_doc
doc/new_features/v84_features_doc
doc/new_features/v85_features_doc
doc/ocr_backends/ocr_backends_doc
doc/observability/observability_doc
doc/operations_layer/operations_layer_doc
Expand Down
37 changes: 37 additions & 0 deletions docs/source/Zh/doc/new_features/v85_features_doc.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
URI-Scheme 值參照
================

``script_vars.interpolate`` 把單一間接寫死(``${secrets.NAME}`` → vault),而 ``AssetStore`` 的憑證
參照僅限 vault 名稱。沒有一個通用、可插拔的讀取時間接機制 —— 也就是現代設定模式:儲存一個*指標*
(``env://TOKEN``、``file://./token``、``secret://api-key``)而非值本身。本功能補上這個解析器。

純標準函式庫(``os`` / ``re``);不匯入 ``PySide6``。env 讀取器、secret 解析器與基底目錄皆可注入,因此
解析安全且在 CI 中具決定性。

無頭 API
--------

.. code-block:: python

from je_auto_control import resolve_ref, resolve_refs_in, RefResolver

token = resolve_ref("env://API_TOKEN")
key = resolve_ref("file://./secrets/key.pem")

config = resolve_refs_in({
"token": "env://API_TOKEN",
"db": {"password": "secret://db-password"},
})

``resolve_ref`` 解析單一參照:``env://`` 讀取環境變數(來自可注入的 mapping,否則回退 ``os.environ``),
``file://`` 讀取檔案(可選的 ``base_dir`` realpath 防穿越保護),``secret://`` 委派給可注入的解析器或
governance 憑證 broker。``resolve_refs_in`` 走訪巢狀 dict/list 並就地解析每個參照,非參照值保持不變。
``is_ref`` 測試一個值,``RefResolver`` 把可注入後端打包以便重複使用。無法解析或未知 scheme 的參照會拋出
``SecretRefError``。

執行器命令
----------

``AC_resolve_ref`` 把單一 ``ref`` 解析成 ``{value}``;``AC_resolve_refs`` 解析 ``obj`` 內每個參照並回傳
``{resolved}``。兩者皆以 MCP 工具(``ac_resolve_ref`` / ``ac_resolve_refs``)以及 Script Builder 中
**Security** 分類下的命令提供。
1 change: 1 addition & 0 deletions docs/source/Zh/zh_index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,7 @@ AutoControl 所有功能的完整使用指南。
doc/new_features/v82_features_doc
doc/new_features/v83_features_doc
doc/new_features/v84_features_doc
doc/new_features/v85_features_doc
doc/ocr_backends/ocr_backends_doc
doc/observability/observability_doc
doc/operations_layer/operations_layer_doc
Expand Down
5 changes: 5 additions & 0 deletions je_auto_control/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -286,6 +286,10 @@
from je_auto_control.utils.layered_config import (
LayeredConfig, SourceTrace, deep_merge,
)
# URI-scheme secret/value reference resolver (env:// / file:// / secret://)
from je_auto_control.utils.secret_ref import (
RefResolver, SecretRefError, is_ref, resolve_ref, resolve_refs_in,
)
# Outbound CloudEvents emitter
from je_auto_control.utils.events import (
EventEmitter, post_cloudevent, to_cloudevent,
Expand Down Expand Up @@ -879,6 +883,7 @@ def start_autocontrol_gui(*args, **kwargs):
"Asset", "AssetStore", "AssetValue", "active_environment",
"dotenv_values", "dump_dotenv", "load_dotenv", "parse_dotenv",
"LayeredConfig", "SourceTrace", "deep_merge",
"RefResolver", "SecretRefError", "is_ref", "resolve_ref", "resolve_refs_in",
"EventEmitter", "post_cloudevent", "to_cloudevent",
"WebhookChannel", "WebhookResult", "notify_webhook", "set_default_poster",
"json_extract", "json_query", "json_query_one",
Expand Down
16 changes: 16 additions & 0 deletions je_auto_control/gui/script_builder/command_schema.py
Original file line number Diff line number Diff line change
Expand Up @@ -1640,6 +1640,22 @@ def _add_resilience_specs(specs: List[CommandSpec]) -> None:
),
description="Serialise items into a percent-encoded baggage header.",
))
specs.append(CommandSpec(
"AC_resolve_ref", "Security", "Secret Ref: Resolve",
fields=(
FieldSpec("ref", FieldType.STRING,
placeholder="env://TOKEN | file://./token | secret://api-key"),
),
description="Resolve an env:// / file:// / secret:// value reference.",
))
specs.append(CommandSpec(
"AC_resolve_refs", "Security", "Secret Ref: Resolve In Structure",
fields=(
FieldSpec("obj", FieldType.STRING,
placeholder='{"token": "env://TOKEN", "url": "..."}'),
),
description="Recursively resolve references inside a JSON structure.",
))
specs.append(CommandSpec(
"AC_profile_rows", "Data", "Data Profile: Profile Rows",
fields=(
Expand Down
17 changes: 17 additions & 0 deletions je_auto_control/utils/executor/action_executor.py
Original file line number Diff line number Diff line change
Expand Up @@ -3000,6 +3000,21 @@ def _trace_extract(headers: Any) -> Dict[str, Any]:
return {"context": ctx.to_dict() if ctx is not None else None}


def _resolve_ref(ref: str) -> Dict[str, Any]:
"""Adapter: resolve an env:// / file:// / secret:// reference."""
from je_auto_control.utils.secret_ref import resolve_ref
return {"value": resolve_ref(ref)}


def _resolve_refs(obj: Any) -> Dict[str, Any]:
"""Adapter: recursively resolve references in a structure (or JSON str)."""
import json
from je_auto_control.utils.secret_ref import resolve_refs_in
if isinstance(obj, str):
obj = json.loads(obj)
return {"resolved": resolve_refs_in(obj)}


def _baggage_parse(header: str) -> Dict[str, Any]:
"""Adapter: parse a W3C baggage header into {items}."""
from je_auto_control.utils.baggage import parse_baggage
Expand Down Expand Up @@ -4220,6 +4235,8 @@ def __init__(self):
"AC_trace_extract": _trace_extract,
"AC_baggage_parse": _baggage_parse,
"AC_baggage_format": _baggage_format,
"AC_resolve_ref": _resolve_ref,
"AC_resolve_refs": _resolve_refs,
"AC_profile_rows": _profile_rows,
"AC_infer_schema": _infer_schema,
"AC_parse_problem": _parse_problem,
Expand Down
23 changes: 22 additions & 1 deletion je_auto_control/utils/mcp_server/tools/_factories.py
Original file line number Diff line number Diff line change
Expand Up @@ -3509,6 +3509,27 @@ def data_profile_tools() -> List[MCPTool]:
]


def secret_ref_tools() -> List[MCPTool]:
return [
MCPTool(
name="ac_resolve_ref",
description=("Resolve a value reference 'ref' (env://VAR, "
"file://path, or secret://name) to {value}."),
input_schema=schema({"ref": {"type": "string"}}, ["ref"]),
handler=h.resolve_ref,
annotations=READ_ONLY,
),
MCPTool(
name="ac_resolve_refs",
description=("Recursively resolve every env:// / file:// / secret:// "
"reference inside 'obj'. Returns {resolved}."),
input_schema=schema({"obj": {"type": "object"}}, ["obj"]),
handler=h.resolve_refs,
annotations=READ_ONLY,
),
]


def baggage_tools() -> List[MCPTool]:
return [
MCPTool(
Expand Down Expand Up @@ -5091,7 +5112,7 @@ def media_assert_tools() -> List[MCPTool]:
search_index_tools, stats_tools, recurrence_tools, text_diff_tools,
feature_flag_tools, provenance_tools, json_contract_tools, chaos_tools,
slo_tools, percentiles_tools, bulkhead_tools, http_cassette_tools,
trace_context_tools, baggage_tools,
trace_context_tools, baggage_tools, secret_ref_tools,
data_profile_tools, http_problem_tools, dotenv_tools,
sse_client_tools, layered_config_tools, data_drift_tools,
dataset_diff_tools,
Expand Down
10 changes: 10 additions & 0 deletions je_auto_control/utils/mcp_server/tools/_handlers.py
Original file line number Diff line number Diff line change
Expand Up @@ -1736,6 +1736,16 @@ def baggage_format(items):
return _baggage_format(items)


def resolve_ref(ref):
from je_auto_control.utils.executor.action_executor import _resolve_ref
return _resolve_ref(ref)


def resolve_refs(obj):
from je_auto_control.utils.executor.action_executor import _resolve_refs
return _resolve_refs(obj)


def profile_rows(rows, columns=None):
from je_auto_control.utils.executor.action_executor import _profile_rows
return _profile_rows(rows, columns)
Expand Down
8 changes: 8 additions & 0 deletions je_auto_control/utils/secret_ref/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
"""URI-scheme value reference resolution for AutoControl."""
from je_auto_control.utils.secret_ref.secret_ref import (
RefResolver, SecretRefError, is_ref, resolve_ref, resolve_refs_in,
)

__all__ = [
"RefResolver", "SecretRefError", "is_ref", "resolve_ref", "resolve_refs_in",
]
Loading
Loading