Merge pull request #287 from Integration-Automation/feat/dotenv-batch

JE-Chen · web-flow · commit 7b86ac0c09be · 2026-06-21T22:31:20.000+08:00
Add .env (dotenv) parsing and serialisation
diff --git a/README.md b/README.md
@@ -13,6 +13,7 @@
 
 ## Table of Contents
 
+- [What's new (2026-06-21) — Dotenv (.env) Parsing](#whats-new-2026-06-21--dotenv-env-parsing)
 - [What's new (2026-06-21) — RFC 9457 Problem Details Parsing](#whats-new-2026-06-21--rfc-9457-problem-details-parsing)
 - [What's new (2026-06-21) — Data Profiling & Schema Inference](#whats-new-2026-06-21--data-profiling--schema-inference)
 - [What's new (2026-06-21) — W3C Trace Context Propagation](#whats-new-2026-06-21--w3c-trace-context-propagation)
@@ -131,6 +132,12 @@
 
 ---
 
+## What's new (2026-06-21) — Dotenv (.env) Parsing
+
+Read 12-factor `.env` files into config. Full reference: [`docs/source/Eng/doc/new_features/v79_features_doc.rst`](docs/source/Eng/doc/new_features/v79_features_doc.rst).
+
+- **`parse_dotenv` / `load_dotenv` / `dotenv_values` / `dump_dotenv`** (`AC_parse_dotenv`, `AC_load_dotenv`): `load_vars_from_json` ingested flat JSON but nothing read the de-facto `.env` file. This parses `KEY=VALUE` lines (`export` prefixes, single/double quoting, `\n`/`\t` escapes, inline comments) into a plain dict — no `python-dotenv` dependency. The loader merges into a caller-supplied mapping rather than mutating `os.environ`, so it stays safe and deterministic. Pure-stdlib.
+
 ## What's new (2026-06-21) — RFC 9457 Problem Details Parsing
 
 Read standardized API errors out of HTTP responses. Full reference: [`docs/source/Eng/doc/new_features/v78_features_doc.rst`](docs/source/Eng/doc/new_features/v78_features_doc.rst).
diff --git a/README/README_zh-CN.md b/README/README_zh-CN.md
@@ -12,6 +12,7 @@
 
 ## 目录
 
+- [本次更新 (2026-06-21) — Dotenv (.env) 解析](#本次更新-2026-06-21--dotenv-env-解析)
 - [本次更新 (2026-06-21) — RFC 9457 Problem Details 解析](#本次更新-2026-06-21--rfc-9457-problem-details-解析)
 - [本次更新 (2026-06-21) — 数据剖析与结构推断](#本次更新-2026-06-21--数据剖析与结构推断)
 - [本次更新 (2026-06-21) — W3C Trace Context 传播](#本次更新-2026-06-21--w3c-trace-context-传播)
@@ -130,6 +131,12 @@
 
 ---
 
+## 本次更新 (2026-06-21) — Dotenv (.env) 解析
+
+把 12-factor `.env` 文件读进配置。完整参考:[`docs/source/Zh/doc/new_features/v79_features_doc.rst`](../docs/source/Zh/doc/new_features/v79_features_doc.rst)。
+
+- **`parse_dotenv` / `load_dotenv` / `dotenv_values` / `dump_dotenv`**(`AC_parse_dotenv`、`AC_load_dotenv`):`load_vars_from_json` 载入扁平 JSON,但没有任何东西读取 de-facto 的 `.env` 文件。本功能把 `KEY=VALUE` 行(`export` 前缀、单/双引号、`\n`/`\t` 转义、行内注释)解析成纯 dict —— 不依赖 `python-dotenv`。载入器合并进调用端提供的 mapping 而非变动 `os.environ`,因此安全且确定。纯标准库。
+
 ## 本次更新 (2026-06-21) — RFC 9457 Problem Details 解析
 
 从 HTTP 响应读取标准化的 API 错误。完整参考:[`docs/source/Zh/doc/new_features/v78_features_doc.rst`](../docs/source/Zh/doc/new_features/v78_features_doc.rst)。
diff --git a/README/README_zh-TW.md b/README/README_zh-TW.md
@@ -12,6 +12,7 @@
 
 ## 目錄
 
+- [本次更新 (2026-06-21) — Dotenv (.env) 解析](#本次更新-2026-06-21--dotenv-env-解析)
 - [本次更新 (2026-06-21) — RFC 9457 Problem Details 解析](#本次更新-2026-06-21--rfc-9457-problem-details-解析)
 - [本次更新 (2026-06-21) — 資料剖析與結構推斷](#本次更新-2026-06-21--資料剖析與結構推斷)
 - [本次更新 (2026-06-21) — W3C Trace Context 傳播](#本次更新-2026-06-21--w3c-trace-context-傳播)
@@ -130,6 +131,12 @@
 
 ---
 
+## 本次更新 (2026-06-21) — Dotenv (.env) 解析
+
+把 12-factor `.env` 檔案讀進設定。完整參考:[`docs/source/Zh/doc/new_features/v79_features_doc.rst`](../docs/source/Zh/doc/new_features/v79_features_doc.rst)。
+
+- **`parse_dotenv` / `load_dotenv` / `dotenv_values` / `dump_dotenv`**(`AC_parse_dotenv`、`AC_load_dotenv`):`load_vars_from_json` 載入扁平 JSON,但沒有任何東西讀取 de-facto 的 `.env` 檔案。本功能把 `KEY=VALUE` 行(`export` 前綴、單/雙引號、`\n`/`\t` 轉義、行內註解)解析成純 dict —— 不依賴 `python-dotenv`。載入器合併進呼叫端提供的 mapping 而非變動 `os.environ`,因此安全且具決定性。純標準函式庫。
+
 ## 本次更新 (2026-06-21) — RFC 9457 Problem Details 解析
 
 從 HTTP 回應讀取標準化的 API 錯誤。完整參考:[`docs/source/Zh/doc/new_features/v78_features_doc.rst`](../docs/source/Zh/doc/new_features/v78_features_doc.rst)。
diff --git a/docs/source/Eng/doc/new_features/v79_features_doc.rst b/docs/source/Eng/doc/new_features/v79_features_doc.rst
@@ -0,0 +1,42 @@
+Dotenv (.env) Parsing
+=====================
+
+``script_vars.load_vars_from_json`` ingests flat JSON, but nothing read the
+de-facto 12-factor ``.env`` file. This parses ``KEY=VALUE`` lines — honouring
+``export`` prefixes, single/double quoting, escapes, and inline comments — into
+a plain dict that can feed a config layer, with no ``python-dotenv`` dependency.
+
+Pure standard library (``re``); imports no ``PySide6``. ``parse_dotenv`` is a
+pure string-to-dict function, and the loader merges into a caller-supplied
+mapping rather than mutating ``os.environ``, so it is safe and deterministic.
+
+Headless API
+------------
+
+.. code-block:: python
+
+    from je_auto_control import parse_dotenv, load_dotenv, dotenv_values, dump_dotenv
+
+    values = parse_dotenv('PLAIN=hello\nexport TOKEN="a\\nb"  # comment')
+    # {"PLAIN": "hello", "TOKEN": "a\nb"}
+
+    config = {}
+    load_dotenv(".env", config)                 # merge file into a dict
+    load_dotenv(".env.local", config, override=True)
+
+``parse_dotenv`` skips blanks and ``#`` comment lines, strips an optional
+``export`` prefix, validates keys, and resolves values: single-quoted values
+are literal, double-quoted values process ``\n`` / ``\t`` / ``\\`` / ``\"``
+escapes, and unquoted values drop a trailing `` #`` comment and surrounding
+whitespace. ``dotenv_values`` reads and parses a file; ``load_dotenv`` merges a
+file into an explicit ``env`` mapping (keeping existing keys unless
+``override``); ``dump_dotenv`` serialises a mapping back to ``.env`` text,
+quoting values that need it.
+
+Executor commands
+-----------------
+
+``AC_parse_dotenv`` parses ``text`` into ``{values}``; ``AC_load_dotenv`` reads
+a file at ``path`` into a fresh ``{values}`` dict. Both are exposed as MCP tools
+(``ac_parse_dotenv`` / ``ac_load_dotenv``) and as Script Builder commands under
+**Data**.
diff --git a/docs/source/Eng/eng_index.rst b/docs/source/Eng/eng_index.rst
@@ -101,6 +101,7 @@ Comprehensive guides for all AutoControl features.
    doc/new_features/v76_features_doc
    doc/new_features/v77_features_doc
    doc/new_features/v78_features_doc
+   doc/new_features/v79_features_doc
    doc/ocr_backends/ocr_backends_doc
    doc/observability/observability_doc
    doc/operations_layer/operations_layer_doc
diff --git a/docs/source/Zh/doc/new_features/v79_features_doc.rst b/docs/source/Zh/doc/new_features/v79_features_doc.rst
@@ -0,0 +1,36 @@
+Dotenv(.env)解析
+================
+
+``script_vars.load_vars_from_json`` 可載入扁平 JSON,但沒有任何東西讀取 de-facto 12-factor 的
+``.env`` 檔案。本功能把 ``KEY=VALUE`` 行 —— 遵循 ``export`` 前綴、單/雙引號、轉義與行內註解 ——
+解析成可餵給設定層的純 dict,且不依賴 ``python-dotenv``。
+
+純標準函式庫(``re``);不匯入 ``PySide6``。``parse_dotenv`` 為純字串轉 dict 函式,載入器會合併進
+呼叫端提供的 mapping 而非變動 ``os.environ``,因此安全且具決定性。
+
+無頭 API
+--------
+
+.. code-block:: python
+
+    from je_auto_control import parse_dotenv, load_dotenv, dotenv_values, dump_dotenv
+
+    values = parse_dotenv('PLAIN=hello\nexport TOKEN="a\\nb"  # comment')
+    # {"PLAIN": "hello", "TOKEN": "a\nb"}
+
+    config = {}
+    load_dotenv(".env", config)                 # 把檔案合併進 dict
+    load_dotenv(".env.local", config, override=True)
+
+``parse_dotenv`` 略過空白與 ``#`` 註解行,去除選用的 ``export`` 前綴,驗證鍵,並解析值:單引號值為
+字面值,雙引號值處理 ``\n`` / ``\t`` / ``\\`` / ``\"`` 轉義,未加引號的值會去除結尾 `` #`` 註解與
+前後空白。``dotenv_values`` 讀取並解析檔案;``load_dotenv`` 把檔案合併進明確的 ``env`` mapping
+(預設保留既有鍵,除非 ``override``);``dump_dotenv`` 把 mapping 序列化回 ``.env`` 文字,並為需要的值
+加上引號。
+
+執行器命令
+----------
+
+``AC_parse_dotenv`` 把 ``text`` 解析成 ``{values}``;``AC_load_dotenv`` 從 ``path`` 讀檔載入到新的
+``{values}`` dict。兩者皆以 MCP 工具(``ac_parse_dotenv`` / ``ac_load_dotenv``)以及 Script Builder
+中 **Data** 分類下的命令提供。
diff --git a/docs/source/Zh/zh_index.rst b/docs/source/Zh/zh_index.rst
@@ -101,6 +101,7 @@ AutoControl 所有功能的完整使用指南。
    doc/new_features/v76_features_doc
    doc/new_features/v77_features_doc
    doc/new_features/v78_features_doc
+   doc/new_features/v79_features_doc
    doc/ocr_backends/ocr_backends_doc
    doc/observability/observability_doc
    doc/operations_layer/operations_layer_doc
diff --git a/je_auto_control/__init__.py b/je_auto_control/__init__.py
@@ -270,6 +270,10 @@
 from je_auto_control.utils.assets import (
     Asset, AssetStore, AssetValue, active_environment,
 )
+# .env file parsing / serialisation (12-factor config ingestion)
+from je_auto_control.utils.dotenv import (
+    dotenv_values, dump_dotenv, load_dotenv, parse_dotenv,
+)
 # Outbound CloudEvents emitter
 from je_auto_control.utils.events import (
     EventEmitter, post_cloudevent, to_cloudevent,
@@ -851,6 +855,7 @@ def start_autocontrol_gui(*args, **kwargs):
     "find_repeated_sequences", "mine_action_log",
     "rank_automation_candidates",
     "Asset", "AssetStore", "AssetValue", "active_environment",
+    "dotenv_values", "dump_dotenv", "load_dotenv", "parse_dotenv",
     "EventEmitter", "post_cloudevent", "to_cloudevent",
     "WebhookChannel", "WebhookResult", "notify_webhook", "set_default_poster",
     "json_extract", "json_query", "json_query_one",
diff --git a/je_auto_control/gui/script_builder/command_schema.py b/je_auto_control/gui/script_builder/command_schema.py
@@ -1654,6 +1654,22 @@ def _add_resilience_specs(specs: List[CommandSpec]) -> None:
         ),
         description="Parse an application/problem+json error response.",
     ))
+    specs.append(CommandSpec(
+        "AC_parse_dotenv", "Data", "Dotenv: Parse Text",
+        fields=(
+            FieldSpec("text", FieldType.STRING,
+                      placeholder='KEY=value\nexport TOKEN="abc"  # comment'),
+        ),
+        description="Parse .env text (KEY=VALUE, quotes, escapes) into values.",
+    ))
+    specs.append(CommandSpec(
+        "AC_load_dotenv", "Data", "Dotenv: Load File",
+        fields=(
+            FieldSpec("path", FieldType.STRING, placeholder=".env"),
+            FieldSpec("override", FieldType.BOOL, optional=True, default=False),
+        ),
+        description="Load a .env file into a values dict.",
+    ))
     specs.append(CommandSpec(
         "AC_rate_limit", "Flow", "Rate Limit (Token Bucket)",
         fields=(
diff --git a/je_auto_control/utils/dotenv/__init__.py b/je_auto_control/utils/dotenv/__init__.py
@@ -0,0 +1,6 @@
+""".env file parsing and serialisation for AutoControl configuration."""
+from je_auto_control.utils.dotenv.dotenv import (
+    dotenv_values, dump_dotenv, load_dotenv, parse_dotenv,
+)
+
+__all__ = ["dotenv_values", "dump_dotenv", "load_dotenv", "parse_dotenv"]
diff --git a/je_auto_control/utils/dotenv/dotenv.py b/je_auto_control/utils/dotenv/dotenv.py
@@ -0,0 +1,95 @@
+""".env file parsing and serialisation (no ``python-dotenv`` dependency).
+
+``script_vars.load_vars_from_json`` ingests flat JSON, but nothing reads the
+de-facto 12-factor ``.env`` file. This parses ``KEY=VALUE`` lines — honouring
+``export`` prefixes, single/double quoting, escapes, and inline comments — into
+a plain dict that can feed a config layer.
+
+Pure standard library (``re``); imports no ``PySide6``. ``parse_dotenv`` is a
+pure string-to-dict function; the loader merges into a caller-supplied mapping
+rather than mutating ``os.environ``, so it is safe and deterministic in CI.
+"""
+import re
+from pathlib import Path
+from typing import Dict, MutableMapping, Optional, Tuple
+
+_KEY_RE = re.compile(r"^[A-Za-z_][A-Za-z0-9_.]*$")
+_ESCAPES = {"n": "\n", "t": "\t", "r": "\r", '"': '"', "\\": "\\", "'": "'"}
+
+
+def _unescape(value: str) -> str:
+    out = []
+    chars = iter(value)
+    for char in chars:
+        if char == "\\":
+            nxt = next(chars, "\\")
+            out.append(_ESCAPES.get(nxt, "\\" + nxt))
+        else:
+            out.append(char)
+    return "".join(out)
+
+
+def _strip_inline_comment(value: str) -> str:
+    marker = value.find(" #")
+    return value[:marker] if marker != -1 else value
+
+
+def _unquote(raw: str) -> str:
+    value = raw.strip()
+    if len(value) >= 2 and value[0] == value[-1] and value[0] in ("'", '"'):
+        inner = value[1:-1]
+        return inner if value[0] == "'" else _unescape(inner)
+    return _strip_inline_comment(value).strip()
+
+
+def _parse_line(line: str) -> Optional[Tuple[str, str]]:
+    stripped = line.strip()
+    if not stripped or stripped.startswith("#"):
+        return None
+    if stripped.startswith("export "):
+        stripped = stripped[len("export "):].lstrip()
+    key, sep, raw = stripped.partition("=")
+    key = key.strip()
+    if not sep or not _KEY_RE.match(key):
+        return None
+    return key, _unquote(raw)
+
+
+def parse_dotenv(text: str) -> Dict[str, str]:
+    """Parse ``.env`` ``text`` into an ordered ``{key: value}`` dict."""
+    result: Dict[str, str] = {}
+    for line in (text or "").splitlines():
+        item = _parse_line(line)
+        if item is not None:
+            result[item[0]] = item[1]
+    return result
+
+
+def dotenv_values(path: str) -> Dict[str, str]:
+    """Read and parse a ``.env`` file at ``path``."""
+    return parse_dotenv(Path(path).read_text(encoding="utf-8"))
+
+
+def load_dotenv(path: str, env: MutableMapping[str, str], *,
+                override: bool = False) -> MutableMapping[str, str]:
+    """Merge a ``.env`` file into ``env`` and return it.
+
+    Existing keys are kept unless ``override`` is set. ``env`` is supplied by
+    the caller (never ``os.environ`` implicitly), keeping the load explicit.
+    """
+    for key, value in dotenv_values(path).items():
+        if override or key not in env:
+            env[key] = value
+    return env
+
+
+def dump_dotenv(mapping: MutableMapping[str, str]) -> str:
+    """Serialise ``mapping`` to ``.env`` text, quoting values when needed."""
+    lines = []
+    for key, value in mapping.items():
+        text = str(value)
+        if text != text.strip() or any(ch in text for ch in ('#', '\n', '"')):
+            text = '"' + text.replace("\\", "\\\\").replace('"', '\\"').replace(
+                "\n", "\\n") + '"'
+        lines.append(f"{key}={text}")
+    return "\n".join(lines)
diff --git a/je_auto_control/utils/executor/action_executor.py b/je_auto_control/utils/executor/action_executor.py
@@ -3032,6 +3032,18 @@ def _parse_problem(response: Any) -> Dict[str, Any]:
     return {"problem": problem.to_dict() if problem is not None else None}
 
 
+def _parse_dotenv(text: str) -> Dict[str, Any]:
+    """Adapter: parse .env text into a {values} dict."""
+    from je_auto_control.utils.dotenv import parse_dotenv
+    return {"values": parse_dotenv(text)}
+
+
+def _load_dotenv(path: str, override: Any = False) -> Dict[str, Any]:
+    """Adapter: load a .env file into a fresh {values} dict."""
+    from je_auto_control.utils.dotenv import load_dotenv
+    return {"values": load_dotenv(path, {}, override=bool(override))}
+
+
 def _percentiles(samples: Any, qs: Any = None) -> Dict[str, Any]:
     """Adapter: exact percentiles of a numeric sample list (or JSON string)."""
     import json
@@ -4113,6 +4125,8 @@ def __init__(self):
             "AC_profile_rows": _profile_rows,
             "AC_infer_schema": _infer_schema,
             "AC_parse_problem": _parse_problem,
+            "AC_parse_dotenv": _parse_dotenv,
+            "AC_load_dotenv": _load_dotenv,
             "AC_unified_diff": _unified_diff,
             "AC_apply_unified": _apply_unified,
             "AC_three_way_merge": _three_way_merge,
diff --git a/je_auto_control/utils/mcp_server/tools/_factories.py b/je_auto_control/utils/mcp_server/tools/_factories.py
@@ -3343,6 +3343,29 @@ def rate_limit_tools() -> List[MCPTool]:
     ]
 
 
+def dotenv_tools() -> List[MCPTool]:
+    return [
+        MCPTool(
+            name="ac_parse_dotenv",
+            description=("Parse .env 'text' (KEY=VALUE lines, export prefixes, "
+                         "quoting, escapes, inline comments) into {values}."),
+            input_schema=schema({"text": {"type": "string"}}, ["text"]),
+            handler=h.parse_dotenv,
+            annotations=READ_ONLY,
+        ),
+        MCPTool(
+            name="ac_load_dotenv",
+            description=("Load a .env file at 'path' into a fresh {values} dict. "
+                         "'override' is accepted for symmetry (fresh dict)."),
+            input_schema=schema(
+                {"path": {"type": "string"}, "override": {"type": "boolean"}},
+                ["path"]),
+            handler=h.load_dotenv,
+            annotations=READ_ONLY,
+        ),
+    ]
+
+
 def http_problem_tools() -> List[MCPTool]:
     return [
         MCPTool(
@@ -4947,7 +4970,7 @@ def media_assert_tools() -> List[MCPTool]:
     search_index_tools, stats_tools, recurrence_tools, text_diff_tools,
     feature_flag_tools, provenance_tools, json_contract_tools, chaos_tools,
     slo_tools, percentiles_tools, bulkhead_tools, http_cassette_tools,
-    trace_context_tools, data_profile_tools, http_problem_tools,
+    trace_context_tools, data_profile_tools, http_problem_tools, dotenv_tools,
     saga_tools, decision_table_tools, locator_repair_tools,
     pii_text_tools, sarif_tools,
     screen_record_tools,
diff --git a/je_auto_control/utils/mcp_server/tools/_handlers.py b/je_auto_control/utils/mcp_server/tools/_handlers.py
@@ -1741,6 +1741,16 @@ def parse_problem(response):
     return _parse_problem(response)
 
 
+def parse_dotenv(text):
+    from je_auto_control.utils.executor.action_executor import _parse_dotenv
+    return _parse_dotenv(text)
+
+
+def load_dotenv(path, override=False):
+    from je_auto_control.utils.executor.action_executor import _load_dotenv
+    return _load_dotenv(path, override)
+
+
 def build_provenance(paths, builder_id="je_auto_control"):
     from je_auto_control.utils.provenance import build_provenance, subject_for
     subjects = [subject_for(path) for path in paths]
diff --git a/test/unit_test/headless/test_dotenv_batch.py b/test/unit_test/headless/test_dotenv_batch.py
