Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@

## Table of Contents

- [What's new (2026-06-21) — Service-Level Objectives (SLO)](#whats-new-2026-06-21--service-level-objectives-slo)
- [What's new (2026-06-21) — Chaos Experiments](#whats-new-2026-06-21--chaos-experiments)
- [What's new (2026-06-21) — JSON Contract & Snapshot Matching](#whats-new-2026-06-21--json-contract--snapshot-matching)
- [What's new (2026-06-21) — SLSA Build Provenance](#whats-new-2026-06-21--slsa-build-provenance)
Expand Down Expand Up @@ -124,6 +125,12 @@

---

## What's new (2026-06-21) — Service-Level Objectives (SLO)

SLI, error budget and burn-rate alerts. Full reference: [`docs/source/Eng/doc/new_features/v72_features_doc.rst`](docs/source/Eng/doc/new_features/v72_features_doc.rst).

- **`evaluate_slo` / `burn_rate` / `burn_alerts` / `default_burn_rules`** (`AC_evaluate_slo`, `AC_burn_alerts`): the framework emitted raw signals but had no SLO layer. This computes the SLI over outcome records (`[{timestamp, ok}]`), the error budget against a target, and the **multi-window multi-burn-rate** alerts from the Google SRE workbook (page 14.4×@1h, 6×@6h; ticket 1×@3d — firing only when both windows exceed the threshold). Records are plain data, clock injectable, fully deterministic. Pure-stdlib.

## What's new (2026-06-21) — Chaos Experiments

Inject faults, verify the system holds. Full reference: [`docs/source/Eng/doc/new_features/v71_features_doc.rst`](docs/source/Eng/doc/new_features/v71_features_doc.rst).
Expand Down
7 changes: 7 additions & 0 deletions README/README_zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@

## 目录

- [本次更新 (2026-06-21) — 服务等级目标(SLO)](#本次更新-2026-06-21--服务等级目标slo)
- [本次更新 (2026-06-21) — 混沌实验](#本次更新-2026-06-21--混沌实验)
- [本次更新 (2026-06-21) — JSON 合约与快照比对](#本次更新-2026-06-21--json-合约与快照比对)
- [本次更新 (2026-06-21) — SLSA 构建来源证明](#本次更新-2026-06-21--slsa-构建来源证明)
Expand Down Expand Up @@ -123,6 +124,12 @@

---

## 本次更新 (2026-06-21) — 服务等级目标(SLO)

SLI、错误预算与燃烧率告警。完整参考:[`docs/source/Zh/doc/new_features/v72_features_doc.rst`](../docs/source/Zh/doc/new_features/v72_features_doc.rst)。

- **`evaluate_slo` / `burn_rate` / `burn_alerts` / `default_burn_rules`**(`AC_evaluate_slo`、`AC_burn_alerts`):框架会发出原始信号却没有 SLO 层。本功能在结果记录(`[{timestamp, ok}]`)上计算 SLI、对目标计算错误预算,以及 Google SRE workbook 的**多窗口多燃烧率**告警(1h 达 14.4×、6h 达 6× 呼叫;3d 达 1× 开单 —— 只有当长短窗口双双超过阈值才触发)。记录为纯数据、时钟可注入、完全确定。纯标准库。

## 本次更新 (2026-06-21) — 混沌实验

注入故障、验证系统仍成立。完整参考:[`docs/source/Zh/doc/new_features/v71_features_doc.rst`](../docs/source/Zh/doc/new_features/v71_features_doc.rst)。
Expand Down
7 changes: 7 additions & 0 deletions README/README_zh-TW.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@

## 目錄

- [本次更新 (2026-06-21) — 服務等級目標(SLO)](#本次更新-2026-06-21--服務等級目標slo)
- [本次更新 (2026-06-21) — 混沌實驗](#本次更新-2026-06-21--混沌實驗)
- [本次更新 (2026-06-21) — JSON 合約與快照比對](#本次更新-2026-06-21--json-合約與快照比對)
- [本次更新 (2026-06-21) — SLSA 建置來源證明](#本次更新-2026-06-21--slsa-建置來源證明)
Expand Down Expand Up @@ -123,6 +124,12 @@

---

## 本次更新 (2026-06-21) — 服務等級目標(SLO)

SLI、錯誤預算與燃燒率警示。完整參考:[`docs/source/Zh/doc/new_features/v72_features_doc.rst`](../docs/source/Zh/doc/new_features/v72_features_doc.rst)。

- **`evaluate_slo` / `burn_rate` / `burn_alerts` / `default_burn_rules`**(`AC_evaluate_slo`、`AC_burn_alerts`):框架會發出原始訊號卻沒有 SLO 層。本功能在結果紀錄(`[{timestamp, ok}]`)上計算 SLI、對目標計算錯誤預算,以及 Google SRE workbook 的**多視窗多燃燒率**警示(1h 達 14.4×、6h 達 6× 呼叫;3d 達 1× 開票 —— 只有當長短視窗雙雙超過門檻才觸發)。紀錄為純資料、時鐘可注入、完全具決定性。純標準函式庫。

## 本次更新 (2026-06-21) — 混沌實驗

注入故障、驗證系統仍成立。完整參考:[`docs/source/Zh/doc/new_features/v71_features_doc.rst`](../docs/source/Zh/doc/new_features/v71_features_doc.rst)。
Expand Down
46 changes: 46 additions & 0 deletions docs/source/Eng/doc/new_features/v72_features_doc.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
Service-Level Objectives (SLO)
==============================

The framework emits raw signals (``observability`` metrics, ``run_history``
durations) but had no operational layer turning them into an SLO, an error
budget, or burn-rate alerts. This adds that: compute the SLI over a window of
outcome records, the error budget against a target, and the **multi-window
multi-burn-rate** alerts from the Google SRE workbook.

Records are plain data (``[{"timestamp": float, "ok": bool}, ...]``) so the
whole thing is offline and deterministic; the clock is injectable. Pure
standard library; imports no ``PySide6``.

Headless API
------------

.. code-block:: python

from je_auto_control import evaluate_slo, burn_alerts

report = evaluate_slo(records, target=0.99)
# {"sli": 0.995, "good": 995, "total": 1000, "target": 0.99,
# "budget_total": 10.0, "budget_remaining": 5.0,
# "budget_remaining_fraction": 0.5, "burn_rate": 0.5}

for alert in burn_alerts(records, target=0.99):
page_oncall(alert) # severity, threshold, long/short burn rates

``evaluate_slo`` computes the SLI (good / total), the error budget
(``(1 - target) * total`` events) and the burn rate (``bad_rate / (1 -
target)`` — 1.0 means spending budget exactly on pace, > 1 means too fast).
``burn_rate`` is the bare number over a window. ``burn_alerts`` evaluates the
canonical Google SRE tiers from :func:`default_burn_rules` — page at 14.4× over
1h (and 5m), page at 6× over 6h (and 30m), ticket at 1× over 3d (and 6h) — and
fires a tier only when **both** its long and short windows exceed the
threshold, which gives fast reset and few false positives. Supply your own
``rules`` (``BurnRule`` list) to customise.

Executor commands
-----------------

``AC_evaluate_slo`` takes ``records`` (a list or JSON string), a ``target`` and
optional ``window_s``, and returns the SLI/budget report. ``AC_burn_alerts``
returns ``{alerts, firing}``. Both are exposed as MCP tools
(``ac_evaluate_slo`` / ``ac_burn_alerts``) and as Script Builder commands under
**Report**.
1 change: 1 addition & 0 deletions docs/source/Eng/eng_index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,7 @@ Comprehensive guides for all AutoControl features.
doc/new_features/v69_features_doc
doc/new_features/v70_features_doc
doc/new_features/v71_features_doc
doc/new_features/v72_features_doc
doc/ocr_backends/ocr_backends_doc
doc/observability/observability_doc
doc/operations_layer/operations_layer_doc
Expand Down
37 changes: 37 additions & 0 deletions docs/source/Zh/doc/new_features/v72_features_doc.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
服務等級目標(SLO)
==================

框架會發出原始訊號(``observability`` 指標、``run_history`` 時長),但沒有把它們轉成 SLO、錯誤
預算或燃燒率警示的運維層。本功能補上:在一段視窗的結果紀錄上計算 SLI、對目標計算錯誤預算,以及
Google SRE workbook 的**多視窗多燃燒率**警示。

紀錄是純資料(``[{"timestamp": float, "ok": bool}, ...]``),因此整體離線且具決定性;時鐘可注入。
純標準函式庫;不匯入 ``PySide6``。

無頭 API
--------

.. code-block:: python

from je_auto_control import evaluate_slo, burn_alerts

report = evaluate_slo(records, target=0.99)
# {"sli": 0.995, "good": 995, "total": 1000, "target": 0.99,
# "budget_total": 10.0, "budget_remaining": 5.0,
# "budget_remaining_fraction": 0.5, "burn_rate": 0.5}

for alert in burn_alerts(records, target=0.99):
page_oncall(alert) # severity、threshold、long/short 燃燒率

``evaluate_slo`` 計算 SLI(good / total)、錯誤預算(``(1 - target) * total`` 個事件)與燃燒率
(``bad_rate / (1 - target)`` —— 1.0 代表剛好按進度消耗預算,> 1 代表太快)。``burn_rate`` 是某
視窗的純數字。``burn_alerts`` 評估 :func:`default_burn_rules` 的標準 Google SRE 分層 —— 1h(與
5m)達 14.4× 呼叫、6h(與 30m)達 6× 呼叫、3d(與 6h)達 1× 開票 —— 且只有當某層的長視窗與短視窗
**雙雙**超過門檻時才觸發,以取得快速重置與少量誤報。可傳入自訂的 ``rules``(``BurnRule`` 清單)。

執行器命令
----------

``AC_evaluate_slo`` 接受 ``records``(清單或 JSON 字串)、``target`` 與選用的 ``window_s``,回傳
SLI/預算報告。``AC_burn_alerts`` 回傳 ``{alerts, firing}``。兩者皆以 MCP 工具(``ac_evaluate_slo``
/ ``ac_burn_alerts``)以及 Script Builder 中 **Report** 分類下的命令提供。
1 change: 1 addition & 0 deletions docs/source/Zh/zh_index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,7 @@ AutoControl 所有功能的完整使用指南。
doc/new_features/v69_features_doc
doc/new_features/v70_features_doc
doc/new_features/v71_features_doc
doc/new_features/v72_features_doc
doc/ocr_backends/ocr_backends_doc
doc/observability/observability_doc
doc/operations_layer/operations_layer_doc
Expand Down
5 changes: 5 additions & 0 deletions je_auto_control/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -365,6 +365,10 @@
ChaosExperiment, Fault, Probe, exception_fault, latency_fault,
run_experiment,
)
# SLO: SLI, error budget and multi-window burn-rate alerts
from je_auto_control.utils.slo import (
BurnRule, burn_alerts, burn_rate, default_burn_rules, evaluate_slo,
)
# Background popup/interrupt watchdog (unattended automation)
from je_auto_control.utils.watchdog import (
PopupWatchdog, WatchdogRule, default_popup_watchdog,
Expand Down Expand Up @@ -860,6 +864,7 @@ def start_autocontrol_gui(*args, **kwargs):
"MatchReport", "diff_json", "match_json", "normalize_json", "snapshot_json",
"ChaosExperiment", "Fault", "Probe", "exception_fault", "latency_fault",
"run_experiment",
"BurnRule", "burn_alerts", "burn_rate", "default_burn_rules", "evaluate_slo",
# MCP server
"AuditLogger", "HttpMCPServer", "MCPContent", "MCPPrompt",
"MCPPromptArgument", "MCPResource", "MCPServer", "MCPTool",
Expand Down
19 changes: 19 additions & 0 deletions je_auto_control/gui/script_builder/command_schema.py
Original file line number Diff line number Diff line change
Expand Up @@ -1388,6 +1388,25 @@ def _add_misc_specs(specs: List[CommandSpec]) -> None:
),
description="Verify a JWT (alg allowlist + exp/nbf/aud); returns {ok, claims}.",
))
specs.append(CommandSpec(
"AC_evaluate_slo", "Report", "SLO: Evaluate (SLI + Error Budget)",
fields=(
FieldSpec("records", FieldType.STRING,
placeholder='[{"timestamp": 1700000000, "ok": true}]'),
FieldSpec("target", FieldType.FLOAT, placeholder="0.99"),
FieldSpec("window_s", FieldType.FLOAT, optional=True),
),
description="SLI + error budget for outcome records vs a target.",
))
specs.append(CommandSpec(
"AC_burn_alerts", "Report", "SLO: Burn-Rate Alerts",
fields=(
FieldSpec("records", FieldType.STRING,
placeholder='[{"timestamp": 1700000000, "ok": false}]'),
FieldSpec("target", FieldType.FLOAT, placeholder="0.99"),
),
description="Multi-window burn-rate alerts (Google SRE tiers).",
))
specs.append(CommandSpec(
"AC_run_chaos", "Flow", "Run Chaos Experiment",
fields=(
Expand Down
22 changes: 22 additions & 0 deletions je_auto_control/utils/executor/action_executor.py
Original file line number Diff line number Diff line change
Expand Up @@ -2928,6 +2928,26 @@ def _rate_limit(name: str, rate: float = 1.0, capacity: float = 1.0,
"wait": round(bucket.time_until_available(float(n)), 4)}


def _evaluate_slo(records: Any, target: float,
window_s: Optional[float] = None) -> Dict[str, Any]:
"""Adapter: SLI + error budget for outcome records (list or JSON string)."""
import json
from je_auto_control.utils.slo import evaluate_slo
if isinstance(records, str):
records = json.loads(records)
return evaluate_slo(records, float(target), window_s=window_s)


def _burn_alerts(records: Any, target: float) -> Dict[str, Any]:
"""Adapter: multi-window burn-rate alerts for outcome records."""
import json
from je_auto_control.utils.slo import burn_alerts
if isinstance(records, str):
records = json.loads(records)
alerts = burn_alerts(records, float(target))
return {"alerts": alerts, "firing": bool(alerts)}


def _chaos_probe_call(actions: List[Any]) -> Any:
def call() -> bool:
executor.execute_action(list(actions), raise_on_error=True)
Expand Down Expand Up @@ -3965,6 +3985,8 @@ def __init__(self):
"AC_match_json": _match_json,
"AC_diff_json": _diff_json,
"AC_run_chaos": _run_chaos,
"AC_evaluate_slo": _evaluate_slo,
"AC_burn_alerts": _burn_alerts,
"AC_unified_diff": _unified_diff,
"AC_apply_unified": _apply_unified,
"AC_three_way_merge": _three_way_merge,
Expand Down
29 changes: 29 additions & 0 deletions je_auto_control/utils/mcp_server/tools/_factories.py
Original file line number Diff line number Diff line change
Expand Up @@ -3343,6 +3343,34 @@ def rate_limit_tools() -> List[MCPTool]:
]


def slo_tools() -> List[MCPTool]:
return [
MCPTool(
name="ac_evaluate_slo",
description=("Compute the SLI and error budget for outcome "
"'records' [{timestamp, ok}] against 'target' "
"(0-1). Returns {sli, budget_remaining, burn_rate, ...}."),
input_schema=schema(
{"records": {"type": "array"}, "target": {"type": "number"},
"window_s": {"type": "number"}},
["records", "target"]),
handler=h.evaluate_slo,
annotations=READ_ONLY,
),
MCPTool(
name="ac_burn_alerts",
description=("Multi-window burn-rate alerts (Google SRE tiers) for "
"outcome 'records' against 'target'. Returns "
"{alerts, firing}."),
input_schema=schema(
{"records": {"type": "array"}, "target": {"type": "number"}},
["records", "target"]),
handler=h.burn_alerts,
annotations=READ_ONLY,
),
]


def chaos_tools() -> List[MCPTool]:
return [
MCPTool(
Expand Down Expand Up @@ -4789,6 +4817,7 @@ def media_assert_tools() -> List[MCPTool]:
license_policy_tools, jwt_tools, rate_limit_tools, json_patch_tools,
search_index_tools, stats_tools, recurrence_tools, text_diff_tools,
feature_flag_tools, provenance_tools, json_contract_tools, chaos_tools,
slo_tools,
saga_tools, decision_table_tools, locator_repair_tools,
pii_text_tools, sarif_tools,
screen_record_tools,
Expand Down
11 changes: 11 additions & 0 deletions je_auto_control/utils/mcp_server/tools/_handlers.py
Original file line number Diff line number Diff line change
Expand Up @@ -1680,6 +1680,17 @@ def run_chaos(spec):
return _run_chaos(spec)


def evaluate_slo(records, target, window_s=None):
from je_auto_control.utils.slo import evaluate_slo as _slo
return _slo(records, float(target), window_s=window_s)


def burn_alerts(records, target):
from je_auto_control.utils.slo import burn_alerts as _alerts
alerts = _alerts(records, float(target))
return {"alerts": alerts, "firing": bool(alerts)}


def build_provenance(paths, builder_id="je_auto_control"):
from je_auto_control.utils.provenance import build_provenance, subject_for
subjects = [subject_for(path) for path in paths]
Expand Down
9 changes: 9 additions & 0 deletions je_auto_control/utils/slo/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
"""SLO evaluation: SLI, error budget and multi-window burn-rate alerts."""
from je_auto_control.utils.slo.slo import (
BurnRule, burn_alerts, burn_rate, default_burn_rules, evaluate_slo,
)

__all__ = [
"BurnRule", "burn_alerts", "burn_rate", "default_burn_rules",
"evaluate_slo",
]
Loading
Loading