Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- All built-in adapters and public fake runtimes now validate returned values
against `output_schema` locally, including textual JSON fallbacks.
- `AgentResult.is_success` provides one canonical success predicate.
- `TaskSupportReport`, `TaskSupportProvider`, and `validate_task()` provide
side-effect-free task preflight without adding a requirement to the
`AgentRuntime` protocol. Registry and `AgentKit` helpers preserve custom
runtime validation and capability-based fallback for older runtimes.
- `COMPATIBILITY_MANIFEST` records each built-in adapter's install extra,
import module, accepted SDK range, tested lockfile version, and runtime binary
versions such as `openai-codex-cli-bin`.

### Changed

Expand All @@ -26,6 +33,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
duplicate MCP names, and conflicting tool filters.
- BREAKING: unknown `Usage` values, including cost, are `None` rather than zero;
an explicit provider-reported zero remains `0`/`0.0`.
- Capability declarations now distinguish budget, reasoning-effort, network,
tool-filter, and per-MCP-server environment support. Built-in `run()` methods
consume the same support report used by preflight, eliminating divergent
rejection paths.

### Fixed

Expand All @@ -39,6 +50,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Permission- and budget-critical vendor options must now be explicit,
introspectable SDK parameters; opaque `**kwargs`, positional-only options,
and uninspectable callables fail closed.
- Configured model allow-lists, Antigravity MCP name syntax, disjoint
allow/deny lists, and Antigravity's legacy reasoning-effort alias are rejected
during static preflight instead of surfacing later or being silently ignored.

## 0.4.0 - 2026-07-02

Expand Down
8 changes: 8 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -141,6 +141,14 @@ a field (for example only Claude maps `budget_usd`; Codex and Antigravity reject
it with a typed `UnsupportedTaskInputError`) the adapter raises rather than
silently dropping it.

Call `validate_task(runtime, task)` (or `kit.validate_task("codex", task)`) to
inspect every statically detectable incompatibility before dispatch. The
returned `TaskSupportReport` is side-effect-free and lists source fields such as
`budget_usd` and `permissions.network`; `run()` still fails closed on the first
issue. Third-party runtimes can opt into provider-specific checks with the
`TaskSupportProvider` protocol, while older `AgentRuntime` implementations
continue to work through capability-based fallback checks.

`AgentResult` returns output, finish reason (see `FinishReason`), locally
validated structured output, usage, cost, session id, tool-call audits, and
provider metadata. `is_success` is true only for a natural, error-free
Expand Down
13 changes: 13 additions & 0 deletions docs/api-stability.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,12 @@ import the names from the top-level package instead.
a task field raises `UnsupportedTaskInputError`; the one exception is
vendor-option drift, which is recorded in
`AgentResult.metadata["dropped_options"]` instead.
- **Task support can be inspected without dispatch.** `validate_task(runtime,
task)`, `RuntimeRegistry.validate_task_for(...)`, and `AgentKit.validate_task(...)`
return every statically detectable incompatibility as a `TaskSupportReport`.
`TaskSupportProvider` is an optional extension, not a new requirement on the
`AgentRuntime` protocol, so existing third-party runtimes retain structural
compatibility and use capability-based fallback checks.
- **`AgentKit` is sugar, not a second API.** The hub assembles the same frozen
`AgentTask` and returns the same `AgentResult` the runtimes produce
(`ParsedResult` is a runtime-identical subclass adding only the typed
Expand Down Expand Up @@ -72,6 +78,13 @@ it reaches installed users; a release above a cap is by design invisible to that
lane until the cap is raised. A separate lane installs every direct dependency at
its declared floor so a stale minimum cannot sit undetected in the metadata.

`COMPATIBILITY_MANIFEST` is the machine-readable form of that policy. Each entry
records the install extra, import module, accepted SDK range, exact lockfile
version exercised by the repository, and any separately versioned runtime binary
(currently `openai-codex-cli-bin`). The manifest is tested against both
`pyproject.toml` and `uv.lock`; it is evidence of the committed test baseline,
not a claim that every version in the accepted range was exhaustively tested.

## Deprecation

When a public name is slated for removal it will be kept working for at least one
Expand Down
14 changes: 14 additions & 0 deletions docs/capability-matrix.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,13 @@ The matrix is intentionally not a lowest-common-denominator contract. Adapters
reject unsupported inputs (see below) when silently dropping them would be
misleading.

The public `AgentCapabilities` value also advertises the granular task controls
used by preflight: `budget`, `reasoning_effort`, `network_control`,
`tool_filters`, and `mcp_server_env`. Use `validate_task(runtime, task)` when a
concrete task is available; unlike a raw capability flag, its
`TaskSupportReport` preserves instance-specific rules such as a configured model
allow-list and reports all detectable issues at once.

For every provider, malformed schemas are rejected locally before dispatch and
returned structured values are validated locally after the SDK completes.
`parsed_output_available` distinguishes valid JSON `null` from no parsed value.
Expand Down Expand Up @@ -63,6 +70,13 @@ tool, uses the SDK's `disabled_tools` route).
Each adapter raises `UnsupportedTaskInputError` for task fields it has no SDK
surface to honor, rather than dropping them silently.

The same fields can be checked before dispatch through `validate_task`,
`RuntimeRegistry.validate_task_for`, or `AgentKit.validate_task`. Built-in
adapters additionally preflight configured model allow-lists. Antigravity also
checks its MCP server-name syntax and the SDK's prohibition on combining an
allow-list with a deny-list. SDK-version-dependent Antigravity tool vocabulary
is still validated at dispatch, after the installed SDK supplies its enum.

| Field | Claude | Codex | Antigravity |
|-------|--------|-------|-------------|
| `budget_usd` | Mapped (`max_budget_usd`, fails closed under SDK drift) | Rejected | Rejected |
Expand Down
10 changes: 8 additions & 2 deletions docs/providers.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,8 @@ All three adapters map the task's system prompt (Claude `system_prompt`, Codex
field (falling back to the `metadata` aliases of the same names).
`reasoning_effort` maps to the Claude and Codex `effort` options; Antigravity
has no reasoning-effort control and rejects the first-class field with a typed
error (its legacy `metadata["reasoning_effort"]` alias stays ignored, as it
always has been). Effort values are passed through to the vendor SDK rather
error. Its legacy `metadata["reasoning_effort"]` alias is rejected too, rather
than being silently ignored. Effort values are passed through to the vendor SDK rather
than validated by this library — each vendor defines its own accepted
vocabulary (for example `claude-agent-sdk` 0.2.x accepts
`low`/`medium`/`high`/`xhigh`/`max`), and an SDK too old to accept `effort` at
Expand All @@ -42,6 +42,12 @@ Usage fields are `None` when a provider omits its usage breakdown or cost.
Provider-reported zeros remain numeric zero, so callers can distinguish a free
or zero-token run from missing telemetry.

Every built-in adapter exposes a pure `validate_task()` extension. The public
`validate_task(runtime, task)`, registry, and `AgentKit` helpers call that richer
validator when present and otherwise fall back to declared capabilities for
older third-party runtimes. A report is a static preflight, not an availability
or credential probe; installed-SDK drift can still make dispatch fail closed.

Claude uses the `claude-agent-sdk` package and maps working directory,
permissions, filesystem access (a `READ_ONLY` filesystem forces `plan` mode),
MCP servers, sessions, structured output, tool allow/deny lists, runtime
Expand Down
1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,7 @@ dev = [
"pytest-asyncio>=0.23",
"pytest-cov>=5.0",
"ruff>=0.8",
"tomli>=2; python_version < '3.11'",
"types-jsonschema>=4.18",
]

Expand Down
18 changes: 18 additions & 0 deletions src/agent_runtime_kit/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,10 +27,19 @@
PermissionProfile,
RuntimeAvailability,
SessionResumeState,
TaskSupportIssue,
TaskSupportProvider,
TaskSupportReport,
ToolCallAudit,
Usage,
runtime_kind_value,
)
from agent_runtime_kit.compatibility import (
COMPATIBILITY_MANIFEST,
PackageVersion,
RuntimeCompatibility,
compatibility_for,
)
from agent_runtime_kit.events import (
output_delta_event,
safe_emit,
Expand All @@ -42,6 +51,7 @@
vendor_turn_event,
)
from agent_runtime_kit.registry import RuntimeRegistry, create_default_registry
from agent_runtime_kit.support import validate_task

__all__ = [
"AgentCapabilities",
Expand All @@ -59,20 +69,27 @@
"FilesystemAccess",
"FinishReason",
"KIND_ALIASES",
"COMPATIBILITY_MANIFEST",
"McpServerConfig",
"OutputSchemaError",
"OutputTypeError",
"PackageVersion",
"ParsedResult",
"PermissionMode",
"PermissionProfile",
"RuntimeAvailability",
"RuntimeCompatibility",
"RuntimeNotRegisteredError",
"RuntimeRegistry",
"SessionResumeState",
"TaskSupportIssue",
"TaskSupportProvider",
"TaskSupportReport",
"ToolCallAudit",
"UnsupportedTaskInputError",
"Usage",
"create_default_registry",
"compatibility_for",
"runtime_kind_value",
"output_delta_event",
"safe_emit",
Expand All @@ -82,4 +99,5 @@
"tool_completed_event",
"tool_requested_event",
"vendor_turn_event",
"validate_task",
]
17 changes: 17 additions & 0 deletions src/agent_runtime_kit/_kit.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,10 @@
PermissionProfile,
RuntimeAvailability,
SessionResumeState,
TaskSupportReport,
)
from agent_runtime_kit.registry import RuntimeFactory, RuntimeRegistry, create_default_registry
from agent_runtime_kit.support import validate_task as validate_runtime_task

_T = TypeVar("_T")
# An event handler receives one normalized event dict; sync or async.
Expand Down Expand Up @@ -109,6 +111,21 @@ def availability_for(self, kind: AgentRuntimeKind | str) -> RuntimeAvailability:
def capabilities_for(self, kind: AgentRuntimeKind | str) -> AgentCapabilities:
return self._registry.capabilities_for(self._normalize_kind(kind))

def validate_task(
self,
runtime: AgentRuntimeKind | str | AgentRuntime,
task: AgentTask,
) -> TaskSupportReport:
"""Report unsupported task fields without starting a run."""

if isinstance(runtime, str):
kind = self._normalize_kind(runtime)
cached = self._runtimes.get(kind)
if cached is None:
return self._registry.validate_task_for(kind, task)
runtime = cached
return validate_runtime_task(runtime, task)

def on(self, event: str = "*") -> Callable[[_HandlerT], _HandlerT]:
"""Register an event handler for tasks assembled by this hub.

Expand Down
38 changes: 9 additions & 29 deletions src/agent_runtime_kit/_runtime.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,14 +5,14 @@
from collections.abc import Mapping
from typing import Any

from agent_runtime_kit._errors import UnsupportedTaskInputError
from agent_runtime_kit._schema import resolve_structured_output
from agent_runtime_kit._types import (
AgentCapabilities,
AgentResult,
AgentRuntimeKind,
AgentTask,
RuntimeAvailability,
TaskSupportReport,
ToolCallAudit,
)
from agent_runtime_kit.events import (
Expand All @@ -25,6 +25,7 @@
tool_requested_event,
vendor_turn_event,
)
from agent_runtime_kit.support import _validate_declared_task_support, require_task_support


class FakeAgentRuntime:
Expand All @@ -47,6 +48,7 @@ def __init__(
streaming=False,
tool_audit=True,
cancellation=True,
mcp_server_env=True,
)
self._output = output
self._metadata = dict(metadata or {})
Expand All @@ -57,12 +59,17 @@ def availability(self) -> RuntimeAvailability:

return RuntimeAvailability.ok(self.kind, package="agent-runtime-kit")

def validate_task(self, task: AgentTask) -> TaskSupportReport:
"""Report unsupported fields without side effects."""

return _validate_declared_task_support(self.kind, self.capabilities, task)

async def run(self, task: AgentTask) -> AgentResult:
"""Return a deterministic result after validating capabilities."""

await safe_emit(task, task_started_event(task, self.kind))
try:
_ensure_supported(self.kind, self.capabilities, task)
require_task_support(self.validate_task(task))
output = self._output if self._output is not None else f"Fake result for: {task.goal}"
parsed = {"output": output} if task.output_schema is not None else None
parsed_available = parsed is not None
Expand Down Expand Up @@ -139,30 +146,3 @@ async def __aenter__(self) -> FakeAgentRuntime:

async def __aexit__(self, _exc_type: object, _exc: object, _tb: object) -> None:
await self.aclose()


def _ensure_supported(
kind: AgentRuntimeKind,
capabilities: AgentCapabilities,
task: AgentTask,
) -> None:
if task.mcp_servers and not capabilities.mcp_support:
raise UnsupportedTaskInputError(kind, "mcp_servers", "runtime does not support MCP")
if task.working_directory is not None and not capabilities.working_directory:
raise UnsupportedTaskInputError(
kind,
"working_directory",
"runtime does not support per-task working directories",
)
if (task.session_id or task.resume_from) and not capabilities.session_resume:
raise UnsupportedTaskInputError(
kind,
"session_id",
"runtime does not support session resume",
)
if task.output_schema is not None and not capabilities.structured_output:
raise UnsupportedTaskInputError(
kind,
"output_schema",
"runtime does not support structured output",
)
50 changes: 50 additions & 0 deletions src/agent_runtime_kit/_types.py
Original file line number Diff line number Diff line change
Expand Up @@ -226,6 +226,56 @@ class AgentCapabilities:
streaming: bool = False
tool_audit: bool = False
cancellation: bool = False
budget: bool = False
reasoning_effort: bool = False
network_control: bool = False
tool_filters: bool = False
mcp_server_env: bool = False


@dataclass(frozen=True)
class TaskSupportIssue:
"""One task field a runtime cannot honor faithfully."""

field: str
message: str

def __post_init__(self) -> None:
_require_nonblank(self.field, "TaskSupportIssue.field")
_require_nonblank(self.message, "TaskSupportIssue.message")


@dataclass(frozen=True)
class TaskSupportReport:
"""Pure compatibility result for one task/runtime pair."""

kind: AgentRuntimeKind | str
issues: tuple[TaskSupportIssue, ...] = ()

def __post_init__(self) -> None:
issues = _tuple_value(self.issues, "TaskSupportReport.issues")
if not all(isinstance(issue, TaskSupportIssue) for issue in issues):
raise ValueError(
"TaskSupportReport.issues must contain only TaskSupportIssue values"
)
object.__setattr__(self, "kind", AgentRuntimeKind.coerce(self.kind))
object.__setattr__(self, "issues", issues)

@property
def supported(self) -> bool:
return not self.issues


@runtime_checkable
class TaskSupportProvider(Protocol):
"""Optional extension for runtimes with provider-specific task checks.

``AgentRuntime`` deliberately does not require this protocol: third-party
runtimes written before task preflight was introduced remain compatible.
"""

def validate_task(self, task: AgentTask) -> TaskSupportReport:
"""Purely report whether this runtime can honor the task."""


@dataclass(frozen=True)
Expand Down
Loading