Skip to content

Dry-run invocation mode #43

@dgenio

Description

@dgenio

Milestone: v0.2.0 | Tier: Medium | Effort: Small–Medium

Problem

Developers cannot test policy + firewall behavior without actually executing driver operations. There is no sandbox/preview mode. This makes it difficult to:

  • Validate policy configurations before deployment
  • Debug "why was my invocation denied/shaped this way?" issues
  • Run CI checks on policy changes without live tool backends
  • Understand what the kernel would do for a given request

Proposed Change

1. Dry-run parameter on invoke()

async def invoke(
    self, token: str, params: dict, *,
    dry_run: bool = False,
) -> Frame | DryRunResult:

When dry_run=True, the kernel runs the full pipeline but short-circuits before driver execution:

  1. ✅ Token verification
  2. ✅ Policy re-evaluation
  3. ✅ Route resolution (which driver, what operation)
  4. ✅ Firewall configuration (response mode, budget)
  5. ❌ Driver execution (skipped)
  6. Return DryRunResult instead of Frame

2. DryRunResult model

@dataclass
class DryRunResult:
    """Result of a dry-run invocation showing what would happen."""
    token_valid: bool
    policy_decision: PolicyDecision
    driver_id: str
    operation: str
    resolved_params: dict
    response_mode: ResponseMode
    budget_remaining: int | None
    estimated_cost: str  # "low" / "medium" / "high" based on driver type

3. Use cases

  • Policy testing: dry_run=True validates that a principal has access without executing
  • CI pipelines: Validate policy YAML/TOML changes against test scenarios
  • Debugging: Compare DryRunResult for two different principals to understand access differences

Acceptance Criteria

  • invoke(token, params, dry_run=True) returns DryRunResult without calling driver
  • DryRunResult includes policy decision, driver info, response mode, and budget
  • Expired/revoked tokens still raise TokenVerificationFailed in dry-run
  • Policy denials still raise PolicyDenied in dry-run
  • No side effects: driver execute() is never called
  • Runs significantly faster than real invocation (no I/O)
  • Works with all driver types (InMemory, HTTP, MCP)

Affected Files

  • src/agent_kernel/kernel.py (add dry_run parameter to invoke())
  • src/agent_kernel/models.py (add DryRunResult dataclass)
  • tests/test_kernel.py (dry-run specific tests)
  • docs/capabilities.md (document dry-run usage)

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions