feat(evaluator-sdk): agent-skill injection for FabricContainerRuntime

[SandyChapman]

What

Adds agent-skill injection to the sandboxed FabricContainerRuntime, so an isolated (Docker) Fabric eval can run WITH an agentskills.io skill — previously only the host FabricAgentRuntime supported skills (limitation L1 from the LAB-examples gap analysis). It mirrors the host runtime's skill design and extends the same two-mode contract to the container.

How

• __init__ gains a skill: AgentSkill | None; new with_skill() clone method so an A/B eval derives baseline (with_skill(None)) and treated (with_skill(skill)) runtimes from one instance.
• The injection mode is resolved once per run (the adapter is constant across the taskset) via Fabric's capability planner (_resolve_skill_mode). nemo_fabric is imported lazily on the host only when a skill is set, so the no-skill container path stays dependency-free.
• New stage_skill_seed() in skills.py — the sandboxed sibling of install_skill. Instead of a host copytree (the container has no host workspace), it renders the bundle into the sandbox seed set (SandboxSpec.files) at the harness's in-sandbox discovery path:
  • native (hermes / any adapter that accepts: ["skills"]) → /in/skills/<name>/ + a skills.paths profile overlay (preserving any pre-configured skill paths, last-wins);
  • codex → <workspace>/.agents/skills/<name>/ (self-discovery), no overlay.
• The codex bundle is scrubbed from the downloaded /out/workspace before it's exposed as evidence (so injected files don't skew workspace-reading metrics). Native staging lives under /in and is never downloaded, so it needs no cleanup.
• Introduces SkillMode = Literal["native", "codex_skills_dir"], threaded through both runtimes and SkillProvenance.mode.

Text bundles only: the seed set is dict[str, str], so a binary file raises SkillInjectionError (the host install_skill copytree path still handles binary bundles).

Tests

• 8 new unit cases in test_fabric_container_runtime.py mirroring the host skill-injection suite: native seed-set + overlay, preconfigured-path preservation, runtime-discovered adapter, codex seed + evidence exclusion, fail-fast on an unsupported adapter, no-skill-no-probe, and with_skill copy. Full fabric suite (container + host + skills) = 76 passing; ruff + format clean; ty clean; SDK mirror re-vendored (make vendor).
• Live-validated (native / hermes): ran one task twice through the runtime against a real Docker sandbox — baseline vs. a skill mandating an unguessable marker token. The marker appeared only in the treated output, confirming the agent genuinely picks up the injected skill. Provenance (mode=native, staged path, content hash) stamped correctly; no leaked containers; native skill correctly absent from workspace evidence.
• Codex mode is unit-tested only. A full sandboxed codex run needs the codex CLI in the image, which the Fabric sandbox image does not provision yet (hermes+relay only) — separate from this change.

Notes

• The codex branch is a workaround for a gap in Fabric's codex adapter, not a missing abstraction: the codex fabric-adapter.json declares accepts: ["models", "telemetry"] (no skills), so Fabric routes codex skills unsupported, while the codex CLI itself self-discovers .agents/skills/. When the codex adapter adds skills, this branch can be deleted and codex flows through the same native path.

[github-actions]

Suite	Lines Covered	Line Rate	Branch Rate
Unit Tests	26546/34202	77.6%	61.9%
Integration Tests	15213/32827	46.3%	18.6%

[ngoncharenko]

one idea to reduce some duplication and also re-use for other runtimes in the future: store skills state in a standalone class

# skills.py
@dataclass(frozen=True)
class SkillSet:
    skills: tuple[AgentSkill, ...] = ()

    def __post_init__(self) -> None:
        require_unique_skill_names(self.skills)

    def with_skills(self, skills: Sequence[AgentSkill]) -> SkillSet:
        combined = (*self.skills, *skills)
        require_unique_skill_names(combined)
        return SkillSet(skills=combined)

    def with_skill(self, skill: AgentSkill) -> SkillSet:
        return self.with_skills([skill])

class XYZRuntime:
   # runtime.py / container_runtime.py
   def __init__(..., skills: Sequence[AgentSkill] | None = None) -> None:
       ...
       self._skill_set = SkillSet(skills=tuple(skills or ()))
   def with_skills(self, skills: Sequence[AgentSkill]) -> FabricAgentRuntime:  # or FabricContainerRuntime
      """Return a copy with ``skills`` added; ``self`` is not modified."""
      clone = copy.copy(self)
      clone._skill_set = self._skill_set.with_skills(skills)
      return clone
  def with_skill(self, skill: AgentSkill) -> FabricAgentRuntime:
      return self.with_skills([skill])

[ngoncharenko]

Task files can overwrite potentially Codex injected skill files if their path matches:

skills files:

myskills/code-review/
└── SKILL.md

task files:

task.inputs = {
    "instruction": "Review this change",
    "files": {
        ".agents/skills/code-review/SKILL.md": """<inline_skill>"""

Is it something we should guard against? Hash will also point to the injected skill, but the content will be different

Example:

reserved = PurePosixPath(".agents/skills") / skill.name

for rel_path in task.inputs.get("files", {}):
    seed_path = PurePosixPath(rel_path)
    if seed_path == reserved or reserved in seed_path.parents:
        raise SkillInjectionError(
            f"task seed collides with injected skill path {reserved}"
        )

[ngoncharenko]

if we injected a binary file (image, for example, as a part of skills assets), this will error out. Should we allow passing binary assets?