Skip to content

v0.1.0 ships without conformance test suite, registry, or steward, leaving the label unfalsifiable #19

@rocklambros

Description

@rocklambros

Surfaced during the v0.1.0 review (PR #2). Governance gap. Filing for tracking and disclosure work.

There's no defined process for declaring conformance. No test suite. No registry of conformant implementations. No steward body. The handshake's profiles_supported / profiles_accepted exchange is pure self-declaration on the wire.

A vendor can advertise "ACS-Core conformant" in marketing. A buyer can procure on that basis. An incident can occur. The buyer asks "who certified this?" The answer is nobody. The label has the same epistemic weight as "military-grade."

There's a defensible argument that v0.1.0 is a spec release, not a certification program. OTel, IETF RFCs, and CycloneDX all shipped wire formats before formal conformance programs existed. The counter-argument is that ACS markets itself as a control standard, not a wire format. The verifiability problem matters more here than for OTel. A safety/security label without a falsification process accumulates negative trust faster than a delayed-but-rigorous one. Six months of "ACS-compliant" marketing with no verifier is precisely the failure mode the spec was built to prevent at the agent layer, now reflected at the standard layer.

Two-phase approach:

v0.1.0 minimum hygiene (before tag):

  • One paragraph in the README declaring conformance is currently self-attested
  • Matching paragraph at the top of conformance.md
  • Roadmap note that a test suite and verification process is targeted for v0.2

That turns unfalsifiability into known-and-flagged. Costs an hour. Saves the standard's credibility from the first vendor over-claim.

v0.2+:

  • Reference Guardian + reference Observed Agent implementations under an open license
  • Test vectors per profile, exercising every MUST and SHOULD in the spec
  • A submission process for vendor implementations to demonstrate conformance against the test vectors
  • A registry, even a simple one, of vendor implementations and the profiles they claim
  • A revocation path for claims that don't hold up under independent testing

Why deferred from v0.1.0: a full conformance program is months of work and shouldn't gate the spec release. The disclosure step in v0.1.0 minimum hygiene is the only piece that should land before tag.

References: conformance.md, acs.md landing

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions