Skip to content

coding-standard skill generates standards that cite volatile codebase state, so produced standards go stale #73

@taminomara

Description

@taminomara

Issue Type

Feature Request

Reported Behavior

Standards authored by the upstream Han coding-standard skill embed volatile
codebase state and therefore go stale quickly. Two specific patterns recur in
generated documents:

  1. file:line references in "Project references" sections (e.g.
    src/foo.ts:142), which drift silently whenever lines are inserted above
    them.
  2. Current-state enumerations ("Atoms today: …", "All 10 current services",
    "Variants registered today") that pin a standard to a momentary codebase
    roster.

Prior research in the consuming repo (SimpleWiki) traced the mechanism to the
skill itself:

  • The skill's SKILL.md Step 4 instructs the evidence-gathering agent to return
    "concrete file paths and line ranges" and "Correct-usage / What-to-avoid
    candidates (file:line references)". Authors transcribe that intermediate
    file:line format directly into the final document.
  • The skill's references/template.md **Project references:** section is
    already file-path-only (path/to/implementation, no :NN), so the template
    structure is not the cause — the leak is upstream of it, in the Step 4
    prompt and the absence of a sanitizing/verification step.
  • The skill carries no durability guidance anywhere (no mention of stale,
    brittle, timeless, line number) and no instruction to frame applicability
    as a rule rather than a current-state roster.

The research recommended a layered fix and explicitly scoped it to the skill
level for the authoring side: change the Step 4 prompt to request
symbol-anchored references and/or add a verification step that strips :NN and
substitutes the stable symbol; add durability guidance; reframe
"current-consumer / today" enumerations as membership rules; and leave the
Project references template structure untouched.

Expected Behavior

The coding-standard skill should generate standards that reference durable
anchors
by default — the exported symbol (function / constant / interface /
type) or a stable heading / section, rather than a file:line — and should
express who a rule applies to as a criterion ("any module that renders
user-supplied HTML") rather than a snapshot roster of current consumers. This
should hold without manual upkeep, so a freshly authored standard does not go
stale on the next unrelated edit to a cited file.

Missing Information

The problem space is already researched (see the SimpleWiki research report
docs/research/coding-standard-durable-references.md); the open items below are
feature-scoping decisions within the chosen direction, not problem-space gaps:

  • Mechanism choice for the file:line leak — whether to (a) rewrite the
    Step 4 evidence-gathering prompt to request symbol-anchored references, (b) add
    a Step 6/Step 10 verification step that strips :NN and substitutes the
    exported symbol, or (c) both. Not yet decided.
  • Success criteria / acceptance threshold at the skill level — e.g. "a
    generated standard contains zero :NN suffixes in Project references", whether
    durability must be machine-checkable, and how much residual staleness (symbol
    renames are still possible) is acceptable.
  • Form of the durability guidance — prose guidance in the template vs. an
    enforced verification gate vs. both.
  • Scope within the template — whether the change covers only the
    Project references section, or also "Applies To" headers and inline
    "today / currently" phrasing throughout the generated document.
  • CI/lint resolve-check — whether the skill should recommend a machine
    resolve-check (cited path/symbol still resolves) as part of the standard it
    emits, or explicitly leave that out of scope. (The research deferred this
    behind a tripwire — no packaged tool verifies a symbol-in-prose today.)
  • Retrofit guidance — whether the skill's "update existing standard" path
    should prescribe converting already-authored file:line references and
    rosters, or whether retrofit is left entirely to the consuming repo.

Suspected Areas

  • Upstream Han coding-standard skill — SKILL.md Step 4 evidence-gathering
    prompt (the file:line / "line ranges" instruction) and the Step 6 / Step 10
    document-assembly and verification steps.
  • Upstream references/template.md — the **Project references:** section
    (already file-path-only; confirm it stays that way) and any "Applies To" /
    current-state phrasing the template seeds.
  • Cross-cutting: the skill's lack of any durability / timeless-documentation
    guidance, which lets file:line and "today" enumerations through unchallenged.

Recommended Next Step

/plan-a-feature

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions