Skip to content

feat: add evaluation tables and data to test skills against updates #104

Description

@ryzizub

Description

Add evaluation tables and reference data so this plugin's skills can be tested against Claude Code updates. Today there is no structured way to verify a skill still behaves correctly after a Claude Code change. Define eval cases (inputs and expected outcomes) per skill so regressions are caught automatically rather than discovered in use.

Requirements

  • Define an eval data format (input scenario, expected behavior/output, pass criteria) reusable across skills.
  • Store eval tables/data co-located with each skill or under a shared evals/ directory.
  • Seed eval cases for the core skills (bloc, testing, navigation, layered-architecture, create-project).
  • Provide a runner that executes the evals and reports pass/fail.
  • Document how to add a new eval case when adding or changing a skill.

Additional Context

Consumed by the weekly changelog-check bot: when a skill is flagged as potentially impacted, its eval table is the gate for confirming it still works. See #103.

Metadata

Metadata

Assignees

Labels

featureA new feature or requestp1High-priority issues at the top of the work list

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions