skillmark

CI-native linter, validator, and quality scorer for Agent Skills (SKILL.md).

• 76 rules implemented (more on the Roadmap) — spec compliance (errors), script-level security (errors), best practices (warnings), quality scoring (info)
• Deterministic security scanning — AST-based checks (tree-sitter) flag dangerous patterns in bundled Python and shell scripts: code execution, obfuscation, credential access, remote-exec pipes, reverse shells
• Quality scoring — 0-100 composite score across 7 weighted categories, letter grades A-F
• 4 output formats — terminal (colored), JSON, SARIF (GitHub code annotations), Markdown (PR comments)
• Fix mode — auto-repair 6 common issues with --dry-run preview
• CI-native — GitHub Action, pre-commit hook, configurable via .skillmark.toml

Quick Start

# Install
cargo install skillmark

# Check a skill
skillmark check path/to/my-skill

# Check all skills in a repo
skillmark check

# Fix issues
skillmark fix path/to/my-skill --dry-run
skillmark fix path/to/my-skill

Example Output

skillmark v0.1.0 — my-skill

  Score: 93/100 (A)

  Spec Compliance    ████████████████████  100%  (35.0/35.0)
  Security           ████████████████████  100%  (15.0/15.0)
  Description        ████████████████████  100%  (15.0/15.0)
  Content Efficiency ████████████████░░░░   80%  (8.0/10.0)
  Composability      ████████████████████  100%  (10.0/10.0)
  Script Quality     ████████████████████  100%  (10.0/10.0)
  Discoverability    ░░░░░░░░░░░░░░░░░░░░    0%   (0.0/5.0)

  0 warnings, 0 errors

GitHub Action

- uses: michellepellon/skillmark@v1
  with:
    min-score: '80'
    format: sarif

- uses: github/codeql-action/upload-sarif@v3
  with:
    sarif_file: skillmark.sarif

Or for PR comments:

- uses: michellepellon/skillmark@v1
  with:
    min-score: '80'
    format: markdown
  id: lint
- run: echo "${{ steps.lint.outputs.summary }}" >> $GITHUB_STEP_SUMMARY

Pre-commit Hook

# .pre-commit-config.yaml
repos:
  - repo: https://github.com/michellepellon/skillmark
    rev: v0.1.0
    hooks:
      - id: skillmark
      - id: skillmark-fix
        stages: [manual]

Configuration

Create .skillmark.toml in your repo root:

fail-on = "errors"
min-score = 80

[rules]
disable = ["W022", "E035"]
experimental = false

[scoring.weights]
spec-compliance = 0.35
security = 0.15
description-quality = 0.15
content-efficiency = 0.10
composability-clarity = 0.10
script-quality = 0.10
discoverability = 0.05

[paths]
exclude = ["drafts/", "vendor/"]

CLI Reference

skillmark check [PATHS...]    Validate, lint, and score skills
skillmark fix [PATHS...]      Auto-repair fixable issues

Options:
  --format <FORMAT>     terminal | json | sarif | markdown  [default: terminal]
  --min-score <N>       Exit non-zero if score < N
  --fail-on <LEVEL>     errors | warnings | none            [default: errors]
  --no-score            Skip scoring (faster)
  --quiet               Only output diagnostics
  --disable <RULES>     Comma-separated rule IDs to disable
  --experimental        Enable Tier 2 heuristic rules
  --exclude <GLOBS>     Comma-separated paths to exclude
  --config <PATH>       Path to .skillmark.toml
  --fix                 Run fix mode (with check)
  --dry-run             Preview fixes without writing (with fix)
  --color <WHEN>        auto | always | never               [default: auto]

Exit Codes

Code	Meaning
0	All checks pass
1	Errors or warnings found (per --fail-on)
2	Score below --min-score threshold
3	No SKILL.md files found

Scoring Categories

Category	Weight	What it measures
Spec Compliance	35%	AgentSkills.io spec conformance (37 rules)
Security	15%	Dangerous patterns in bundled scripts (AST-based, 9 rules)
Description Quality	15%	Trigger language, action verbs, keyword diversity
Content Efficiency	10%	Line count, word count, progressive disclosure
Composability & Clarity	10%	Structure, no filler, no placeholders
Script Quality	10%	Error handling, no hardcoded paths, help docs
Discoverability	5%	License, references, examples, troubleshooting, gotchas, validation

Rule Reference

Errors (E001-E037) — Spec Compliance

37 rules, all default-on. These fail CI by default. See the AgentSkills.io specification for the authoritative reference.

Rule	What it checks
E001-E005	SKILL.md existence, frontmatter validity
E006-E013	Name format (length, chars, case, directory match, NFKC)
E014-E016	Description (required, non-empty, length limit)
E017-E029	Optional field types and constraints
E030	Unknown frontmatter fields
E031	Broken file references
E032-E034	Structural issues (unclosed frontmatter, BOM)
E035	Secret/credential detection
E036	XML angle brackets (<, >) in frontmatter values
E037	Reserved name prefix (claude, anthropic)

Security (S001-S012) — Script-Level Checks

9 rules, all default-on, severity error (gate CI like spec errors). Deterministic AST-based checks on bundled scripts — every .py/.sh/.bash/.zsh in the skill, not just scripts/. Python is parsed with tree-sitter-python, shell with tree-sitter-bash. Skills with no scripts score 100% on this dimension.

Rule	Language	What it checks
S001	Python	exec() of dynamically-constructed code
S002	Python	eval() of dynamic expressions
S003	Python	subprocess(…, shell=True) — shell injection surface
S004	Python	os.system / os.exec* / os.spawn* command execution
S005	Python	Obfuscated execution — exec/eval fed by base64/codecs/marshal decode
S006	Python	Hardcoded credential-file access (.ssh/id_rsa, .aws/credentials, …)
S010	Shell	Remote-exec pipe — curl/wget piped into sh/bash
S011	Shell	Reverse shell — /dev/tcp//dev/udp redirect or nc -e
S012	Shell	Destructive rm -rf on a system or home path

Reserved for follow-ups: S007–S009 (Python), S013+ (shell).

Warnings — Best Practices

13 rules implemented. Advisory by default. Based on AgentSkills.io best practices and SkillsBench findings.

Rule	What it checks
W001	Body exceeds 500 lines
W002	Body exceeds 5,000 words
W003	Description shorter than 50 characters
W004	Description lacks trigger language ("Use when…")
W005	Description starts with passive voice
W006	Body contains placeholder text
W007	Name contains TODO/FIXME/TBD marker
W008	Description contains TODO/FIXME/TBD marker
W009	Body has no headings
W020	Description matches a vague anti-pattern
W021	Body contains filler phrases
W023	Description lacks action verbs
W024	File in scripts//references//assets/ not referenced in body
W029	README.md (or other README variant) inside the skill folder

Info — Quality Scoring Inputs

11 rules implemented. Not shown in terminal output by default. Affect the quality score only.

Rule	What it checks
I001	No scripts/ directory
I002	No references/ directory
I003	No assets/ directory
I004	No examples/ directory
I005	No heading for gotchas/pitfalls/caveats
I006	No heading for validation/verification/testing
I007	No heading for examples (e.g. ## Examples)
I008	No heading for troubleshooting / error handling
I009	No metadata.author declared (guide suggests it)
I010	No metadata.version declared (guide suggests it)
I011	Body mentions MCP but no metadata.mcp-server declared
I012	No license field and no LICENSE file
I013	Description has no negative-trigger phrasing (Tier 2: requires --experimental)
I014	Body has imperative validation language but no scripts/ directory (Tier 2)
I015	Description has fewer than 2 trigger scenarios
I016	Progressive-disclosure ratio below 0.2 on bodies ≥2 KB (move detail to references/)

Roadmap

Rule IDs reserved for future work — referenced by scoring or experimental gates but not yet implemented. PRs welcome.

Rule	Planned check
W018, W025–W028	Script-quality and content-efficiency rules referenced in scorer.rs
W030–W032	Composability heuristics (Tier 2)
S007–S009	Additional Python security checks (reserved in SECURITY_RULES)
S013+	Additional shell security checks (reserved in SECURITY_RULES)

Acknowledgments

Built on the AgentSkills.io open specification. Informed by:

• agent-skills-lint (Rust)
• agent-skill-validator (TypeScript)
• SkillsBench (benchmark findings on skill quality)
• Official skills-ref library
• SkillSpector (NVIDIA — inspiration for the deterministic script-level security checks)

License

MIT