Skip to content

fix(cli): make template directives true on the file's language and tests - #38

Merged
jameslcowan merged 4 commits into
mainfrom
fix/directive-truth
Oct 25, 2026
Merged

jameslcowan merged 4 commits into
mainfrom
fix/directive-truth

Conversation

@jameslcowan

Copy link
Copy Markdown
Collaborator

What changed

An audit of 20 hosted maps checked 95 "Before editing this file" directives of template prose against the repositories. 11 were false. They came from 3 causes in the CLI.

  1. Wrong test command for the file's language (4 directives). Template prose used 1 repository wide command from detectTestCommand. hkuds/nanobot's webui/src/lib/types.ts got pytest from the root pyproject.toml, though webui/package.json runs vitest run. 3 Python files in debpalash/voicestudio's backend/services/ got npm test, which runs only the Electron suite. Fix: testCommandFor in cli/src/signals.js takes the nearest test setup of the file's language. That is the nearest package.json with a test script for JavaScript and TypeScript, the nearest pytest.ini, conftest.py or pytest section for Python, go.mod for Go and Cargo.toml for Rust. A setup in a subfolder is named ("run the full test suite in webui with npm test"). When no setup fits, the directive names no command. check and the MCP server still use detectTestCommand.
  2. A named covering test that does not test the file (2 directives). jamiepine/voicebox named the empty backend/tests/__init__.py. vitejs/vite named packages/plugin-legacy/src/__tests__/index.spec.ts for packages/vite/src/node/server/index.ts. The import resolver is correct: that spec's '../index' resolves to plugin-legacy's own src/index.ts. The name pass in cli/src/graph.js caused both cases. It matched the stems index and __init__ for any test under the same top folder (packages/, backend/). Fix: a covering test must run a test case. __init__.py never counts, and a Python or Go file counts only with a test function. A name match needs the same package (the folder of the nearest manifest) and the same language family. A stem that 2 ranked files of 1 package share matches only the file beside the test or beside its test folder.
  3. "no test covers it" for a file that has a test (5 directives, chartjs/chart.js). test/specs/core.scale.tests.js and 4 more specs test their files through the global Chart, with no import. The .tests.js suffix kept the stem from matching, and the old top folder rule kept test/ apart from src/. Fix: the stem drops .tests and .specs, and the signals carry name_tests, the tests that only the name found. The paragraph and the directive now state how each test was found: "run test/specs/core.scale.tests.js, which matches it by name". When there is no test at all, the prose says "no test file imports it or matches its name", because that is what the tool measured.

content/spec.md states the new covering test rule. Its partners cli/README.md, skill/SKILL.tmpl.md and content/index.html do not describe the stem rule, so they needed no change. cli/bin needed no change for the cli/bin and cli/src seam. The .tests stem change is in the first commit (graph.js), although the third commit message describes it as part of the chart.js fix.

How it was tested

  • cd cli && npm test: 793 tests, exit 0. cd cli && node --test test/truth.test.js: 45 pass, exit 0. npm run build at the root: exit 0.
  • cli/test/directive-truth.test.js adds 5 tests. They rebuild the nanobot, voicestudio, voicebox, vite and chart.js cases in small file trees, and they cover testCommandFor per language. The 4 case tests failed on origin/main with the reported wrong output. The vite assertion failed on its own against the original graph.js.
  • 8 Python fixtures in cli/test/classify.test.js gain the def test_ function that a real pytest file has. The 2 signal goldens gain the empty name_tests key. No other golden value changed.
  • I checked the 5 repositories at the audited commits with node cli/bin/complex-md.js generate --template --no-wire --yes. I used no provider key and no --via, so no model ran. Each clone was a fetch of the audited sha with 2001 commits of history, and I deleted the clones after. The 11 directives before and after:
Repository File Before After
hkuds/nanobot webui/src/lib/types.ts run the full test suite with pytest run the full test suite in webui with npm test
jamiepine/voicebox backend/backends/__init__.py run backend/tests/__init__.py, test_designed_profiles.py and test_kokoro_trim.py run test_designed_profiles.py, test_kokoro_trim.py and test_models_load_by_name.py
vitejs/vite packages/vite/src/node/server/index.ts run plugin-legacy/src/__tests__/index.spec.ts, config.spec.ts and dev.spec.ts run config.spec.ts, dev.spec.ts and environment.spec.ts
debpalash/voicestudio backend/services/model_manager.py run the full test suite with npm test run the full test suite with pytest
debpalash/voicestudio backend/services/tts_backend.py run the full test suite with npm test run the full test suite with pytest
debpalash/voicestudio backend/services/asr_backend.py run the full test suite with npm test run the full test suite with pytest
chartjs/chart.js src/core/core.controller.js open 2 importers, because no test covers it run test/specs/core.controller.tests.js, which matches it by name
chartjs/chart.js src/core/core.scale.js open 2 importers, because no test covers it run test/specs/core.scale.tests.js, which matches it by name
chartjs/chart.js src/core/core.datasetController.js open 2 importers, because no test covers it run test/specs/core.datasetController.tests.js, which matches it by name
chartjs/chart.js src/core/core.layouts.js open 2 importers, because no test covers it run test/specs/core.layouts.tests.js, which matches it by name
chartjs/chart.js src/plugins/plugin.tooltip.js open 2 importers, because no test covers it run test/specs/plugin.tooltip.tests.js, which matches it by name
  • The newly named tests exist and exercise the file. test_models_load_by_name.py imports backend.backends and defines 2 tests. environment.spec.ts imports createServer from ../server. The 5 chart.js specs exist under test/specs/. voicestudio's root pytest (testpaths = ["tests"]) runs tests/test_model_manager_preload.py and other tests of these modules. Its backend/tests/ folder runs as a separate pytest session in CI.
  • 5 other directives in these maps changed wording, and all are true. 2 voicestudio Electron files now say "in electron with npm test", the suite that the root script delegates to. 2 voicebox app files now say "no test file imports it or matches its name"; that app has no test files.

AI assistance

  • An AI tool helped write this change. The tool and the part it wrote are stated above.
  • Each commit the tool helped write has an Assisted-by: or a Co-Authored-By: trailer.
  • A person read the whole diff and will answer the review comments.

Claude (Claude Code) wrote all of this change: the code, the tests, the spec and CHANGELOG text, the check on the 5 repositories and this description. Each commit carries Assisted-by: Claude. No person has read the whole diff yet.

🤖 Generated with Claude Code

Problem: an audit of 20 hosted maps found 11 false directives. 2 of them
named a test that does not test the file. On jamiepine/voicebox the
directive for backend/backends/__init__.py named backend/tests/__init__.py,
a 7 line package file. On vitejs/vite the directive for
packages/vite/src/node/server/index.ts named
packages/plugin-legacy/src/__tests__/index.spec.ts. That spec imports
'../index', and the resolver maps it correctly to plugin-legacy's own
src/index.ts. The name pass caused both: it matched the stem "index" or
"__init__" for any test under the same top folder, packages/ or backend/.

Fix: a covering test must run a test case. __init__.py never counts, and
a Python or Go file counts only with a test function. The name pass now
needs the same package (the folder of the nearest manifest) and the same
language family. A stem that 2 ranked files of 1 package share matches
only the file beside the test or beside its test folder. The pass also
records which tests only the name found. The spec states the rule.
cli/test/directive-truth.test.js rebuilds both cases; 8 Python fixtures
in classify.test.js gain the test function a real pytest file has.
cd cli && npm test: 789 tests, exit 0. The signal goldens did not change.

Intent: a directive names only tests that exercise the file.
Assisted-by: Claude
Problem: an audit of 20 hosted maps found 11 false directives. 4 of them
named a test command for the wrong language, because template prose used
1 repository wide command. hkuds/nanobot's webui/src/lib/types.ts got
pytest from the root pyproject.toml, though webui/package.json runs
vitest run. 3 Python files of debpalash/voicestudio's backend got npm
test, which runs only the Electron suite, though pytest runs their tests.

Fix: testCommandFor in signals.js gives each file the nearest test setup
of its language: the nearest package.json with a test script for
JavaScript and TypeScript, the nearest pytest.ini, conftest.py or pytest
section for Python, go.mod for Go, Cargo.toml for Rust. A setup in a
subfolder is named: "run the full test suite in `webui` with `npm
test`". When no setup fits, the directive names no command. check and
the MCP server keep detectTestCommand. 4 tests in
cli/test/directive-truth.test.js rebuild the 2 cases and cover each
language. cd cli && npm test: 792 tests, exit 0; truth.test.js passes.

Intent: a "run the full test suite" directive names a command that runs
the tests of that file's language.
Assisted-by: Claude
…easured

Problem: an audit of 20 hosted maps found 11 false directives. 5 of them,
all on chartjs/chart.js, said "because no test covers it" for
src/core/core.controller.js, core.scale.js, core.datasetController.js,
core.layouts.js and src/plugins/plugin.tooltip.js. Each has a test:
test/specs/core.scale.tests.js and the others reach the file through the
global Chart, with no import. The name pass missed them for 2 reasons:
the .tests.js suffix kept the stem from matching, and test/ and src/ are
different top folders. The previous commit made the package the scope.
The prose then claimed more than the tool measured.

Fix: the name pass strips .tests and .specs, so the stem of
core.scale.tests.js is core.scale. The signals carry name_tests, the
covering tests that only the file name found. When every covering test
is such a match, the paragraph says "No test file imports it, and
`test/specs/core.scale.tests.js` matches its name", and the directive
says "run `test/specs/core.scale.tests.js`, which matches it by name".
With no test at all, the prose says "no test file imports it or matches
its name", which is what was measured. The 2 signal goldens gain the
empty name_tests key. cli/test/directive-truth.test.js rebuilds the
chart.js layout. cd cli && npm test: 793 tests, exit 0; truth.test.js
passes.

Intent: a directive never says no test covers a file that has a test,
and it states how each named test was found.
Assisted-by: Claude
Problem: the 3 fixes on this branch change the directives that template
prose writes, and CHANGELOG.md had no entry for them. An audit of 20
hosted maps found 11 false directives among 95.

Fix: 1 Unreleased entry states the audit, the 3 causes with counts, the
new rules and wording, and the result: on the 5 audited repositories at
the audited commits, all 11 directives now name a true command or test.

Intent: record why the directive wording and the covering test rules
changed.
Assisted-by: Claude
@jameslcowan
jameslcowan merged commit 06e8a1a into main Oct 25, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant