C2: single-source template render + fence-aware managed-file copier#155
Merged
Conversation
Scaffold src/mapify_cli/delivery/template_renderer.py: D7 custom-delimiter Jinja2 Environment ([% %]/<% %>/[# #], keep_trailing_newline, autoescape off), lazy jinja2 import (INV-9/VC4), render-to-tempdir byte-parity gate writing .claude/hooks/ LAST (INV-9/HC-8), and assert_no_stray_delimiters guard (D7a). Adds tests/test_template_render.py (26 tests). No templates_src yet (ST-002/3). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Create 82 passthrough .jinja under src/mapify_cli/templates_src/ (agents,
hooks, references, skills, root configs, map/scripts, map/static-analysis,
rules/learned README scaffold) — verbatim copies of committed Claude files,
no fences (C1). Extend template_renderer.py with a destination-resolver
layer: render_repo_trees() + _build_claude_resolver routes each rendered
file to BOTH src/mapify_cli/templates/ and the dev tree (.claude/, with
map/ -> .map/ remap), keeping the 4 root configs + hooks/README.md +
rules/learned/README.md shipped-only. hooks-last (INV-9) now spans both
.claude/hooks/ and templates/hooks/. _build_codex_resolver is an ST-003
stub. ST-001 identity render_tree preserved.
render_repo_trees('claude') reproduces .claude/** and templates/**
byte-identically (empty git diff, HC-5/AC-1); lint-hooks green (INV-4);
ruff/mypy/pyright 0/0/0; 36 render tests + full suite (1823) green.
Scope note: template_renderer.py edit was a user-approved expansion of
ST-002 to build the destination-map.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Create 13 passthrough .jinja under templates_src/codex/ (AGENTS.md, config.toml, hooks.json, agents/*.toml [3x ~44KB], hooks/workflow-gate.py, skills/**) — verbatim copies, no fences (C1). Implement _build_codex_resolver (replacing the ST-002 stub): codex skills -> templates/codex/skills + .agents/skills; everything else -> templates/codex/ + .codex/. Scope codex render to templates_src/codex. Extend _HOOK_PARENT_SEQUENCES with (.codex,hooks)+(codex,hooks) so all 4 workflow-gate.py copies sort LAST (INV-9). Add codex/ early-exit to claude resolver so claude render never leaks into .claude/codex/. render_repo_trees for claude (158) + codex (26) reproduces .claude/**, .codex/**, .agents/skills/**, templates/** byte-identically (empty git diff, HC-5). 4-copy workflow-gate parity + guard-free (VC3); lint-hooks green; ruff/mypy/pyright 0/0/0; 47 render tests + full suite (1834) green. Per-provider .jinja bodies, no forced shared body (D2/SC-1). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ST-004) Add dev-only `make render-templates` target (renders claude + codex via python -m mapify_cli.delivery.template_renderer). Fix the renderer __main__ entrypoint to call render_repo_trees (was render_tree identity). Ship the .jinja sources for transparency (D6): add templates_src/**/*.jinja to hatch sdist.include + artifacts and templates_src to wheel.force-include (additive — templates/ still ships). sync-templates kept until ST-007. make render-templates exits 0 with empty git diff; uv build packages 95 .jinja in both wheel and sdist; full suite (1834) green; 0/0/0. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…_sync (ST-005) Delete tests/test_template_sync.py (dual-copy parity test, superseded). Add TestGoldenFixtures to tests/test_template_render.py: per-provider golden byte-equality vs committed snapshots loaded from disk (tests/fixtures/claude/references/host-paths.md, tests/fixtures/codex/ config.toml) — independent ground truth, NOT render==render (HC-2) — plus negative mutation tests proving the gate catches divergence. ci.yml repoint deferred to ST-007 (planned). 52 render tests green; full suite 1786 green; ruff/mypy/pyright 0/0/0. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add `make check-render`: renders claude+codex then `git diff --exit-code` across templates/**, .claude/**, .codex/**, .agents/skills/**, restoring those paths via `git checkout --` on both pass and fail (INV-2). Wire it into `make check` and add a "Render parity check" CI step. A stale templates_src edit without re-render now fails the gate (negative-proven). test_template_sync ci steps left for ST-007 to remove. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…nder-templates - Delete scripts/sync-templates.sh and remove Makefile target + .PHONY + help entry - ci.yml: replace tests/test_template_sync.py::TestCodexTemplateSynchronization with tests/test_template_render.py::TestRenderRepoTreesCodex - src/mapify_cli/delivery/template_renderer.py: error msg sync→render - src/mapify_cli/repo_insight.py + schemas.py: suggested_checks sync→render - scripts/lint-hooks.py: docstring sync→render - tests/test_skills.py: failure message strings sync→render - tests/test_template_render.py: skip-reason text sync→render - tests/test_repo_insight.py: assertion strings sync→render - tests/test_mapify_cli.py: comments repointed to test_template_render.py - templates_src/CLAUDE.md.jinja + skills/README.md.jinja + hooks/end-of-turn.sh.jinja: sync→render model; re-rendered generated outputs (.claude/, templates/) - Repo-root CLAUDE.md: rewrite "Critical invariant" section to single-source render model - docs/ARCHITECTURE.md, roadmap.md, improvement-plan*.md, context-compression-plan.md, triz-cheatsheet.md, improvements-plan.md, MAP_PLATFORM_SPEC.md: sync→render - RELEASING.md: sync→render + test_template_render.py - .claude/rules/learned/architecture-patterns.md: rewrite Dual-Copy + N-Copy learned rules to describe make render-templates single-source render model rg -n 'sync-templates|sync_templates' --glob '!.map/**' → ZERO hits make sync-templates → "No rule to make target" make test (1785 passed), make lint (0/0/0), YAML OK Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add tests/test_init_import_graph.py (7 tests): subprocess fresh-interpreter assertions that importing the mapify init dispatch chain loads NEITHER mapify_cli.delivery.template_renderer NOR jinja2 (INV-6/AC-9), plus checks that providers install via plain copy (copy_managed_file/create_codex_files, no render_tree/render_repo_trees) and jinja2 stays a runtime dep (AC-9). providers.py unchanged — init path was already renderer/jinja2-free. 7 import-graph + jinja2_dep green; full suite 1794; ruff/mypy/pyright 0/0/0. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Make copy_managed_file's write/merge side fence-aware (C2). Per-format fences: md HTML-comment, py/sh/toml/yaml hash, JSON none (fully-managed via _map_managed + .bak). On re-copy, refresh the managed region and preserve below-fence user content BYTE-FOR-BYTE (INV-5). INV-T transition (metadata-but-no-fence -> fully managed + migration notice). D12 recovery (deleted/malformed fence -> user-owned, warn, no clobber). All writes routed through O_NOFOLLOW atomic write with symlink refusal; never writes outside the target (VC5). extract/inject/detect_drift logic unchanged (D3; .sh/.toml/.yaml metadata branches additive). _split_fence uses FULL-LINE standalone matching (ln.strip()==token) with count-based strictness so a fence sentinel literal in user content is data, not a marker — fixing an INV-5 data-loss edge case (Monitor round 1). Duplicate-start/missing-end/inverted -> D12 user-owned. 77 copier tests incl. sentinel-in-tail round-trip (negative-proven); full suite 1863; 0/0/0. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…rc text files Wraps every managed region in templates_src/**/*.jinja with per-format fence markers (md: <!-- map:start/end -->, py/sh/toml: # map:start/end); JSON skipped. Re-renders all generated trees (.claude/, .codex/, .agents/skills/, src/mapify_cli/templates/) to propagate fences. Updates ST-005 golden fixtures (escalation-matrix.md, config.toml). Bumps test_skills.py SKILL.md line budget 500→502 (deliberate C2 fence addition, per learned 'always-loaded skill body line budget' rule). 90 templates_src files fenced, 267 generated files updated. - All safety checks green: lint-hooks.py, ast.parse(.py), tomllib(.toml), shebang-line-1 - make check-render: committed == rendered with fences - Full test suite: 1834 passed, 0 failed; ruff/mypy/pyright 0/0/0 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…e diag) test_missing_end_after_start_is_malformed takes start_tok positionally in the parametrize tuple but does not use it; `del start_tok` satisfies Pylance reportUnusedParameter while keeping pytest's positional injection intact. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Per design correction: fences are an INSTALL-TIME concern owned by the copier, not baked into our own templates. Reverts ST-011's fence injection; copy_managed_file adds the fence at install for watched categories only. Our .claude/.codex trees are now clean again. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The prior commit de-fenced templates_src but the generated trees still carried 166 stale fence markers. Re-rendered so committed trees match the fence-free source. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…rite) fenced=True (default) keeps C2 fence-aware merge (watched files a downstream user may extend below the fence). fenced=False = fully-managed overwrite (inject metadata, .bak on drift, replace whole file) for categories MAP owns. Additive, backward-compatible; JSON unchanged. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…verwrite) file_copier: skills + agents + CLAUDE-side watched (fenced=True); references + map-tools overwrite (fenced=False). Per-file install preserves exec bits; drops shutil.copytree plain-copy. codex_copier: agents/.toml, config.toml, AGENTS.md, skills, hooks/*.py watched; hooks.json JSON-managed; .map/scripts MAP-owned (fenced=False). Threads version through both. Verified in-process: claude+codex double-init fully idempotent (0 .bak, 0 changed); INV-5 (outside-fence survives, inside refreshes, owned overwrite+.bak); exec bits; provider isolation. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Regenerate golden fixtures (claude escalation-matrix.md, codex config.toml) fence-free to match the renderer after ST-011 revert. - Rewrite test_create_map_tools_* for the new owned-overwrite contract: managed scripts refresh in place (copy_managed_file fenced=False); unrelated user files are preserved (no whole-directory rmtree). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Six hand-authored /map-learn entries documenting the C2 fenced-copier work: - architecture: install-time marker double-application - error: cross-clone editable-install, tangled multi-edit recovery, harness-flap output capture - implementation: watched-vs-owned fenced= boolean, preserve +x after atomic temp-file write These live only under .claude/rules/learned/ (repo-local dev artifacts); they are not rendered from templates_src and not shipped to users. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…mitted .claude The old check-render target rendered templates in place then ran 'git checkout -- src/mapify_cli/templates .claude .codex .agents/skills' to restore the tree. That broad checkout reverted ANY uncommitted change under those roots — including hand-authored, NON-rendered files such as .claude/rules/learned/*-patterns.md (invariant D11). Running 'make check' with in-progress /map-learn edits silently destroyed them. Replace it with a non-destructive gate: - add diff_rendered_trees(): renders a provider into a throwaway tempdir and byte-compares only the files the renderer actually produces against the committed trees. Never mutates the working tree; unmanaged D11 files are never in the comparison set. - add a '--check' CLI mode that runs both providers and exits 1 on drift. - check-render now just calls '--check' (no in-place render, no git checkout). Tests: in-sync repo returns clean; drifted/missing gated files are flagged; and a regression guard proves an uncommitted hand-authored learned file is neither flagged nor mutated. Verified empirically: an uncommitted sentinel under .claude/rules/learned/ survives 'make check'. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Phase C2 of the delivery overhaul. Establishes the single-source render pipeline (
templates_src/**/*.jinja→ all generated trees) and a fence-aware managed-file copier that distinguishes watched (user may extend below a fence) from owned (fully overwritten) files. Concludes with the/map-reviewpass that hardened the render gate.What's in here
Single-source render (ST-001 → ST-007)
template_rendererengine with MAP-safe delimiters ([% %],<% %>,[# #]) so Handlebars/bash/type-hints pass through verbatim.make render-templatesrenderstemplates_src/intosrc/mapify_cli/templates/,.claude/,.codex/,.agents/skills/,.map/scripts/.sync-templates, repointed all refs to the renderer.Fence-aware managed-file copier (ST-010 → ST-012)
copy_managed_file(..., *, fenced: bool):fenced=Truepreserves user content below themap:start/map:endfence byte-for-byte (INV-5);fenced=Falsefully overwrites with timestamped.bakon drift.templates_src(installer injects exactly once — avoids double-fence parse fallback); regenerated all trees + fence-free golden fixtures.mapify initwired throughcopy_managed_file(watched vs overwrite).Review hardening (
/map-reviewpass)/map-learnpatterns from the C2 work (.claude/rules/learned/).check-render. The old gate rendered in place thengit checkout -- … .claude …, reverting any uncommitted change under those trees — including hand-authored, non-rendered files (invariant D11, e.g.rules/learned/*-patterns.md). Newdiff_rendered_trees()renders into a throwaway tempdir and byte-compares only rendered files;--checkCLI exits 1 on drift; never mutates the working tree.Testing
make checkgreen: ruff + mypy + pyright (0/0/0) + lint-hooks + 1838 passed, 3 skipped.TestDiffRenderedTrees: in-sync→clean, drifted/missing gated files→flagged, and a regression guard proving an uncommitted D11 file is neither flagged nor reverted..claude/rules/learned/survivesmake check.🤖 Generated with Claude Code