Skip to content

feat: Milestone 0 — project foundation and package skeleton (v0.1.0)#3

Merged
shaypal5 merged 4 commits into
mainfrom
feat/milestone-0-foundation
Apr 18, 2026
Merged

feat: Milestone 0 — project foundation and package skeleton (v0.1.0)#3
shaypal5 merged 4 commits into
mainfrom
feat/milestone-0-foundation

Conversation

@shaypal5

Copy link
Copy Markdown
Contributor

Summary

Bootstraps the leadforge codebase from empty to a fully installable, lint-clean, tested package skeleton. This is the complete Milestone 0 delivery.

All acceptance criteria pass:

  • pip install -e .
  • leadforge --help shows all four commands ✓
  • leadforge list-recipes returns b2b_saas_procurement_v1
  • 20 tests passing ✓
  • ruff + mypy clean ✓
  • CI configured ✓

What's included

Package scaffold

  • pyproject.toml — setuptools build, Typer + PyYAML runtime, ruff/mypy/pytest dev deps, leadforge CLI entry point
  • Full subpackage skeleton (api, cli, core, narrative, schema, structure, mechanisms, simulation, render, exposure, validation, recipes) with __init__.py stubs
  • README.md — install, CLI quickstart, Python API snippet, doc links
  • .pre-commit-config.yaml — ruff (lint + format) + pre-commit-hooks

Core primitives (leadforge/core/)

  • enums.pyExposureMode + DifficultyProfile as StrEnum
  • exceptions.pyLeadforgeError base + 6 typed subclasses
  • models.pyGenerationConfig, WorldSpec, WorldBundle dataclass stubs
  • rng.py, ids.py — documented stubs (implemented in M1)

Recipe system (leadforge/recipes/)

  • registry.pylist_recipes() + load_recipe() reading YAML files
  • b2b_saas_procurement_v1/recipe.yaml — id, title, primary task, modes, difficulty levels

CLI (leadforge/cli/)

  • main.py — Typer app with --version and four registered commands
  • list-recipes — fully implemented with Rich table output
  • generate — stub with full option spec per architecture doc; exits 1 with "coming in v0.2.0"
  • inspect, validate — stubs with correct argument spec

CI (.github/workflows/ci.yml)

  • Three jobs: lint (ruff check + format), typecheck (mypy), test matrix (Python 3.11 + 3.12)
  • Coverage artifacts uploaded under pr-agent-context-coverage-py* prefix for pr-agent-context integration

Tests (20 passing)

  • tests/test_cli.py — help, version, list-recipes content, stub exit codes
  • tests/core/test_enums.py — values + string construction
  • tests/core/test_exceptions.py — hierarchy + message preservation
  • tests/recipes/test_registry.py — list/load, required fields, error case

Test plan

  • CI passes (lint, typecheck, test matrix)
  • leadforge --help shows all four commands
  • leadforge list-recipes shows b2b_saas_procurement_v1
  • leadforge --version prints leadforge 0.1.0

🤖 Generated with Claude Code

Bootstraps the leadforge codebase from empty to a fully installable,
testable, lint-clean package skeleton. All Milestone 0 acceptance
criteria pass: `pip install -e .` works, `leadforge --help` shows all
four commands, `leadforge list-recipes` returns the v1 recipe, and CI
is configured.

Package scaffold
- pyproject.toml: setuptools build, Typer+PyYAML runtime deps, ruff/mypy/
  pytest dev deps, `leadforge` CLI entry point
- Full subpackage skeleton with __init__.py stubs for every module in the
  canonical layout (api, cli, core, narrative, schema, structure,
  mechanisms, simulation, render, exposure, validation, recipes)
- leadforge/version.py: __version__ = "0.1.0"
- README.md: install, quickstart, API snippet, doc links
- .pre-commit-config.yaml: ruff (lint+format) + pre-commit-hooks

Core primitives (leadforge/core/)
- enums.py: ExposureMode (StrEnum), DifficultyProfile (StrEnum)
- exceptions.py: LeadforgeError base + 6 typed subclasses
- models.py: GenerationConfig, WorldSpec, WorldBundle dataclass stubs
- rng.py, ids.py: documented stubs for Milestone 1

Recipe system (leadforge/recipes/)
- registry.py: list_recipes() + load_recipe() reading from YAML files
- b2b_saas_procurement_v1/recipe.yaml: id, title, primary_task,
  supported_modes, supported_difficulty, default_population

CLI (leadforge/cli/)
- main.py: Typer app with --version flag and four registered commands
- commands/list_recipes.py: fully implemented with Rich table output
- commands/generate.py: stub with full option spec (--recipe, --seed,
  --mode, --out, --difficulty, --n-accounts, --n-contacts, --n-leads,
  --horizon-days, --override); exits 1 with "coming in v0.2.0"
- commands/inspect.py, validate.py: stubs with correct argument spec

CI (.github/workflows/ci.yml)
- Three jobs: lint (ruff check+format), typecheck (mypy), test matrix
  (Python 3.11 + 3.12 with pytest-cov + coverage artifact upload for
  pr-agent-context integration)

Tests (20 passing)
- tests/test_cli.py: help, version, list-recipes output, stub exit codes
- tests/core/test_enums.py: values and string construction
- tests/core/test_exceptions.py: hierarchy and message preservation
- tests/recipes/test_registry.py: list/load, required fields, error case

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@shaypal5 shaypal5 added type: feature New capability layer: core core/ primitives (RNG, IDs, models, exceptions) layer: cli cli/ command-line interface layer: recipes recipes/ recipe assets and registry labels Apr 18, 2026
Copilot AI review requested due to automatic review settings April 18, 2026 08:34
@shaypal5 shaypal5 added type: feature New capability layer: core core/ primitives (RNG, IDs, models, exceptions) layer: cli cli/ command-line interface layer: recipes recipes/ recipe assets and registry labels Apr 18, 2026
@github-actions

This comment has been minimized.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Bootstraps the leadforge repository into an installable Python package with a Typer-based CLI, an initial YAML-backed recipe registry, CI/tooling configuration, and a baseline test suite to support future milestones.

Changes:

  • Added project packaging/tooling (pyproject.toml, pre-commit, CI) and a full package/module skeleton.
  • Implemented a first working CLI command (list-recipes) plus stubs for generate, inspect, validate.
  • Added a YAML recipe registry + an initial b2b_saas_procurement_v1 recipe and accompanying tests.

Reviewed changes

Copilot reviewed 23 out of 44 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
tests/test_cli.py CLI smoke tests for help/version and command stubs
tests/recipes/test_registry.py Tests for recipe listing/loading and error case
tests/recipes/init.py Test package marker
tests/core/test_exceptions.py Exception hierarchy tests
tests/core/test_enums.py Enum value/from-string tests
tests/core/init.py Test package marker
tests/init.py Test package marker
pyproject.toml Packaging metadata, deps, ruff/mypy/pytest configuration
leadforge/version.py Defines __version__
leadforge/validation/init.py Subpackage stub
leadforge/structure/init.py Subpackage stub
leadforge/simulation/init.py Subpackage stub
leadforge/schema/init.py Subpackage stub
leadforge/sample_data/public/.gitkeep Placeholder for sample data
leadforge/sample_data/instructor/.gitkeep Placeholder for sample data
leadforge/render/init.py Subpackage stub
leadforge/recipes/registry.py Implements recipe discovery/loading from YAML
leadforge/recipes/b2b_saas_procurement_v1/recipe.yaml Adds initial recipe metadata YAML
leadforge/recipes/b2b_saas_procurement_v1/init.py Subpackage stub
leadforge/recipes/init.py Subpackage stub
leadforge/narrative/init.py Subpackage stub
leadforge/mechanisms/init.py Subpackage stub
leadforge/exposure/init.py Subpackage stub
leadforge/examples/notebooks/.gitkeep Placeholder for examples
leadforge/examples/configs/.gitkeep Placeholder for examples
leadforge/core/rng.py RNG utilities stub docstring
leadforge/core/models.py Dataclass stubs for config/spec/bundle
leadforge/core/ids.py ID scheme stub docstring
leadforge/core/exceptions.py Defines project exception hierarchy
leadforge/core/enums.py Defines ExposureMode / DifficultyProfile
leadforge/core/init.py Core subpackage stub
leadforge/cli/main.py Typer app entrypoint + command registration
leadforge/cli/commands/validate.py validate command stub
leadforge/cli/commands/list_recipes.py list-recipes command implementation (Rich table)
leadforge/cli/commands/inspect.py inspect command stub
leadforge/cli/commands/generate.py generate command stub + option surface
leadforge/cli/commands/init.py Commands subpackage stub
leadforge/cli/init.py CLI subpackage stub
leadforge/api/init.py API subpackage stub
leadforge/init.py Package init exporting __version__
README.md Project README with install/quickstart/docs links
.pre-commit-config.yaml Pre-commit hooks (ruff, formatting, basic checks)
.github/workflows/ci.yml CI for lint/typecheck/tests with coverage artifacts
.agent-plan.md Updates project plan to mark M0 complete / M1 next

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread README.md
Comment thread leadforge/recipes/registry.py Outdated
Comment thread leadforge/recipes/registry.py Outdated
Comment thread leadforge/recipes/registry.py Outdated
Comment thread README.md Outdated
@github-actions

This comment has been minimized.

@shaypal5 shaypal5 self-assigned this Apr 18, 2026
COPILOT-1 — Add Generator stub to leadforge/api
- leadforge/api/generator.py: Generator class with from_recipe() and
  generate() raising NotImplementedError with "coming in v0.2.0" messages
- leadforge/api/__init__.py: export Generator so `from leadforge.api
  import Generator` resolves correctly

COPILOT-2 — Sort list_recipes() by recipe id field, not path
- return sorted(recipes, key=lambda r: r["id"]) instead of relying on
  filesystem iteration order

COPILOT-3 — Validate yaml.safe_load() result in registry
- Extract _parse_and_validate() helper; raises InvalidRecipeError if the
  parsed value is not a dict or is missing the required 'id' key; used
  by both list_recipes() and load_recipe()

COPILOT-4 — Guard load_recipe() against path traversal
- Resolve the candidate path and verify it stays within _RECIPES_DIR
  before checking existence or opening; raises InvalidRecipeError for
  any recipe_id that would escape the recipes directory

COPILOT-5 — Comment out unimplemented CLI commands in README
- Quickstart now shows generate/inspect/validate as commented-out
  examples with "Coming in v0.x.0" labels; only `list-recipes` is shown
  as immediately runnable
- Python API snippet annotated with "(coming in v0.2.0)"

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@github-actions

This comment has been minimized.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 25 out of 45 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread leadforge/recipes/registry.py Outdated
Comment thread pyproject.toml Outdated
…mits

Documents the mandatory step of resolving GitHub review threads via
GraphQL after addressing PR comments, so the omission from PR #3
does not recur.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@github-actions

This comment has been minimized.

- registry.py: replace string-prefix path traversal guard with
  Path.is_relative_to() (Python 3.11+), closing the prefix-collision
  bypass (e.g. recipes_evil alongside recipes)
- pyproject.toml: add "S" (bandit) ruleset to ruff select so security
  checks are active on non-test code; widen per-file-ignores glob from
  tests/* to tests/**/* to cover subdirectories; add S108 to test
  ignores to suppress the /tmp false-positive in test CLI invocations

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings April 18, 2026 14:07
@github-actions

Copy link
Copy Markdown

pr-agent-context report:

This run includes unresolved review comments on PR #3.

For each unresolved review comment, recommend one of: resolve as irrelevant, accept and implement
the recommended solution, open a separate issue and resolve as out-of-scope for this PR, accept and
implement a different solution, or resolve as already treated by the code.

After I reply with my decision per item, implement the accepted actions, resolve the corresponding
PR comments, and push all of these changes in a single commit.

# Copilot Comments

## COPILOT-1
Location: leadforge/recipes/registry.py
URL: https://github.com/leadforge-dev/leadforge/pull/3#discussion_r3105050071
Status: outdated
Root author: copilot-pull-request-reviewer

Comment:
    The path traversal guard is bypassable because it uses a string prefix check. For example, a resolved path like `/.../recipes_evil` will still start with `/.../recipes`, so `recipe_id='../recipes_evil'` can incorrectly pass this check. Use `Path.is_relative_to()` (Py3.11+) or `recipe_dir.relative_to(base_dir)` in a try/except to ensure the resolved path is actually within `_RECIPES_DIR`.
    ~~~suggestion
        base_dir = _RECIPES_DIR.resolve()
        recipe_dir = (base_dir / recipe_id).resolve()
        try:
            recipe_dir.relative_to(base_dir)
        except ValueError:
    ~~~

## COPILOT-2
Location: pyproject.toml
URL: https://github.com/leadforge-dev/leadforge/pull/3#discussion_r3105050075
Status: outdated
Root author: copilot-pull-request-reviewer

Comment:
    The Ruff per-file-ignores glob `tests/*` only matches files directly under `tests/` and won’t apply to tests in subdirectories like `tests/core/...` or `tests/recipes/...`. If the intent is to ignore rules across the whole test tree, use `tests/**` (or similar). Also, `S101` is currently a no-op unless the `S` ruleset is enabled in `select`.

Run metadata:

Tool ref: v4
Tool version: 4.0.18
Trigger: commit pushed
Workflow run: 24606387604 attempt 1
Comment timestamp: 2026-04-18T14:07:41.401806+00:00
PR head commit: ac22ad52c0c25ff581cb8af14d2a97de9b664c63

@shaypal5 shaypal5 merged commit da98878 into main Apr 18, 2026
9 checks passed
@shaypal5 shaypal5 deleted the feat/milestone-0-foundation branch April 18, 2026 14:10

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 26 out of 46 changed files in this pull request and generated 3 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread leadforge/recipes/registry.py
Comment thread tests/test_cli.py
Comment thread tests/test_cli.py
shaypal5 added a commit that referenced this pull request May 1, 2026
Review feedback addressed:

- Remove primary_task/label_window_days as explicit kwargs from
  resolve_config() and Generator.from_recipe() — these fields are
  resolved from recipe YAML and override dict only, not casually
  overridable, since the generation pipeline doesn't yet support
  arbitrary task types (Copilot-1, Copilot-3, shaypal5 #1, #2)
- Add label_window_days <= horizon_days validation in
  GenerationConfig.__post_init__ (Copilot-2, shaypal5 #3)
- Add tests for invalid primary_task on GenerationConfig: empty
  string, non-string type (shaypal5 #6, pr-agent-context)
- Add tests for invalid label_window_days on Recipe.from_dict: bool,
  non-positive, float (shaypal5 #7, pr-agent-context)
- Add test for label_window_days > horizon_days rejection
- Fix existing test using horizon_days=30 (now conflicts with default
  label_window_days=90)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
shaypal5 added a commit that referenced this pull request May 1, 2026
* feat: carry primary_task and label_window_days into WorldSpec for dataset card

Add `primary_task` and `label_window_days` fields to `GenerationConfig`
(with defaults preserving current behavior). Propagate through
`Recipe.from_dict()`, `resolve_config()`, and `Generator.from_recipe()`
so recipe YAML can override them. Update `render_dataset_card()` to read
from `world_spec.config` instead of hard-coded string literals.

Closes #6

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: update .agent-plan.md for WorldSpec task fields (PR #36)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address review feedback — tighten scope, add validation + tests

Review feedback addressed:

- Remove primary_task/label_window_days as explicit kwargs from
  resolve_config() and Generator.from_recipe() — these fields are
  resolved from recipe YAML and override dict only, not casually
  overridable, since the generation pipeline doesn't yet support
  arbitrary task types (Copilot-1, Copilot-3, shaypal5 #1, #2)
- Add label_window_days <= horizon_days validation in
  GenerationConfig.__post_init__ (Copilot-2, shaypal5 #3)
- Add tests for invalid primary_task on GenerationConfig: empty
  string, non-string type (shaypal5 #6, pr-agent-context)
- Add tests for invalid label_window_days on Recipe.from_dict: bool,
  non-positive, float (shaypal5 #7, pr-agent-context)
- Add test for label_window_days > horizon_days rejection
- Fix existing test using horizon_days=30 (now conflicts with default
  label_window_days=90)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
shaypal5 added a commit that referenced this pull request May 6, 2026
Fold the brutal self-review's findings back into the PR before review.

Bugs:
- (#1) run_packager validate→write order — both packagers wrote
  README/metadata on validation failure, leaving corrupt artifacts on
  disk that would silently get committed.  Gated on `errors == ()`;
  added no-write tests for both packagers.
- (#2) Instructor README inlined the public 3-tier README into a
  1-tier dataset card.  Replaced with a dedicated `INSTRUCTOR_BODY`
  constant that links to the public dataset and describes only the
  instructor-specific additions (full-horizon tables, hidden DAG,
  latent registry, mechanism summary).
- (#3) validate_upload_dir_safe also blocks strict descendants of
  release_dir; `--huggingface-dir release/intro` would otherwise
  rmtree the intro bundle.

Architecture:
- (#5) Finished shared-primitives extraction: SOURCE_TREE_BLOCK,
  validate_readme_substitution, replace_file, replace_dir,
  load_manifest now live in scripts/_release_common.py.  Both
  packagers reduced to imports.
- (#6) Replaced 60-line hand-rolled YAML renderer with yaml.safe_dump
  + a 4-line _IndentedDumper subclass.
- (#7) Removed dead --owner / --dataset-slug CLI flags.
- (#8) assemble_upload_dir now takes rendered_readme and writes it.
- (#9) build_config_for_tier made pure (no I/O); cheap manifest-stat
  preflight via _assert_tier_dir_exists.
- (#10) --default-config with --variant=instructor errors loudly.

CI:
- (#4) Added [publish] extra (datasets>=2.14, kaggle>=1.6) so the
  gated G12.3 / G12.4 / G11.3 tests install in one line.

Cleanups: visual cruft (#13#16), test cruft (#17 — unused tmp_path,
dead tag_lines), em-dash YAML round-trip parametrised for the
instructor pretty_name.

Verification: 1223 tests pass + 5 gated skips; ruff + mypy clean;
hash determinism PASS 67/67; leakage probes 0/3 reconstruct on every
tier; validate_release_candidate --no-rebuild exits 0.
release/{kaggle,huggingface,huggingface-instructor}/dataset-metadata
.json|README.md regenerated; audit-artifact-sync tests guard them.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
shaypal5 added a commit that referenced this pull request May 6, 2026
* PR 5.2: HuggingFace release packager + load_dataset smoke test

Add `scripts/package_hf_release.py` to generate `release/huggingface/README.md`
with G12.1-compliant YAML frontmatter (pretty_name, license, language,
task_categories, size_categories, tags, three configs with `default: true`
on intermediate per G12.2), inlining the rewritten `release/README.md`
body with HF-specific link rewrites.  `--variant=instructor` packages the
companion repo (G12.4) from `release/intermediate_instructor/` into a
separate `release/huggingface-instructor/` upload tree.  G12.3 covered
by a parametrised `load_dataset()` smoke test gated on the optional
`datasets` SDK.

Extract shared release-packaging primitives (link rewriter, dir-safety
guard, cover-image validator) into `scripts/_release_common.py`; refactor
the Kaggle packager to import them.  `release/kaggle/dataset-metadata.json`
is byte-stable across the refactor.

Delete the legacy `release/HF_DATASET_CARD.md` stub — superseded by the
generated card.  Gitignore `release/huggingface{,-instructor}/*` except
the committed README.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* PR 5.2 self-review fixes (Kaggle + HF packagers)

Fold the brutal self-review's findings back into the PR before review.

Bugs:
- (#1) run_packager validate→write order — both packagers wrote
  README/metadata on validation failure, leaving corrupt artifacts on
  disk that would silently get committed.  Gated on `errors == ()`;
  added no-write tests for both packagers.
- (#2) Instructor README inlined the public 3-tier README into a
  1-tier dataset card.  Replaced with a dedicated `INSTRUCTOR_BODY`
  constant that links to the public dataset and describes only the
  instructor-specific additions (full-horizon tables, hidden DAG,
  latent registry, mechanism summary).
- (#3) validate_upload_dir_safe also blocks strict descendants of
  release_dir; `--huggingface-dir release/intro` would otherwise
  rmtree the intro bundle.

Architecture:
- (#5) Finished shared-primitives extraction: SOURCE_TREE_BLOCK,
  validate_readme_substitution, replace_file, replace_dir,
  load_manifest now live in scripts/_release_common.py.  Both
  packagers reduced to imports.
- (#6) Replaced 60-line hand-rolled YAML renderer with yaml.safe_dump
  + a 4-line _IndentedDumper subclass.
- (#7) Removed dead --owner / --dataset-slug CLI flags.
- (#8) assemble_upload_dir now takes rendered_readme and writes it.
- (#9) build_config_for_tier made pure (no I/O); cheap manifest-stat
  preflight via _assert_tier_dir_exists.
- (#10) --default-config with --variant=instructor errors loudly.

CI:
- (#4) Added [publish] extra (datasets>=2.14, kaggle>=1.6) so the
  gated G12.3 / G12.4 / G11.3 tests install in one line.

Cleanups: visual cruft (#13#16), test cruft (#17 — unused tmp_path,
dead tag_lines), em-dash YAML round-trip parametrised for the
instructor pretty_name.

Verification: 1223 tests pass + 5 gated skips; ruff + mypy clean;
hash determinism PASS 67/67; leakage probes 0/3 reconstruct on every
tier; validate_release_candidate --no-rebuild exits 0.
release/{kaggle,huggingface,huggingface-instructor}/dataset-metadata
.json|README.md regenerated; audit-artifact-sync tests guard them.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* PR 5.2 Copilot-review fixes (Kaggle + HF packagers)

Fold Copilot's two real findings on the self-review revision back in.

COPILOT-1 — validate_upload_dir_safe was only invoked inside
assemble_upload_dir, which --dry-run skips.  A dry-run with
--huggingface-dir release (or .) would write the README into the
unsafe path BEFORE the safety net fired.  Hoist the check into
run_packager (both packagers) so it runs before any mkdir or write;
the inner assemble_upload_dir call stays as defence-in-depth for
direct callers.  New tests: dry-run with unsafe upload-dir raises
without writing; the same path through main() returns rc=2.

COPILOT-2 — Cover-image path resolution was inconsistent:
validate_cover_image used cover_image as passed, while
assemble_upload_dir did a separate ``release_dir / cover_image.name``
fallback.  Diverged for bare-basename inputs (false validation
failures) and two-paths-sharing-a-basename (assembler shadowing the
explicit path).  Added resolve_cover_image_path() to
_release_common.py (explicit-wins, release-dir fallback);
run_packager calls it once and threads the resolved path through
validation, the metadata's image field, and assembly.  New
tests/scripts/test_release_common.py covers the four resolution
branches; new packager-side tests confirm bare-basename success +
metadata field plumbing.

COPILOT-3 — outdated; already addressed by self-review fix #8 in
commit f2fc4a2.  Resolved as already treated; no code change.

Verification: 1232/1232 tests pass + 5 gated skips; ruff + mypy
clean; hash determinism PASS 67/67; leakage probes rc=0 on every
tier; validate_release_candidate --no-rebuild exits 0;
BUNDLE_SCHEMA_VERSION unchanged at 5.
release/{kaggle,huggingface,huggingface-instructor}/* artifacts
regenerated byte-identically.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
shaypal5 added a commit that referenced this pull request Jun 12, 2026
…1] (#121)

* refactor(render): scheme-agnostic build_manifest + schema v6 [LTV-Pn.1]

First sub-PR of the split LTV-Pn (M6). Decouples the manifest builder from
the lead-scoring scheme so the lifecycle scheme can reuse it, and records
which scheme produced each bundle.

- build_manifest no longer takes the lead-scoring `world_graph`. It takes
  `generation_scheme: str` (required), `motif_family: str | None = None`,
  and an `extra_fields` mapping for scheme-specific top-level keys (the
  lifecycle scheme will add `observation_date` / forward windows). A
  collision guard rejects extra_fields that would clobber a core key.
- Every manifest now records `generation_scheme` (lead_scoring / lifecycle).
- BUNDLE_SCHEMA_VERSION 5 -> 6 (history note added). The lead-scoring
  published *shape* is unchanged: tables/ and tasks/ parquet files are
  byte-identical to v5 (verified via pinned-timestamp SHA-256 across the
  full instructor bundle); only manifest.json changes (new field + version).
- Removes the render.manifests -> lead_scoring.structure.graph
  TYPE_CHECKING back-reference (partial discharge of carried cleanup #3;
  the core.models.WorldBundle back-refs follow in Pn.2).
- Schema-contract gate renamed test_bundle_schema_v5_contract.py ->
  _v6_, asserts version "6", and adds a generation_scheme assertion. The
  pinned column/table sets carry over verbatim (shape unchanged).

Callers updated: lead_scoring.write_bundle passes
generation_scheme=self.name, motif_family=world_graph.motif_family.

Plan: docs/ltv/roadmap.md records the four-way LTV-Pn split (Pn.1…4) and
marks the cleanup-#3 partial discharge.

Tests: new tests/render/test_manifest_scheme_agnostic.py (generation_scheme
recorded, motif_family default/passthrough, extra_fields merge + collision
guard, required-arg); v6 contract test. Full suite 1801 passed / 51
skipped; ruff + mypy clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(cli): surface generation_scheme in inspect; honest fixtures [LTV-Pn.1]

Self-review findings on the manifest-generalization PR.

1. Verification gap (mine): the byte-identity proof covered only the
   research_instructor bundle. Confirmed the student_public path — which
   takes the snapshot-safe relational route and also calls build_manifest —
   is unchanged and deterministic, and carries generation_scheme + v6.
   No code change.

2. The inspect CLI is the user-facing manifest viewer but ignored the new
   first-class generation_scheme field. Added a conditional "Scheme:" row
   (conditional so pre-v6 bundles still render without a "?" placeholder,
   matching the existing v3+/v4+ field handling). Also made the
   motif_family row null-safe: a lifecycle bundle records motif_family=null,
   which .get(..., '?') would render as the literal "None"; now shows "?".

3. Bumped four release/critique test fixtures from a hardcoded
   bundle_schema_version "5" to "6". They are inert (nothing compares them
   to the current version, which is why the suite stayed green), but a
   fixture claiming "5" while the package emits "6" is misleading.

Tests: inspect-shows-generation-scheme CLI test; the header-order
regression guard still holds (the new row sits between Recipe and Seed
without reordering the pinned eight). Full suite 1802 passed / 51 skipped;
ruff + mypy clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* docs(test): fix stale v5 docstring in snapshot-safe integration test [LTV-Pn.1]

Copilot review on #121: the module docstring opening line still said
"bundle schema v5" while line 9 (updated in this PR) references v6. Make the
version consistent.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
shaypal5 added a commit that referenced this pull request Jun 13, 2026
…n.2] (#122)

* refactor: scheme-agnostic WorldBundle + exposure metadata hook [LTV-Pn.2]

Second sub-PR of the split LTV-Pn. Removes the last core/shared-layer
references to the lead-scoring scheme (carried cleanups #2 and #3), so the
lifecycle scheme can plug into the same envelope.

WorldBundle (cleanup #3):
- Replaced the three lead-scoring-typed fields (population / simulation_result
  / world_graph) with a single opaque `artifacts: Any`. Each scheme defines
  and unwraps its own container; lead-scoring adds LeadScoringArtifacts
  (schemes/lead_scoring/artifacts.py). core.models no longer imports any
  lead_scoring type — the layering inversion introduced in LTV-Pf.1 is gone.

Exposure (cleanup #2):
- apply_exposure is now scheme-agnostic: it writes the generic, spec-only
  world_spec.json (kept in exposure/metadata.py as write_world_spec_json) and
  dispatches the scheme-specific hidden-truth files to a new
  GenerationScheme.write_metadata(bundle, meta_dir) hook, resolved from
  bundle.spec.scheme via the registry. The lead-scoring graph / latent
  registry / mechanism-summary writers moved out of exposure/ into
  LeadScoringScheme.write_metadata. exposure/ no longer references lead_scoring.
- Protocol gains write_metadata; the lifecycle stub implements it (raises
  NotImplementedError until Pn.4).

Byte-identity: the full lead-scoring bundle is byte-identical across BOTH
exposure modes (research_instructor 21 files, student_public 14) — verified
against a pre-refactor SHA-256 reference. The metadata writers moved, not
changed; world_spec.json content/order is preserved.

Re-scope: the shared bundle orchestrator (cleanup #1) moves from this PR to
LTV-Pn.4. Per the roadmap's own note it is best designed with the second
scheme's write_bundle in hand; extracting it now against one scheme would
guess the hook shape. Roadmap + deferred-cleanups updated (#2, #3 done; #1 →
Pn.4).

Tests: new tests/schemes/test_scheme_metadata_hook.py (artifacts populated;
WorldBundle has only spec+artifacts; generic world_spec writer; lead-scoring
hook emits the 4 hidden-truth files; unpopulated-bundle + lifecycle-stub
raise). Updated field-access sites in 5 test modules. Full suite 1808 passed /
51 skipped; ruff + mypy clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(scripts): migrate build_v*_snapshot to bundle.artifacts [LTV-Pn.2]

Self-review finding on the WorldBundle generalization: five dataset-build
scripts still read the removed bundle.simulation_result / bundle.population
fields and would AttributeError at runtime.

This slipped both nets:
- mypy leadforge does not type-check scripts/, and
- the existing tests/scripts/test_build_v*_snapshot.py cover only the
  pipelines.build_v* transform helpers (pure DataFrame functions), never the
  generate_bundle() entry point that touches the bundle.
CI's "Validate v6/v7" jobs validate a pre-built CSV; they do not regenerate,
so they would not have caught it either.

Fix: build_v4/v5/v6/v7_snapshot.py and build_midproject_lead_scoring.py now
read bundle.artifacts.simulation_result / bundle.artifacts.population. Smoke-
ran the v6 builder end-to-end through Generator.generate() to confirm.

Guard: tests/scripts/test_build_v6_snapshot.py gains
TestGenerateBundleArtifactsPath, which loads the script via importlib and runs
generate_bundle(small) so a future WorldBundle field rename can't silently
break the generate path again. The other four scripts share the identical
access pattern.

Full suite 1809 passed / 51 skipped; ruff + mypy clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(exposure): clear metadata/ before rewrite; fix stale docstring [LTV-Pn.2]

Two Copilot review findings on #122.

1. apply_exposure reused an existing metadata/ via mkdir(exist_ok=True) when
   writing hidden truth, so a reused output path could retain stale files.
   Pre-existing behavior, but Pn.2 makes it newly dangerous: once the
   lifecycle scheme writes a different hidden-truth file set, regenerating a
   lifecycle bundle over a path that previously held a lead-scoring bundle
   would orphan graph.graphml / mechanism_summary.json into the new bundle.
   Now apply_exposure always removes any existing metadata/ first, then
   recreates it when writing — so contents exactly match the current bundle
   (mirroring the non-writing branch, which already rmtree'd it). Byte-identity
   preserved for both modes (fresh paths have no metadata/, so the rmtree is a
   guarded no-op). Regression tests: a pre-seeded stale file is gone after an
   instructor rewrite; student_public still removes the dir entirely.

2. The lead_scoring.write_bundle docstring still described apply_exposure as
   writing the lead-scoring hidden graph + latent registry directly. Updated
   to reflect the Pn.1/Pn.2 reality: build_manifest and apply_exposure are
   scheme-agnostic, and hidden truth is delegated to write_metadata; the
   remaining shared-orchestrator extraction is deferred to Pn.4.

Full suite 1811 passed / 51 skipped; ruff + mypy clean.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

layer: cli cli/ command-line interface layer: core core/ primitives (RNG, IDs, models, exceptions) layer: recipes recipes/ recipe assets and registry type: feature New capability

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants