feat(lifecycle): weekly simulation engine [LTV-Pk]#118
Merged
Conversation
Second half of LTV-M4 — the weekly lifecycle simulator that turns a customer
population into the three event tables plus terminal subscription state.
leadforge/schemes/lifecycle/engine.py:
- simulate_lifecycle(population, seed, *, forward_window_days=730,
early_tenure_weeks=4) → LifecycleSimulationResult{subscriptions,
subscription_events, health_signals, invoices}.
- Fully simulated target windows (D6): each customer runs through
max(observation_date, start + early_tenure_weeks) + forward_window_days, so
all 90/365/730d pLTV targets are complete for BOTH observation regimes.
- Per-customer RNG substreams (lifecycle_sim::<customer_id>): one customer's
trajectory is invariant to every other customer — stronger stability than
the lead-scoring shared streams, and provable in a test (solo-resimulation
equals the full-population trajectory).
- Fixed weekly step order: health signal → invoice/dunning → churn draw →
renewal event → expansion draw. Causal churn reasons: payment_failure
(write-off), non_renewal (spiked draw on anniversary week), voluntary.
- Health signals: weekly active_users (plan-seat base x adoption x onboarding
usage ramp), feature_depth_score (latent plateau x ramp — feeds the same
week's expansion hazard, creating latents → health → expansion causality),
Knuth-Poisson support tickets (fit-driven), quarterly NPS (null off-cycle).
- Invoices on month boundaries (12/52 weeks) at current MRR; failures enter
dunning and resolve to recovered or written_off → forced churn.
- subscription_id derived from the customer index up front and threaded into
every event (no back-fill pass).
leadforge/schemes/lifecycle/population.py:
- CustomerPopulationResult records motif_family so the engine fetches the same
family's mechanism params (passing it separately would invite silent drift).
leadforge/schemes/lifecycle/mechanisms.py — ENGINE CALIBRATION (discharges the
obligation recorded in the #117 review):
- Measured simulated first-year churn on motif-biased populations (n=600,
3 seed pairs) and retuned churn base rates, payment-failure rates, and
recovery rates over three rounds. Before: payment_fragile 80.5%,
churner_dominated 66.8%. After: product_led ~19-21%, relationship ~23-27%,
expansion_led ~13-18%, payment_fragile ~35-39%, churner_dominated ~41-44% —
matching the per-motif intent while honouring the Pi directional-test
constraints (fragile failure > 2x others, fragile recovery strictly lowest).
- Calibration comment rewritten from "expected to be tuned DOWN" to
ENGINE-CALIBRATED with the measured targets and a pointer to the guard test.
tests/schemes/lifecycle/test_engine.py (25 tests): shape/validation,
byte-equal determinism across all four tables, per-customer independence
(solo-resimulation), weekly health cadence, monthly invoice cadence (±1),
quarterly-only NPS, full-window coverage for active customers, event FK
integrity, churn-state consistency + no-events-after-churn, expansion MRR
chain reconciliation (events chain to subscription.current_mrr and
expansion_count), renewal-events-only-on-anniversaries + renewal_count
reconciliation, dunning resolution / write-off → payment_failure churn,
per-motif year-1 churn bands, expansion-world dominance, majority-active-at-
observation sanity.
Engine docstring records the two tracked exclusions: downgrade events (no
mechanism params yet — downgrade_count would be zero-variance; revisit before
LTV-M5 ships the feature) and difficulty-tier scaling (LTV-M6).
Full suite 1712 passed / 51 skipped; ruff + mypy clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
shaypal5
added
type: feature
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This comment has been minimized.
This comment has been minimized.
There was a problem hiding this comment.
Pull request overview
Implements the weekly lifecycle simulation engine (simulate_lifecycle) that evolves each customer from their staggered start date through the required forward windows, drawing against the lifecycle hazard functions and emitting the lifecycle event tables plus a terminal subscription row per customer. This completes the “engine” portion of the LTV lifecycle milestone and adds a dedicated test suite to lock down determinism, cadences, and calibration bands.
Changes:
- Added
leadforge.schemes.lifecycle.engine.simulate_lifecycle()with per-customer RNG substreams, weekly step ordering, and emission ofsubscription_events,health_signals, andinvoices. - Calibrated lifecycle mechanism base rates and added engine-level calibration/consistency tests.
- Recorded
motif_familyon generated populations so the engine uses the matching parameter family.
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/schemes/lifecycle/test_engine.py | Adds comprehensive engine tests (determinism, cadences, integrity checks, calibration bands). |
| leadforge/schemes/lifecycle/population.py | Persists motif_family on CustomerPopulationResult and populates it in the builder. |
| leadforge/schemes/lifecycle/mechanisms.py | Updates calibration commentary and retunes churn/payment parameters to match engine-calibrated targets. |
| leadforge/schemes/lifecycle/engine.py | Introduces the weekly simulator, event emission, health/invoice generation, and churn/renewal/expansion logic. |
| docs/ltv/roadmap.md | Updates roadmap references to include PR #118 and marks #117 complete. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| nps_score=nps, | ||
| ) | ||
| ) | ||
| return depth |
Comment on lines
+287
to
+291
| month_index = int(week / _WEEKS_PER_MONTH) | ||
| if month_index > last_month_index: | ||
| last_month_index = month_index | ||
| counters["invoice"] += 1 | ||
| failed = rng.random() < payment_failure_probability(payment_p, latents) |
Comment on lines
+299
to
+302
| result.invoices.append(invoice) | ||
| if failed and state.pending_failure is None: | ||
| state.pending_failure = (invoice, week) | ||
| _emit_event( |
Comment on lines
+344
to
+346
| # -- 3. Churn draw --------------------------------------------------- | ||
| renewal_week = is_renewal_week(week, state.contract_term_months) | ||
| p_churn = churn_probability(churn_p, latents, week, state.contract_term_months) |
Comment on lines
+332
to
+342
| _emit_event( | ||
| result, | ||
| counters, | ||
| subscription_id=subscription_id, | ||
| customer_id=customer.customer_id, | ||
| week_date=week_date, | ||
| event_type="churn", | ||
| mrr_before=state.current_mrr, | ||
| mrr_after=0, | ||
| ) | ||
| break |
Comment on lines
+349
to
+359
| _emit_event( | ||
| result, | ||
| counters, | ||
| subscription_id=subscription_id, | ||
| customer_id=customer.customer_id, | ||
| week_date=week_date, | ||
| event_type="churn", | ||
| mrr_before=state.current_mrr, | ||
| mrr_after=0, | ||
| ) | ||
| break |
Comment on lines
+257
to
+267
| def test_payment_failures_resolve_or_censor(population, sim) -> None: | ||
| # Every failed invoice ends as recovered / written_off, unless the customer | ||
| # churned (or the window ended) before the dunning period elapsed. | ||
| churned = {s.customer_id: s for s in sim.subscriptions if s.churn_at} | ||
| for inv in sim.invoices: | ||
| assert inv.payment_status in ("paid", "failed", "recovered", "written_off") | ||
| if inv.payment_status == "written_off": | ||
| sub = churned.get(inv.customer_id) | ||
| assert sub is not None | ||
| assert sub.churn_reason == "payment_failure" | ||
|
|
Comment on lines
+225
to
+240
| def test_mrr_chain_consistency(population, sim) -> None: | ||
| initial = {c.customer_id: c.initial_mrr for c in population.customers} | ||
| expansions: dict[str, list] = {} | ||
| for e in sim.subscription_events: | ||
| if e.event_type == "expansion": | ||
| expansions.setdefault(e.customer_id, []).append(e) | ||
| assert e.mrr_after > e.mrr_before | ||
| for sub in sim.subscriptions: | ||
| chain = expansions.get(sub.customer_id, []) | ||
| mrr = initial[sub.customer_id] | ||
| for e in chain: | ||
| assert e.mrr_before == mrr | ||
| mrr = e.mrr_after | ||
| assert sub.current_mrr == mrr | ||
| assert sub.expansion_count == len(chain) | ||
|
|
Three findings from hostile self-review of the initial engine commit: 1. ACCOUNT LATENTS WERE COMPLETELY DEAD (the real one). The customer latent dict contains every account latent key (latent_budget_stability, latent_organizational_stability) plus three more, and the merge let customer values shadow account values — so every account-level draw was discarded for every customer, and the comment called the collision "(deliberate)". This also destroyed the within-account correlation that account-level draws exist to provide (~3 customers share an account). Fix: explicit _merge_latents helper that blends shared traits 50/50 — the account component is a shared random effect, giving correlated churn and payment behaviour within an account (mixed-effects structure). Calibration re-verified after the change: all five motifs stay inside their year-1 churn bands across three seed pairs. Regression test: swinging account latent_budget_stability 0.0 vs 1.0 must change the population's failed-invoice count. 2. Intra-week ordering: invoices were issued BEFORE pending dunning resolved. Consequences: (a) a customer's write-off churn week could include a fresh same-week invoice (whose paid amount would count toward pLTV revenue); (b) for dunning_weeks=4 motifs with a 4-week month gap, a second invoice could fail while one was pending and be silently dropped — terminal "failed" status forever, no event, no dunning. Fix: dunning resolution now runs before invoice issuance, so the pending slot is always free by issuance time. The only remaining "failed" terminal states are genuine censoring (churn for another reason mid-dunning, or window end) — now documented in the module docstring and pinned by a test that checks every dangling "failed" invoice against exactly those two conditions. 3. The recorded feature_depth_score was rounded to 4dp but the UNROUNDED value fed the expansion hazard — the published observable differed (by ε) from the value that drove behaviour, breaking the data↔causality equivalence this dataset exists to teach. The hazard now consumes the exact rounded value the row records; round-trip test added. Full suite 1715 passed / 51 skipped; ruff + mypy clean; calibration bands re-verified post-blend. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
pr-agent-context report: This run includes unresolved review comments on PR #118 in repository https://github.com/leadforge-dev/leadforge
For each unresolved review comment, recommend one of: resolve as irrelevant, accept and implement
the recommended solution, open a separate issue and resolve as out-of-scope for this PR, accept and
implement a different solution, or resolve as already treated by the code.
After I reply with my decision per item, implement the accepted actions, resolve the corresponding
PR comments, and push all of these changes in a single commit.
# Copilot Comments
## COPILOT-1
Location: leadforge/schemes/lifecycle/engine.py:517
URL: https://github.com/leadforge-dev/leadforge/pull/118#discussion_r3402279614
Root author: copilot-pull-request-reviewer
Comment:
The expansion hazard is supposed to be driven by the same week’s `feature_depth_score`, but `_emit_health_signal()` returns the unrounded `depth` while persisting `round(depth, 4)` in the table. This makes the stored health signal slightly inconsistent with the value used for expansion draws (and can break attempts to replay/inspect hazard inputs from the emitted tables).
## COPILOT-2
Location: leadforge/schemes/lifecycle/engine.py
URL: https://github.com/leadforge-dev/leadforge/pull/118#discussion_r3402279660
Status: outdated
Root author: copilot-pull-request-reviewer
Comment:
If an invoice fails in the same week that a prior pending failure is recovered (dunning resolution), `state.pending_failure` is cleared later in the loop and the new failed invoice is left permanently in `payment_status == "failed"` with no dunning resolution path. Track the newly created invoice so you can re-promote it to `pending_failure` after clearing the prior one.
## COPILOT-3
Location: leadforge/schemes/lifecycle/engine.py
URL: https://github.com/leadforge-dev/leadforge/pull/118#discussion_r3402279679
Status: outdated
Root author: copilot-pull-request-reviewer
Comment:
`new_invoice` needs to be set when an invoice is created so later logic can detect the “failed invoice created during recovery week” case and attach dunning to it.
## COPILOT-4
Location: leadforge/schemes/lifecycle/engine.py:374
URL: https://github.com/leadforge-dev/leadforge/pull/118#discussion_r3402279705
Status: outdated
Root author: copilot-pull-request-reviewer
Comment:
After recovering a prior pending failure, if the invoice created this same week also failed, it should immediately become the new `pending_failure` (and emit a `payment_failure` event). Otherwise that invoice will remain stuck in `failed` forever with no recovery/write-off outcome.
## COPILOT-5
Location: leadforge/schemes/lifecycle/engine.py:344
URL: https://github.com/leadforge-dev/leadforge/pull/118#discussion_r3402279729
Root author: copilot-pull-request-reviewer
Comment:
When churn happens, the churn event records `mrr_after=0` but the terminal subscription row will still carry the pre-churn `current_mrr`, which makes the terminal state internally inconsistent ("churned" subscriptions still have positive MRR). Set `state.current_mrr = 0` when churn is triggered so the subscription row reflects the post-churn terminal state.
## COPILOT-6
Location: leadforge/schemes/lifecycle/engine.py:387
URL: https://github.com/leadforge-dev/leadforge/pull/118#discussion_r3402279759
Root author: copilot-pull-request-reviewer
Comment:
Same as the payment-failure churn path: after emitting a churn event with `mrr_after=0`, the simulated terminal subscription should have `current_mrr == 0` so the output tables are consistent.
## COPILOT-7
Location: tests/schemes/lifecycle/test_engine.py:267
URL: https://github.com/leadforge-dev/leadforge/pull/118#discussion_r3402279786
Root author: copilot-pull-request-reviewer
Comment:
This test is named/commented as if it asserts that failed invoices "resolve or censor", but it currently only checks that `payment_status` is in an allowed set and that `written_off` implies payment-failure churn. As written, it will not catch failures that remain stuck as `"failed"` long past the dunning window (including the engine edge case where a new failure occurs on a recovery week). Strengthen it to assert that any remaining `"failed"` invoice is within the dunning window of churn or the simulation horizon for that customer.
## COPILOT-8
Location: tests/schemes/lifecycle/test_engine.py:240
URL: https://github.com/leadforge-dev/leadforge/pull/118#discussion_r3402279807
Root author: copilot-pull-request-reviewer
Comment:
`test_mrr_chain_consistency` currently asserts `sub.current_mrr == mrr` for *all* subscriptions, which will mask (or codify) the inconsistency of churned subscriptions having a non-zero terminal `current_mrr`. If churn sets terminal MRR to 0 (matching churn event `mrr_after`), this test should assert that behavior explicitly for churned rows.Run metadata: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Second half of
LTV-M4(LTV-Pk) — the weekly lifecycle simulator.simulate_lifecycle()evolves each customer week by week from their staggered start, drawing against the #117 hazard functions, and emits the three event tables (subscription_events,health_signals,invoices) plus one terminal-state subscription row per customer. This completesLTV-M4and is the last piece before snapshots/targets (LTV-M5).Engine contract
max(observation_date, start + early_tenure_weeks) + forward_window_days(default 730d), so all 90/365/730d pLTV targets are complete for both observation regimes.lifecycle_sim::<customer_id>) — one customer's trajectory is invariant to every other customer. Stronger than the lead-scoring shared-stream design, and provable: a test re-simulates a single customer solo and asserts the identical event sequence.payment_failure(dunning write-off),non_renewal(spiked draw on anniversary),voluntary.feature_depth_scorefeeds the same week's expansion hazard (latents → health signals → expansion).Engine calibration — discharging the #117 review obligation
The #117 review documented that the tenure shape invalidated the parameter annotations and deferred calibration here. Done: measured simulated first-year churn on motif-biased populations (n=600, 3 seed pairs) and retuned over three rounds:
All Pi directional-test constraints held through the retune (fragile failure >2× others, fragile recovery strictly lowest, churn orderings).
mechanisms.py's calibration comment rewritten from "expected to be tuned DOWN" to ENGINE-CALIBRATED with measured targets + guard-test pointer.Also
CustomerPopulationResultnow recordsmotif_family— the engine fetches the same family's params; passing it separately would invite silent drift.subscription_idis derived from the customer index up front and threaded into every event (an earlier draft back-filled it in an O(customers×events) pass; caught and removed before commit).downgrade_countwould be zero-variance; must revisit beforeLTV-M5ships the feature) and difficulty-tier scaling (LTV-M6).Tests (25)
Shape/validation; byte-equal determinism across all four tables; per-customer independence (solo re-simulation); weekly health cadence; monthly invoice cadence (±1); quarterly-only NPS; full-window coverage; event FK integrity; churn-state consistency + no-events-after-churn; expansion MRR chain reconciliation; renewal-events-only-on-anniversaries +
renewal_countreconciliation; dunning write-off →payment_failurechurn; per-motif year-1 churn bands; expansion-world dominance; majority-active-at-observation.ruff+mypyclean.Next
LTV-M5/LTV-Pl— calendar-anchored customer snapshot: aggregate health/events/invoices at the absoluteobservation_date, computemrr_change_at_snapshot+ themrr_change_full_periodtrap, and derive the threeltv_revenue_{90,365,730}dregression targets.🤖 Generated with Claude Code