Strategy selection as a domain-neutral engine capability#17
Merged
Conversation
BuildPlannerRouter (and backtest.New) now return errors instead of silently degrading a misconfigured domain to the text planner; a nil planner still declines gracefully. strategy.WeightKey is the single definition of the outcome-weight key syntax, used by both the fitter and the selector lookup. The disagreement penalty takes its quantum from the caller — wire threads the domain's materiality ConfidenceDelta — and the hysteresis margin defaults to that same step, so the 'haircuts are always material' invariant holds for any domain, not just investing's 0.05.
Selector assembly moves out of the finance builder into BuildPlannerRouter: any domain with enabled strategies competes them. A per-kind StrategyKit (child builder + context classifier with its CLOSED label set + data availability) serves numeric kinds — finance registers the existing lens builder and regime classifier — while kindless domains get prompt-variant text children composed Guided(Caching(base)) so variants cache distinctly, with the strategy ID suffixed into the prompt version for provenance. Guard rails from the design review: a kind domain whose kit is unavailable declines to plain guided exactly as without strategies (never into mis-parameterized text children); the default lives in pack data (Descriptor.SelectionDefaultOn — investing true, earned by its gates) so declaring strategies never silently changes a deployment's cost profile; declared-but-invalid configuration (unknown regime labels, empty or duplicate text templates, strategies on the generic domain, all-gated strategy sets) fails the build instead of degrading.
Text prompt variants self-report confidences with no shared yardstick; the verify comparator has one PlanVerifier critique every candidate (dropping unsupported moves, adjusting miscalibrated confidence) before the pure utility math arbitrates. The contract is all-or-nothing: any review error discards every adjustment and the competition runs on raw numbers with the reason recorded in provenance — partially reviewed competitions would compare adjusted confidences against raw ones. A candidate whose moves are all rejected becomes inadmissible rather than restored. Provenance records the comparator mode, and dde strategy-fit filters outcome samples to the domain's configured mode before fitting — weights observed under one comparator are not valid under another. Policy validates the enum (judge reserved for a future single-call side-by-side reviewer) and rejects the double-verification of comparator verify over a verify multi-planner.
The strategy selector fans its children out in parallel, so a cold cache key issued one vendor request per child at once — a pure TTL cache only helps the second decision. CachingProvider now runs one flight per key: the leader fetches and stores per the method's own policy, every concurrent caller shares its outcome, and a double-check after winning leadership closes the lookup-to-flight race that would otherwise refetch right behind a retiring leader. Quote/fundamentals waiters re-check the point-in-time lookahead guard, since the leader may have fetched with a later asOf in the same TTL bucket. Verified with a blocking counting provider under the race detector: four concurrent callers, one vendor call per method.
…t the architecture Growth competes expansion-first / retention-efficiency / experiment-driven; career competes compounding-mastery / optionality / network-leverage. Both packs keep SelectionDefaultOn false — no validation gates have earned a default flip, and each competing lens is a full model call — so behaviour is byte-identical until a policy opts in. Under the mock the variants tie and the canonical first lens wins deterministically. docs/concepts.md gains the Strategy / selection / comparator / regime vocabulary; docs/architecture.md gains the strategy-selection flow and its rules (kit decline, earned defaults, fail-fast validation, per-comparator outcome fitting).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Generalizes multi-strategy selection (PR #16, investing-only) into a first-class capability any domain can use. A pack declares named strategies as pure data; the engine competes them, selects against the goal, and audits the full competition — with the comparability, gating, and feedback mechanisms each domain type needs.
How
BuildPlannerRouter. A per-kindStrategyKit(child builder + context classifier with a closed label set + data availability) serves numeric kinds; kindless domains get prompt-variant children composedGuided(Caching(base))so variants cache distinctly, with the strategy ID stamped into the prompt version.Descriptor.SelectionDefaultOnlives in pack data — investing stays on (earned by its backtest gates); growth and career declare lenses (expansion/retention/experiment; mastery/optionality/network) but stay off until a policy opts in, so declaring strategies never silently changes a deployment's cost profile.BuildPlannerRouternow surfaces builder errors instead of degrading to text. Unknown regime labels, duplicate or empty prompt templates, strategies on the generic domain, and all-gated strategy sets fail the build. A kind domain whose data dependency is missing declines to its plain fallback — never into mis-parameterized text children.CandidateReviewerseam, onePlanVerifiercritiques every candidate (all-or-nothing — any error reverts the whole competition to raw numbers with the reason in provenance) before the pure utility math arbitrates;judge(single side-by-side call) is reserved in the policy enum. Provenance records the comparator anddde strategy-fitpartitions outcome samples by it.ConfidenceDelta;strategy.WeightKeyis the single definition of the outcome-weight key syntax.CachingProvidercoalesces concurrent identical fetches (with a post-leadership double-check), so the selector's parallel fan-out costs one vendor call per (method, args) instead of one per child — verified under the race detector.Verification
Full race suite green; investing regression e2e identical through the generic path (same winner/candidates on the example thesis); growth e2e off-by-default and competing 3 lenses under a policy opt-in with a deterministic canonical winner; docs updated (concepts vocabulary + architecture flow).