Skip to content

feat(rtmg): spectral steering (on models/backend)#233

Merged
ryanontheinside merged 2 commits into
ryanontheinside/feat/models/spectral/spectral-controlfrom
ryanontheinside/feat/models/spectral/demo-on-spectral-control
Jun 9, 2026
Merged

feat(rtmg): spectral steering (on models/backend)#233
ryanontheinside merged 2 commits into
ryanontheinside/feat/models/spectral/spectral-controlfrom
ryanontheinside/feat/models/spectral/demo-on-spectral-control

Conversation

@ryanontheinside

Copy link
Copy Markdown
Collaborator

feat(rtmg): spectral steering, re-expressed in the registry-driven control surface

Merge order

Stacked: opened against ryanontheinside/feat/models/spectral/spectral-control. Merge #224 -> #220 -> #229 -> spectral-control before this.
Stack: main <- #224 <- #220 <- #229 (models/backend) <- spectral-control <- this.
Supersedes #163 (same feature, ported from the pre-seam ryanontheinside/feat/spectral/demo-on-spectral-control; close that one in favor of this).
Heads-up for whoever merges this after the mrt2 (#230) / sa3 (#231) leaves (or vice versa): acestep/streaming/families.py will conflict — this PR extends the ACE factory + knob universe, those add their families to the same registries. Resolution is mechanical (keep both).

The demo/control surface for the spectral-control backend underneath: four perceptual steering knobs (bright / warm / rough / density) plus runtime-allocatable manual slots that fire any vector in the probe catalog at an operator-chosen (layer, step, alpha).

What's here (two commits)

4c3eff1 tab rename. "Channels" tab -> "Experimental"; section labels become "channel highlights" / "channel groups".

dd556b8 spectral steering. The original feature, re-expressed in the post-refactor architecture instead of the old hand-rolled knob banks:

  • Knob registry (acestep/streaming/knobs.py): pure steering_axis_spec() / manual_slot_specs() factories (same pattern as lora_strength_spec). ACEStepBackend.knob_specs() appends them when the session's checkpoint has a vector bundle, so /api/knobs-shaped manifests, the wire ready frame, MCP list_knobs, and DynamicKnobPanel all pick them up with zero per-transport code.
  • Backend (ace_backend.py): owns the SteeringController (constructed in families._make_acestep via ensure_steering_vectors(checkpoint)); _sync_steering() pushes configs on snapshot change (pipeline identity is part of the key so a steps_override rebuild re-arms), sitting next to _sync_channel_guidance in _prepare_tick. New Capabilities.steering bit, true only when vectors loaded.
  • Session (session.py): manual_slot_add / manual_slot_pop ops gated by @requires_capability("steering", ...); KnobState + the cached spec map mirror the controller's LIFO slots; ManualSlotCount bus event published on success AND refusal; steering_payload() (count / cap / available) shared by snapshot and the ready frame.
  • Wire contract (protocol.py): manual_slot_add / manual_slot_pop commands (requires="steering"), manual_slot_count event, three wire-optional ready fields. wireContract.gen.ts regenerated; all drift guards green.
  • MCP (mcp_server.py): add_manual_slot / pop_manual_slot / list_manual_steering_vectors tools; list_knobs and knob validation now prefer the session snapshot's backend-owned knob_manifest (the only projection that can carry backend-specific knobs), falling back to the static registry for older servers.
  • Client: SDK stays store-free per Backend-owned control contract + demon-client SDK #220's convention — manualSlotCount / manualSlotCap / steeringAvailable are RemoteBackend instance fields + a typed manual_slot_count switch case; the host mirrors them in useStartSession. Senders typed against the generated command types. VoiceTile grows the steering knob rack + manual-slot rows with +/- controls; tooltips in SliderTile; SLIDER_META pre-registers man_*_{1..16}.

Deliberately dropped from the original commit: the unrelated config.json default LoRA/prompt swaps (funkv1 / Phonk, prompt A) — debugging defaults that rode into #163.

Validation

  • 149/149 unit tests (incl. wire-contract drift guards over the new commands/event/capability and the knob-homonym guard over the extended universe).
  • Web: tsc clean, next build clean, vitest unit 35/35, replay tier 6/6 (bit-exact).
  • Offline smoke against the real cached v15-turbo bundle: 144-cell catalog loads, auto-axis + manual configs build correctly (sign/layer-offset/fractional-step semantics intact).
  • Golden harness: pending a free GPU window; will be run at this tip and reported here.

Also relabels the two sections inside as "channel highlights" and
"channel groups" so the section ↔ tab vocabulary stays honest.
@ryanontheinside ryanontheinside merged commit 6dbb2fc into ryanontheinside/feat/models/spectral/spectral-control Jun 9, 2026
@ryanontheinside ryanontheinside deleted the ryanontheinside/feat/models/spectral/demo-on-spectral-control branch June 9, 2026 11:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant