berd-call: tune listening and interruption per audio setup

src-tauri/crates/berd-call/Cargo.toml

@@ -66,6 +66,7 @@ objc2 = "0.6"
 objc2-app-kit = "0.3.2"
 objc2-avf-audio = { version = "0.3.2", features = ["AVAudioApplication", "block2"] }
 objc2-core-audio = "0.3.2"
+objc2-core-foundation = "0.3.2"
 objc2-foundation = { version = "0.3.2", features = ["NSNotification", "NSOperation", "NSString", "block2"] }
 
 [target.'cfg(target_os = "macos")'.build-dependencies]

src-tauri/crates/berd-call/PROTOCOL.md

@@ -58,12 +58,12 @@ initial policy; a host-specific `auto` mode must be resolved before the request.
 {"type":"hello","id":1,"input_during_tts":"allow_barge_in","status_sound_output_device":"MacBook Pro Speakers"}
 ```
 
-The response uses `protocol:5` as the exact session message-set version. The
+The response uses `protocol:7` as the exact session message-set version. The
 parent must reject a version it does not support; the JSON version is independent
 of the fixed binary framing marker described below:
 
 ```json
-{"type":"ready","id":1,"protocol":5,"session":{"tts":{"revision":1,"backend":"siri","voice":"Aaron","language":"en-US","rate":1.0},"input_during_tts":{"revision":1,"policy":"allow_barge_in"}}}
+{"type":"ready","id":1,"protocol":7,"session":{"tts":{"revision":1,"backend":"siri","voice":"Aaron","language":"en-US","rate":1.0},"input_during_tts":{"revision":1,"policy":"allow_barge_in"}}}
 ```
 
 The `session.tts` object is the authoritative, sanitized TTS configuration.
@@ -397,3 +397,12 @@ receipts, flushes, and exits. EOF, malformed framing, fatal
 input failure, or process death cancels both authorities without transparent
 restart. A fatal error is flushed exactly once and followed by no protocol
 output.
+
+
+## Local speech detection (protocol 7)
+
+Conventional session readiness includes an optional `session.detection` snapshot with `settings.listening` and `settings.interruption` (`value`, `dynamic`), plus `adjustments`, `no_result` and `missed_detection` counters. These local controls are separate from Realtime Spokesperson turn detection.
+
+A host sends `{"type":"set_detection_settings","id":42,"settings":{"listening":{"value":0.7,"dynamic":true},"interruption":{"value":0.8,"dynamic":false}}}` to replace both settings atomically and discard stale input/learning evidence. The session returns `detection_settings_result` with that ID and an applied `snapshot`, or `snapshot:null` and a `message` explaining rejection. Invalid settings leave the current values unchanged. Dynamic choices require macOS STT.
+
+During adaptive tuning, the session sends `detection_settings_result` with `id:null` and the new snapshot. Hosts must handle these unsolicited updates, report applied values, and keep profile persistence associated with their actual input/output device pair. Protocol 7 adds `settings.timing` (`onset_ms`, `release_ms`) for Apple local VAD. Older profiles omit this field and receive defaults of 16 ms onset and 300 ms release. Final transcripts are emitted immediately, independently of the local activity gate.

src-tauri/crates/berd-call/README.md

@@ -9,6 +9,27 @@ management, synthesis and benchmark tools described below. Desktop Berd keeps
 its persisted settings and user interface. The standalone command includes a
 small macOS host for default-device capture, playback, and transcript delivery.
 
+### Local speech detection
+
+Conventional calls accept independent `--listening-threshold 0..1` (default `0.5`) and `--interruption-threshold 0..1` (default `0.65`) at `start` or `session`. Higher values require stronger evidence of speech. Listening controls detection before agent playback, including the gate that holds replies while input is speaking or awaiting recognition. Interruption controls detection during agent playback. These local thresholds do not change provider endpointing or Spokesperson server VAD.
+
+On macOS, a local speech detection also marks recognition pending. After local speech ends, admission continues waiting for recognition to settle, with a five-second no-result fallback. Raising the listening threshold can reduce false starts in background noise; it does not shorten that fallback or prove a particular microphone's false detections are resolved.
+
+Device profiles retain both values independently; legacy saved call arguments are accepted during migration. To change a running call, supply its complete session configuration, for example `settings --restart --voice Aaron --language en-US --listening-threshold 0.8 --interruption-threshold 0.7`. Restart replaces the session options; include the existing backend, model, rate and other choices you want to retain.
+
+### Experimental dynamic tuning and audio profiles
+
+With macOS speech recognition, `--dynamic-listening true` and `--dynamic-interruption true` independently opt into outcome-based tuning. Both default to false. Each enabled threshold moves one 0.01 tick after each qualifying unmatched outcome, without a cooldown. Dynamic values start and remain within 0.10 to 0.95; manual thresholds still accept 0 to 1.
+
+Apple recognition also supports per-audio-setup `timing` settings through `detection-profile --settings`: `onset_ms` (16 to 1000, default 16) and `release_ms` (16 to 3000, default 300). Timings round up to 16 ms audio frames and apply only to local activity detection. Finalized transcripts are forwarded immediately, independently of release timing. The recognizer continues to receive all microphone audio during onset gating. Other recognition backends retain their existing segmentation timing.
+
+
+A local detection that settles without a nonempty transcript suggests raising sensitivity's threshold. A nonempty transcript after at least five seconds without local detection suggests lowering it. Recognized speech clears false-trigger evidence. Split-result grace periods, mode changes, mute/reset epochs and device changes discard ambiguous evidence. These are experimental heuristics: recognition failure and playback leakage can still mislead them. They do not establish an optimal threshold or replace acoustic echo cancellation.
+
+The standalone macOS host saves applied thresholds and dynamic choices in `detection-profiles.json` beside its saved settings, keyed by the stable input/output device UID pair. It restores the pair on call startup and checks for device changes once per second. Explicit threshold or dynamic flags override the matching profile choice; unspecified choices retain their saved values. Detection flags are removed from inherited session arguments so a later voice-only restart restores learned values. A new pair starts with defaults during a call. Profile files contain settings, not audio recordings or transcript text. An explicit profile edit takes precedence over a late automatic update.
+
+`berd-call detection-profile` reports the default audio setup and saved choices without starting capture. `berd-call detection-profile --settings '{"listening":{"value":0.70,"dynamic":true},"interruption":{"value":0.80,"dynamic":false}}'` saves choices for that pair. During a call, `berd-call status` exposes applied values and evidence counters in `session.detection`, plus `audioSetup`. Saving a profile alone affects its next restoration; the plugin restarts the call session when a user changes a control. Dynamic adjustments apply during the current session.
+
 ## Command-line interface
 
 On macOS, install [Berd](https://github.com/block/berd/releases) first. Its

src-tauri/crates/berd-call/VAD_VISION.md

@@ -0,0 +1,11 @@
+# Adaptive speech detection
+
+Proposed direction, October 22, 2026. Independent static thresholds are implemented by PR 421. Adaptive tuning and audio profiles below are proposed until their implementation is validated.
+
+A caller should be able to stop background noise holding replies without losing quiet speech. Listening and interruption each have a slider, visible current value, and an opt-in Dynamically adjust checkbox. Adjustments move one 0.01 tick at a time within 0.10 to 0.95. Manual values remain available across the full 0 to 1 range.
+
+For the initial experimental macOS recognizer, local detections that settle without a nonempty transcript are evidence to increase the threshold. Nonempty transcripts without corresponding local detection provide evidence to decrease it. These are heuristics, not ground truth: recognizer failures, late or split results and playback leakage can mislead them. Ignore ambiguous observations across mute/reset, playback-mode and device changes; move one tick per qualifying outcome and expose evidence counters with current values. Other recognition backends require their own reliable outcome signals before enabling adaptive tuning.
+
+Save independent values and dynamic choices by stable input/output device identity pair. AirPods input/output, another headset, and the built-in input/output pair retain separate profiles. Restoring a profile must discard old learning evidence so one setup cannot train another. No microphone recordings or transcript text are stored in these profiles.
+
+The host owns device identity and persistence. The shared input runtime owns thresholds, and the host-facing session protocol carries updates and reports applied values. Existing settings files remain readable by older binaries. Prefer a separate bounded, atomically replaced profile file to changing their strict schema.

src-tauri/crates/berd-call/src/cli_help.rs

@@ -147,13 +147,21 @@ Usage:
     --model-dir ABSOLUTE_PATH --voice ID
 
 Each form also accepts:
-    [--rate FLOAT]
+    [--rate FLOAT] [--listening-threshold 0..1] [--interruption-threshold 0..1]
+    [--dynamic-listening true|false] [--dynamic-interruption true|false]
     [--stt-backend macos|parakeet|openai] [--stt-model-dir PATH] [--stt-hint PHRASE]...
     [--mode conventional|expert-spokesperson] [--allow-direct-agent-speech]
     [--spokesperson-voice NAME] [--spokesperson-rate 0.25..1.5]
     [--spokesperson-options JSON]
     [--realtime-url WS_URL] [--stt-url WS_URL] [--tts-url HTTP_URL]
 
+--listening-threshold sets local speech detection before agent playback (default 0.5).
+--interruption-threshold sets local speech detection during agent playback (default 0.65).
+Higher values require stronger evidence of speech before pausing playback.
+Dynamic adjustment is experimental, off by default, and requires macOS STT.
+Each enabled threshold learns independently, moving by 0.01 within 0.10..0.95.
+Use detection-profile [--settings JSON] to read or save the default audio setup profile.
+
 The host owns microphone capture, audio playback, transcript delivery, and
 agent integration. See PROTOCOL.md for the framed stdin, stdout, and PCM
 contracts. Endpoint URL flags override only the matching service for this call