Improve overlay feedback, Edit Mode capture, and Codex command routing#408
Improve overlay feedback, Edit Mode capture, and Codex command routing#408tarushvkodes wants to merge 7 commits into
Conversation
tarushvkodes
changed the title
|
Update: improved Edit Mode replacement for Google Docs and similar browser editors.\n\n- Captures the original target PID when selected text is captured, so accepting a rewrite can return to the document instead of aiming at the FluidVoice overlay.\n- Routes Edit Mode acceptance through the reliable paste path, which web rich-text editors handle better than synthetic direct typing.\n- Gives clipboard-based selection capture a longer window for Google Docs/browser editors that update the pasteboard more slowly after Cmd+C.\n\nValidated locally:\n- swiftc parse over Sources/Fluid Swift files\n- git diff --check\n- plutil lint for project/Info/entitlements\n- xcodebuild clean build for macOS arm64\n- installed and ad-hoc signed local /Applications/FluidVoice.app |
tarushvkodes
force-pushed
the
codex/lower-visualizer-threshold
branch
from
003a852 to
314f9c7
Compare
|
Update: rebased this branch onto upstream main at v1.6.1 (49e91c0) and reinstalled a fresh local build.\n\nNotes:\n- Dropped the old Package.resolved-only MCP SDK bump during rebase because upstream removed that dependency.\n- Preserved the visualizer threshold, Codex command mode, notch output, and Google Docs/Edit Mode replacement changes.\n- PR head is now 314f9c7 and reports mergeable.\n\nValidated locally:\n- swiftc parse over Sources/Fluid Swift files\n- git diff --check\n- plutil lint for project/Info/entitlements\n- xcodebuild clean build for macOS arm64\n- installed and ad-hoc signed /Applications/FluidVoice.app |
tarushvkodes
changed the title
There was a problem hiding this comment.
Pull request overview
This PR improves FluidVoice’s user-facing feedback and reliability across three flows: overlay waveform responsiveness, Edit Mode selection capture + rewrite insertion, and optional Command Mode routing to Codex (notch or app handoff).
Changes:
- Makes the overlay visualizer respond to quieter mic input by lowering the default noise threshold and adds an integration test for the new default.
- Improves Edit Mode reliability by adding clipboard-based selected-text capture fallback and forcing a “reliable paste” insertion path for accepted rewrites, with PID tracking.
- Adds an opt-in Command Mode “Route to Codex” setting with notch/app handoff styles, updates UI controls, and stabilizes expanded notch output behavior.
Reviewed changes
Copilot reviewed 12 out of 12 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| Tests/FluidDictationIntegrationTests/DictationE2ETests.swift | Adds an integration test asserting the new, more sensitive default visualizer noise threshold. |
| Sources/Fluid/Views/CommandModeView.swift | Adds Codex routing UI controls and adjusts readiness/notch-output behavior based on routing settings. |
| Sources/Fluid/UI/SettingsView.swift | Updates the visualizer threshold reset behavior to use the new default constant. |
| Sources/Fluid/Services/TypingService.swift | Introduces a “force reliable paste” path (typeTextReliably) and plumbs it through the insertion pipeline. |
| Sources/Fluid/Services/TextSelectionService.swift | Adds clipboard-preserving Cmd+C fallback for selected-text capture when AX APIs fail. |
| Sources/Fluid/Services/RewriteModeService.swift | Tracks target PID for Edit Mode context and routes accepted rewrites via the reliable paste path. |
| Sources/Fluid/Services/NotchOverlayManager.swift | Stabilizes expanded command-notch presentation state transitions and cleanup behavior. |
| Sources/Fluid/Services/CommandModeService.swift | Adds Codex routing branch and notch sync behavior for Codex “notch” handoff. |
| Sources/Fluid/Services/CodexHandoffService.swift | New service that runs Codex CLI for notch output or activates/pastes into the Codex app. |
| Sources/Fluid/Persistence/SettingsStore+CommandMode.swift | Bypasses command readiness issues when Codex routing is enabled. |
| Sources/Fluid/Persistence/SettingsStore.swift | Adds default visualizer threshold constant and persists Codex routing + handoff-style settings. |
| Sources/Fluid/ContentView.swift | Routes rewrite acceptance through RewriteModeService.acceptRewrite(...) with PID targeting. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| } | ||
|
|
||
| private func pasteAndSubmit(_ text: String) async -> Bool { | ||
| Self.pasteboardSessionSemaphore.wait() |
| pasteboard.clearContents() | ||
| guard pasteboard.setString(text, forType: .string) else { | ||
| self.restorePasteboardSnapshot(snapshot, to: pasteboard) | ||
| Self.pasteboardSessionSemaphore.signal() | ||
| return false | ||
| } | ||
|
|
||
| guard self.sendCommandKey("v") else { | ||
| self.restorePasteboardSnapshot(snapshot, to: pasteboard) | ||
| Self.pasteboardSessionSemaphore.signal() | ||
| return false | ||
| } | ||
|
|
||
| try? await Task.sleep(nanoseconds: 100_000_000) | ||
| guard self.sendReturnKey() else { | ||
| self.restorePasteboardSnapshot(snapshot, to: pasteboard) | ||
| Self.pasteboardSessionSemaphore.signal() | ||
| return false | ||
| } | ||
|
|
||
| try? await Task.sleep(nanoseconds: 1_000_000_000) | ||
| self.restorePasteboardSnapshot(snapshot, to: pasteboard) | ||
| Self.pasteboardSessionSemaphore.signal() |
| let deadline = Date().addingTimeInterval(1.0) | ||
| var copiedText: String? | ||
| repeat { | ||
| if pasteboard.changeCount != clearedChangeCount, | ||
| let text = pasteboard.string(forType: .string), | ||
| !text.isEmpty | ||
| { | ||
| copiedText = text | ||
| break | ||
| } | ||
| usleep(15_000) | ||
| } while Date() < deadline | ||
|
|
||
| self.restorePasteboardSnapshot(snapshot, to: pasteboard) | ||
|
|
||
| guard let copiedText else { | ||
| self.diag("Clipboard fallback failed: clipboard did not receive selected text") | ||
| return nil | ||
| } |
| func captureSelectedText() -> Bool { | ||
| let targetPID = TypingService.captureSystemFocusedPID() | ||
| ?? NSWorkspace.shared.frontmostApplication?.processIdentifier | ||
| if let text = textSelectionService.getSelectedText(), !text.isEmpty { |
| .outputDelivered, | ||
| properties: [ | ||
| "mode": AnalyticsMode.rewrite.rawValue, | ||
| "method": AnalyticsOutputMethod.typed.rawValue, |
| self.asr.typeTextToActiveField( | ||
| self.rewriteModeService.rewrittenText, | ||
| preferredTargetPID: typingTarget.pid | ||
| self.rewriteModeService.acceptRewrite( |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 314f9c7e81
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if self.shouldSyncCommandNotchState { | ||
| NotchContentState.shared.addCommandMessage(role: result.success ? .assistant : .status, content: result.message) | ||
| NotchContentState.shared.setCommandProcessing(false) |
There was a problem hiding this comment.
When a Codex notch handoff starts, the earlier block can set isCommandProcessing to true, but this completion block recomputes shouldSyncCommandNotchState after the async Codex run. If the user toggles Codex routing off or changes the handoff style while Codex is still running, cleanup is skipped and the expanded notch can remain stuck in a processing/disabled state even though isProcessing is false; capture the initial sync decision or always clear command processing on completion.
Useful? React with 👍 / 👎.
Summary
This PR includes a focused set of user-facing improvements to FluidVoice's overlay, Edit Mode, and Command Mode.
Overlay feedback
0.4to0.12.Edit Mode reliability
Cmd+Cfallback afterAXSelectedText/ selected-range capture fails.Command Mode to Codex
CommandModeRouteToCodexsetting, defaulting off, so existing Command Mode behavior remains unchanged unless enabled.Notch: keeps FluidVoice's Command Mode notch UX, runs the bundled Codex CLI in the background, and displays the result in the notch without focusing the Codex app.App: activates the Codex desktop app and pastes/submits the spoken command there.Why
A few real-world workflows were failing or feeling confusing:
The changes are preference-backed and conservative: normal dictation and default Command Mode behavior are preserved unless the user enables the Codex route.
Validation
Validated locally on macOS with Xcode:
swiftc -parse $(rg --files Sources/Fluid -g '*.swift')git diff --checkplutil -lint Fluid.xcodeproj/project.pbxproj Info.plist Fluid.entitlementsxcodebuild clean build -project Fluid.xcodeproj -scheme Fluid -destination 'platform=macOS,arch=arm64' CODE_SIGN_IDENTITY='' CODE_SIGNING_REQUIRED=NO CODE_SIGNING_ALLOWED=NO/Applications/FluidVoice.appNotes
mainatv1.6.1(49e91c0).