Proposal: agent-native interactive preview app (serve-sim integration)

[obj-p]

Summary

Build a compelling, agent-native preview experience on top of PreviewsMCP, comparable to OpenAI Codex's Build iOS Apps plugin. The idea is to pair PreviewsMCP's standalone #Preview rendering + hot reload with a live, interactive view of the running simulator streamed to the agent, using serve-sim (referred to as "sim-serve") as the streaming layer.

Why

Codex's Build iOS Apps plugin closes the build → view → test → fix loop inside the agent: it mirrors the simulator in a browser, opens SwiftUI previews, and hot-reloads edits without leaving Codex. PreviewsMCP already has the hard part for the preview half:

• It is an MCP server that renders #Preview in its own host process (macOS and the iOS simulator), outside Xcode.
• It supports hot reload and interactions (tap / toggle / swipe).
• It works across SPM, Xcode, and Bazel — including Bazel Build-with-Bazel Apple bundle targets (see Bazel: render previews for Apple bundle targets (exec app + framework) #226), so the preview half is not tied to one build system.

What is missing is a live, interactive surface the agent (and the human) can watch and drive. serve-sim provides exactly that: it captures a booted simulator's framebuffer via simctl io and exposes an MJPEG stream + WebSocket control channel with a browser UI, with no Xcode plugin or in-app instrumentation, and it is already meant to be hosted for agent tools (Codex, Cursor, Claude Desktop).

Proposed shape

• PreviewsMCP renders and hot-reloads the preview into a booted iOS simulator (its existing iOS path).
• serve-sim streams that same simulator to a browser, giving a live interactive view and input channel.
• The MCP surface exposes the stream/control endpoint so an agent can open the preview, watch it update on edits, and interact, all in-loop.

The result is an MCP-native "preview app" experience: the differentiator over the Codex plugin is that PreviewsMCP renders the #Preview standalone (no Xcode preview pipeline, any build system), so it can cover projects Xcode's canvas cannot.

Open questions / risks

• Integration model. serve-sim is Node.js (arm64 macOS only) and JS/React, so it is an orchestrated external process, not a Swift dependency. Decide bundle vs. optional documented integration, and how PreviewsMCP launches or discovers it.
• Licensing. Confirm serve-sim's license allows the intended use/distribution.
• Scope. This is a product-level experience, likely several pieces (simulator lifecycle, stream lifecycle, MCP endpoints, an agent-facing UI). Worth breaking into sub-issues.
• Overlap. Codex uses XcodeBuildMCP; clarify where PreviewsMCP's standalone-render niche fits versus a full build-and-run loop.

Notes

Idea raised by @jasonprasad. Related: #226 (Bazel Apple-bundle previews), #227 (build-system override).