Drive the iOS Simulator from the command line the way agent-browser drives a browser - accessibility snapshots with element refs, taps, drags, typing, and screenshots. Built for AI coding agents, useful for humans too.
An open-source project from SquadUp Labs.
$ agent-ios snapshot -i
snapshot gen=2 app=com.apple.mobilesafari screen=402x874
@e16 TextField "Address" id=TabBarItemTitle val="example.com" (149,806 104x20)
@e18 Button "More" (320,792 48x48)
$ agent-ios tap @e16
tapped @e16
$ agent-ios type "anthropic.com" --submit
typed 13 chars + return
xcrun simctl gives you screenshots, app launching, deep links, and permissions control - but no touch injection and no way to read the UI. Every tool in this space exists to fill that one gap.
agent-ios fills it with Apple's supported XCUITest APIs and nothing else:
- 100% public API. No private frameworks (the reason idb keeps breaking across Xcode releases).
- Zero third-party runtime dependencies. The CLI's only package dependency is apple/swift-argument-parser. The in-simulator runner uses only Apple frameworks.
- Real accessibility tree. Snapshots come from the same source VoiceOver uses, so they work on React Native / Expo, SwiftUI, UIKit, and system apps alike.
- Real touch events. Gestures go through the UI-testing stack, not synthesized mouse clicks on a window - the simulator can stay in the background while an agent works.
agent-ios CLI (Swift binary)
| \
| HTTP \ xcrun simctl (screenshot, openurl, listapps, privacy)
v v
+---------------------------------------------------------+
| iOS Simulator |
| AgentIOSRunnerUITests-Runner.app (auto-generated) |
| - HTTP server (Network.framework, 127.0.0.1:22087) |
| - XCUITest: tap / drag / swipe / type / keys |
| - accessibility snapshot -> ref'd tree (@e1, @e2...) |
+---------------------------------------------------------+
The "runner" is a tiny XCUITest bundle whose single never-ending test method serves HTTP. The CLI builds it once (xcodebuild build-for-testing, cached in ~/.agent-ios/DerivedData and keyed on the runner sources + Xcode version), launches it detached (xcodebuild test-without-building), waits for /status, and then every command is a fast HTTP round trip. Simulator processes share the Mac's network stack, so 127.0.0.1:22087 just works.
Screenshots never touch the runner - they go straight through simctl io screenshot.
Per-device state (pid, port, launch generation) lives in ~/.agent-ios/run/<udid>/server.json. Logs live in ~/.agent-ios/logs/<udid>.log.
- macOS with Xcode installed (verified on Xcode 26.6)
- A booted iOS Simulator (verified on iOS 26.3)
- Swift toolchain (comes with Xcode)
git clone <this repo> && cd agent-ios
swift build
.build/debug/agent-ios doctor # environment checks
.build/debug/agent-ios snapshot -i # auto-starts the server, prints refs
.build/debug/agent-ios tap @e3 # tap an element by ref
.build/debug/agent-ios screenshot # prints a PNG pathThe first command that needs the runner builds and starts it automatically (tens of seconds the first time, ~3s warm).
To put agent-ios on your PATH, build a release binary and link it into a PATH directory:
swift build -c release
ln -sf "$(pwd)/.build/release/agent-ios" ~/.local/bin/agent-ios # if ~/.local/bin is on PATHThe link resolves back into the checkout, so the CLI still finds the Runner/ project (or set AGENT_IOS_ROOT). Keep the checkout around - it holds the runner project the tool builds at runtime.
- Usage - full command reference, the ref workflow, exit codes, examples.
- Configuring your app - point agent-ios at any iOS app (native, SwiftUI, UIKit, React Native, Expo, Flutter), not just the examples here.
- Integrations - wire agent-ios into Claude Code, OpenAI Codex CLI, Cursor, and other agents.
- HTTP protocol - the contract between the CLI and the in-simulator runner.
- Contributing · Releasing · Changelog · Security
agent-ios is a plain CLI, so any agent that can run shell commands can drive the simulator with it. See docs/integrations.md for copy-paste setups for Claude Code (a skill), OpenAI Codex CLI (AGENTS.md), Cursor (.cursor/rules), and a note on MCP. The canonical agent workflow lives in docs/skill/SKILL.md.
agent-ios snapshot(orsnapshot -ifor only interactive, on-screen elements) prints the accessibility tree. Interactive or labeled elements get refs:@e1,@e2, ...- Act on refs:
tap @e3,drag @e3 @e7,type "hi" --ref @e5. - Refs belong to one snapshot generation. After the UI changes, take a new snapshot - stale refs exit with code 3 and a "take a new snapshot" message.
screenshotany time you want visual confirmation.
Snapshot lines carry the element type, label in quotes, id= (accessibility identifier / React Native testID), val=, ph= (placeholder), the frame in points (x,y wxh), and flags [disabled] [selected] [focused] [offscreen]. Offscreen elements (closed drawers, scrolled-away rows) keep their refs but refuse to tap with a helpful error.
| Command | What it does |
|---|---|
agent-ios devices |
List simulators, booted first |
agent-ios apps |
User-installed apps on the target sim |
agent-ios status |
Device, server, and session state |
agent-ios doctor |
Environment checks with fixes |
agent-ios server start|stop|restart|logs [-f] [-l N] |
Runner lifecycle (auto-starts on demand) |
agent-ios attach [bundleId] |
Set the session app; no arg auto-detects the foreground user app |
agent-ios launch <bundleId> [--relaunch] [--no-attach] |
Launch an app (implies attach) |
agent-ios terminate <bundleId> |
Kill an app |
agent-ios open <url> |
Deep link via simctl (Expo dev URLs work) |
agent-ios screenshot [-o path] |
PNG of the screen; prints the path |
agent-ios snapshot [-i] [--json] [--app id] |
Accessibility tree with refs |
agent-ios tap @e3 | 200 400 [--double] [--long] [--duration S] |
Tap / double tap / long press |
agent-ios drag @e3 @e7 | x1 y1 x2 y2 [--duration S] [--velocity slow|fast|N] [--hold S] |
Drag between targets |
agent-ios swipe up|down|left|right [--from @ref|x,y] [--distance N] |
Swipe (defaults to screen center) |
agent-ios type "text" [--ref @e5] [--submit] |
Type into the focused field; --ref taps first; --submit presses Return |
agent-ios key home|return|delete|tab|escape|space [--times N] |
Key / device button presses |
agent-ios alert [accept|dismiss|"Label"] |
Inspect or answer system alerts |
agent-ios permissions grant|revoke|reset <service> [bundleId] |
Pre-answer permission dialogs |
agent-ios install <path.app> / uninstall <bundleId> |
App management |
Global: --udid <UDID> on every command (defaults to the single booted simulator, or AGENT_IOS_UDID).
Exit codes: 0 ok, 1 generic failure, 2 runner not running, 3 stale/unknown ref, 4 no session app.
Typing fails with "no keyboard on screen". Tap a text field first (or pass --ref). If a field is clearly focused but the software keyboard never appears, the Simulator's hardware keyboard is capturing input:
defaults write com.apple.iphonesimulator ConnectHardwareKeyboard -bool false
# then quit and reopen Simulator.appagent-ios doctor checks this.
First server start is slow. It compiles the runner once. Warm starts are ~3 seconds. Rebuilds happen automatically only when the runner sources or the Xcode version change.
Multiple booted simulators. Pass --udid or set AGENT_IOS_UDID. Each device gets its own server, port, state, and log.
A system alert is blocking everything. Action responses and snapshots warn with [!] a system alert is present. Use agent-ios alert to list its buttons, agent-ios alert accept / dismiss / an exact label to answer it, or agent-ios permissions grant ... beforehand to avoid it entirely.
React Native elements all show as Other. That is how RN exposes tappables to accessibility. Labeled Other nodes get refs too, so tap @ref works fine. Give elements testID props to get stable id= values in snapshots.
Out-of-process views. The system share sheet renders in a separate scene whose element frames are in its own coordinate space; snapshot detects this and translates them to screen space, so the share sheet is tappable by ref like anything else. A few surfaces run in a fully separate process that is not bridged into the app's accessibility tree at all - most notably the Files "save to" / document picker - so snapshot returns nothing for them. Drive those from a screenshot with coordinate taps, or pre-grant access with agent-ios permissions grant ... so the picker never appears.
- idb (Facebook/Meta): drives the sim through private CoreSimulator/SimulatorKit frameworks - powerful, but each Xcode release can break it and maintenance has slowed. agent-ios trades some power for public-API stability.
- Appium / WebDriverAgent: same architecture idea (an in-simulator XCUITest server), but WDA is a large WebDriver-protocol implementation with private-API optimizations. agent-ios is a minimal from-scratch runner with a purpose-built protocol for agent workflows.
- Maestro: a full test-authoring product with YAML flows. agent-ios is not a test framework - it is a low-level CLI designed for an AI agent (or a human) to poke at whatever is on screen.
Contributions are welcome. See CONTRIBUTING.md for how to build, test, and submit changes. We use Conventional Commits and Semantic Versioning; releases are described in RELEASING.md. Please follow the Code of Conduct, and report security issues per SECURITY.md.
MIT - see LICENSE. Copyright (c) 2026 SquadUp Labs.