zhongerxin/iPhone-use
AI EngineeringLet Codex, WorkBuddy or Doubao work drive a physical iPhone over USB through WDA and MCP tools.
What it does
iPhone Use lets an AI assistant on macOS — Codex, WorkBuddy or Doubao work — operate a physical iPhone over USB. You describe a task in natural language; the assistant opens apps, reads the control tree, taps, scrolls, types Unicode text, collects list items, and shows the live phone screen in a sidebar widget. The phone side is WebDriverAgent, which the project pins to a verified build; the Mac side is a local MCP service plus an install/setup skill.
The project adds Codex plugin packaging, 18 model-facing MCP tools, and a screen widget on top of Appium's WDA, without needing a separately run Appium Server. It is aimed at people who want to automate real-iPhone workflows (list collection, app navigation, text entry) with an assistant rather than writing XCUITest scripts.
Requirements
- macOS plus one of the supported hosts: Codex (with Codex CLI for the Codex installer), WorkBuddy or Doubao work.
- A full Xcode installation, launched once and configured, with your own Apple account and development team able to sign the WDA Runner. Command Line Tools alone are not enough.
- A physical iPhone connected over USB, trusted with the Mac, with developer mode enabled and unlocked during install/launch.
- Python 3.9+, and Node.js 20.19+/22.12+/24+ with npm 10+.
- Xcode must support the phone's iOS version. No jailbreak is required.
Quick start
- Clone and run the installer:
git clone https://github.com/zhongerxin/iPhone-use.git
cd iPhone-use
sh scripts/install.sh
The script stages the source, registers a local marketplace through Codex CLI, installs the plugin and skills, and registers the MCP service as iphone_use. Reconnect or open a new Codex chat afterwards.
-
In the new chat, enable iPhone Use and ask the setup skill to configure the device, for example: "用 iphone-use-setup 帮我配置通过 USB 连接的 iPhone,安装并启动 WDA,验证 READY,然后显示手机屏幕。" The skill walks through diagnosis, device discovery, signing configuration, fetching WDA, and background build/launch. Signing uses your own Apple account, team and bundle ID — not the author's.
-
Wait until
pua_readyreturnsready=true, then issue phone tasks, e.g. "打开备忘录,新建一条草稿,输入下面的中文内容,核对文字后保留。" Screenshots and the control tree are returned by the tools; the live preview appears in the widget.
For WorkBuddy run python3 workbuddy/install.py and trust iphone_use in Connector Management (WorkBuddy guide); for Doubao work run python3 doubao/install.py and create an STDIO connector plus upload the two skill folders as printed (Doubao guide). These variants do not need Codex CLI and disable anonymous analytics by default.
Notes and limits
- Signing, WDA installation and launch are per-device; configured devices and builds are reused, so normal tasks don't reinstall each time.
- The tool does not submit typed text by default; input and submission are separate steps, and the model is expected to verify key screens and final results.
- Authentication (passwords, verification codes, Face ID) stays with you; the widget pauses and overlays a "take over on the phone" prompt when handoff is needed.
- Multiple processes share an operation lock, so using two hosts on the same phone requires waiting; action timeouts or disconnects may be marked uncertain, and you should read actual state before replaying taps or submissions.
- Data lives under
~/.local/share/iphone-use/(overridable viaIPHONE_USE_STATE_DIR), with 700 directory and 600 config/image permissions. Preview streams are not written to disk.WDA_URLonly accepts local addresses; the default forwarding port is 18100. - By default the plugin sends anonymous usage analytics to PostHog and GA4 (usage start, tool calls, connection state, timing, error class, explicitly launched app identifiers and public directory names). Disable with
IPHONE_USE_ANALYTICS=0orDO_NOT_TRACK=1and restart the MCP service; details in ANALYTICS.md. - MIT licensed; third-party notices are in THIRD_PARTY_NOTICES.md.
HotRepo's take
There are no HotRepo ranking or star-growth records for this repository, so there's no trend to read from the site data. The useful signal is the design itself: it wraps Appium's WDA in an MCP layer and a setup skill so an assistant can drive a real iPhone without writing automation code, and it explicitly documents its analytics, state directory permissions and local-only WDA forwarding. That transparency, plus the screenshot-fallback approach for failed control lookups, makes it easier to evaluate before you invest in signing setup. The main cost is real: you need a Mac with full Xcode, an Apple development team, and a USB phone, so this is not a drop-in tool for everyone.
Written by AI from the project's README, updated . The project's own docs are the source of truth.
You might also like
- JosssphZhou/pokiLet Claude Code, Codex, or other MCP clients drive a real iPhone from a Mac, without jailbreak or extra hardware.
- phoneuse-official/phoneuseOpen-source tools to drive a USB-connected iPhone from a Mac with prompts, live video, REST and MCP.
- ZSeven-W/dsh-iosGives coding agents 22 tools to boot, build, and drive iOS Simulators or USB-connected iPhones, with a live interactive sidebar.
- phone-harness/mcpA hosted MCP server that lets AI agents control real iPhones and Android emulators in the cloud.
- DIYgod/pinkyA desktop gateway that lets any MCP-capable AI agent operate a real iPhone or Android phone under human approval.