Find your next open-source project

Search by name, purpose, or keyword — or paste a GitHub repository URL

phoneuse-official/phoneuse

AI Engineering

Open-source tools to drive a USB-connected iPhone from a Mac with prompts, live video, REST and MCP.

132
16
Python
MIT
20 days old

PhoneUse Local

PhoneUse Local is an experimental developer preview for controlling a USB-connected iPhone from a Mac. A local web page at localhost:3109 sends prompts to a local Python API, which asks the Codex CLI for the next action and executes it through Appium's WebDriverAgent on the phone. It also exposes a REST API and an MCP server so your own scripts or AI agents can list phones, take screenshots, read on-screen elements, tap, type, and open or manage apps source:1.

The project targets developers who want to experiment with phone-use agents on their own hardware and are willing to handle Xcode signing and a cable connection. It ships no hosted phones, no pre-signed IPA, and no production infrastructure; the hosted product at phoneuse.com is a separate deployment with separate credentials source:2.

Quick start

Prerequisites: macOS with full Xcode, an unlocked and trusted iPhone supported by that Xcode, Developer Mode, a legitimately signed WebDriverAgentRunner, Python 3.13, Node.js 24+, npm, Git, and a signed-in Codex CLI source:23.

  1. From the repository folder, run ./"Install PhoneUse.command". It creates the Python environment, installs backend dependencies, and builds the web app.
  2. Enable Developer Mode on the iPhone, then get WebDriverAgent from Appium's WebDriverAgent repository and follow Appium's real-device setup. In Xcode, select the WebDriverAgentRunner target, pick your signing team, choose the iPhone as destination, run Product → Test, and keep it running source:26.
  3. In a Terminal, run ./platform/start-api.sh and leave it open. http://127.0.0.1:8765/health should return "ok":true source:30.
  4. In a second Terminal, run ./"Start PhoneUse.command" to open http://localhost:3109/ with the API key already wired in source:29.
  5. In the Your iPhone panel, select the phone and wait for moving live video. The badge should read Ready; View only means WDA is not reachable.

For prompts, sign in once with codex login; if the API cannot find the CLI, restart it with PHONEUSE_CODEX_BIN="$(command -v codex)" ./platform/start-api.sh source:33. Start with a reversible prompt such as “Open Safari and describe the current screen. Do not submit anything.”

Integrating with code and agents

The REST API exposes interactive docs at http://127.0.0.1:8765/docs; every request needs Authorization: Bearer <key>. To connect an MCP client, point it at the absolute path of platform/start-mcp.sh; the MCP layer provides manual phone controls and does not need Codex. Ready-to-copy configs for Claude Code, Claude Desktop, Cursor, and Codex are in the integration guide. A script can get a device-scoped key from create-key.py source:38.

Constraints and data handling

  • macOS-only; Windows and Linux are not supported for the phone connection path.
  • Requires Xcode signing and Developer Mode with the WDA test kept running. PhoneUse cannot bypass Apple's signing steps.
  • The model integration was exercised with Codex CLI 0.154.0; its structured-output flags may change, and model access is bounded by your own account.
  • The local API and the web app use fixed ports 8765 and 3109. Another running PhoneUse install on those ports will be picked up by the launcher.
  • The API can tap, type, install and remove apps, and read the screen. It is bound to loopback and checks browser origins, but the bearer key is the control for non-browser local programs; it is stored at platform/.state/admin-token and should never be pasted into issues or chats source:36.
  • Prompt tasks send the task text plus screen text and accessibility data to the model provider through your Codex login; screenshots go only when you tick Use screenshots if controls cannot be found. This is not fully offline inference.
  • The agent acts on explicit instructions and is not a security sandbox: it can send messages, make purchases, delete data, or change settings when asked. Keep important results checked on the phone.
  • Apps with icon-only or custom-drawn controls may fail because there is no text or accessibility label to target.
  • Source is MIT; dependencies keep their own licenses source:44.

HotRepo's take

The HotRepo record has no daily-rank or snapshot history for this repository, so no trend claim can be made from the site data. What the materials do show is a real implementation surface behind the demos: a Python backend, a local web UI, an agent loop through Codex CLI, WDA-based device control, and an MCP entry point, with a CI workflow and a published setup/troubleshooting path. The honest framing matters more than the star count: this is a developer preview that requires Xcode signing and a live WDA session, and it explicitly says the agent is not a sandbox. It fits developers evaluating phone-use agents on their own desk, not users looking for a plug-and-play hosted phone service.

Written by AI from the project's README, updated . The project's own docs are the source of truth.

You might also like

  1. ZSeven-W/dsh-iosGives coding agents 22 tools to boot, build, and drive iOS Simulators or USB-connected iPhones, with a live interactive sidebar.TypeScript469
  2. JosssphZhou/pokiLet Claude Code, Codex, or other MCP clients drive a real iPhone from a Mac, without jailbreak or extra hardware.JavaScript109
  3. zhongerxin/iPhone-useLet Codex, WorkBuddy or Doubao work drive a physical iPhone over USB through WDA and MCP tools.Python5,022
  4. phone-harness/mcpA hosted MCP server that lets AI agents control real iPhones and Android emulators in the cloud.Python812
  5. lxe/phoneuseMirror an unmodified iPhone on Linux via AirPlay and control it with Bluetooth mouse and keyboard or a CLI.Python102