Find your next open-source project

Search by name, purpose, or keyword — or paste a GitHub repository URL

ananhq/anan-iphone-use

Hardware & IoT

ESP32-S3 firmware that turns an M5Stack StopWatch into an MCP server for sending tap, drag, scroll and keyboard input to an iPhone.

160
22
C
GPL-3.0
36 days old

What it does

This is ESP-IDF firmware for the M5Stack StopWatch (ESP32-S3) that exposes an HTTP MCP server. A phone-side or computer-side agent observes the iPhone screen and decides what to do; the StopWatch translates MCP tool calls into taps, drags, scrolling and keyboard input sent to the iPhone over USB or experimental BLE. The firmware runs no model and captures no screenshots — screen observation stays with the client.

The input tools are tap, drag, press_key, type_ascii, scroll, plus device_status and stop. Coordinates are portrait-normalized 0…1 and times are in milliseconds. The project is built around the Anan client, but any HTTP MCP client that can reach the USB subnet or Wi-Fi LAN should work; the bundled skill also covers adapting the firmware to other ESP32 boards.

Quick start

You need an M5Stack StopWatch, a computer and data cable for the first flash, and a USB-C iPhone with AssistiveTouch enabled.

git clone https://github.com/ananhq/anan-iphone-use.git
cd anan-iphone-use

Open your agent in the repository directory and ask it to use $anan-iphone-use to identify the board, flash the firmware and set up iPhone MCP. If the skill is not discovered, read the skill file first. Flashing and connection guidance is in the setup documentation.

After flashing, pick a connection:

  • Wireless: join the StopWatch setup hotspot, open http://192.168.4.1, confirm MCP pairing, select a 2.4 GHz Wi-Fi network, copy the MCP configuration before leaving, then rejoin your normal network and pair Anan StopWatch over BLE. Wireless setup and recovery covers both routes.
  • USB: connect the StopWatch to the iPhone, configure USB Ethernet at 192.168.7.2 with mask 255.255.255.0 and no router/DNS, then open http://192.168.7.1 to pair and import MCP.

Enable AssistiveTouch, then tap Start control on the device. The default MCP endpoint is http://stopwatch.local/mcp, with http://192.168.7.1/mcp as the USB fallback. Import the exported config including its Bearer header; switching between USB and BLE input does not require a new import.

Constraints

  • Hardware is limited to the M5Stack StopWatch (ESP32-S3) out of the box; other boards require adaptation.
  • Wi-Fi must be 2.4 GHz on an accessible local network. The setup hotspot pauses BLE and is not an Internet router; up to five networks are remembered.
  • Input is portrait, single-pointer and US-layout ASCII only; there is no direct Chinese text input.
  • BLE input is experimental — pointer accuracy, reconnection, client interoperability and endurance still need device acceptance. Build and offline results are tracked in validation.
  • Anan's system screen sharing requires iOS 27+.
  • The existing code is GPL-3.0-only; the avatar engine from the upstream StopWatch avatar project is AGPL-3.0-or-later, and the combined build follows GPLv3/AGPLv3 section 13.

HotRepo's take

The repository has no recorded GitHub daily ranking, star snapshots or Hacker News threads, so there is no trend data to read. The useful signal is structural: this is a narrow hardware target with a clear split of responsibilities — the agent supplies vision and decisions, the firmware only supplies input. That makes the MCP tool surface easy to audit (seven methods with documented parameters), but it also means the project's real utility depends on a client that can observe the iPhone screen, and the README names Anan as the primary one while its open-source release is still pending. Buyers of the StopWatch should weigh the experimental BLE path and the portrait/ASCII-only constraints before expecting general-purpose phone automation.

Written by AI from the project's README, updated . The project's own docs are the source of truth.

You might also like

  1. onehumanbeing/esp32-iphone-agentControl a real iPhone with a cheap ESP32 board acting as AirPlay receiver and USB HID input, driven by a vision model.C3,790
  2. 78/xiaozhi-esp32Firmware for ESP32 voice-assistant devices that connects cheap hardware to cloud LLMs, wake word included.C++31,875
  3. Liu-zhongxian/zongzhu-botA round-screen ESP32-S3 desk voice terminal that listens on-device, runs PC commands locally, and forwards other requests to a Grok bot.C++220
  4. m5stack/StackChanOpen-source firmware, controller, app and server resources for the M5Stack StackChan AI desktop robot.Go1,476
  5. socquique/TamaPokeAn open-firmware virtual pet game for one specific round ESP32-S3 AMOLED board, with a browser flashing tool.C++221