AcidGr/agent-mobile-use
AI EngineeringRuns Android apps on a background virtual display so an agent can operate them without touching the main screen.
What it does
agent-mobile-use is a Mobile Use backend for DeepSeek Harness (DSH). It creates a privileged virtual display on an Android device so an agent can launch apps, read the accessibility tree, send taps, swipes, key events and silent text input, and capture frames — all in a background display while the physical main screen remains usable.
It has four cooperating parts: a CLI bus installed at /system/bin/vd, a Java daemon (agent_vd.dex) that manages the virtual display and frame cache, an accessibility tool (agent_tools.dex) for UI dumps and keyboard-free text injection, and a statically linked ARM64 Go server on port 3070 exposing a REST API and an H.264 WebSocket stream. An LSPosed hook in system_server and com.android.systemui handles cross-display task focus, input-method isolation and Status Bar fluid-cloud filtering. A native SettingsActivity in agent_hook.apk provides status monitoring, DSH credentials and virtual-display switches.
It is aimed at people building or running DSH-based phone agents on rooted devices, not at ordinary app automation on stock Android.
Quick start
Prerequisites: a rooted Android device (KernelSU, APatch or Magisk), LSPosed or Vector 2.x installed, and DSH available locally if you want the console integration. The author's tested setup is a ColorOS 16 / Android 15–16 device with KernelSU and LSPosed.
- Download
agent-mobile-use-ksu-v0.8.11.zipfrom the GitHub Releases page or therelease/directory. - Copy the zip to the phone and install it through the KernelSU (or APatch/Magisk) module manager.
- The install script extracts
agent_hook.apk, detects LSPosed/Vector databases and activates thesystem,android,com.android.systemuiandcom.agent.mobileusescopes, and deploysvdto/system/bin/vdwith links under the KSU and AP trees. - Reboot so the LSPosed hook and system services load.
- Verify with
vd status, start the virtual display withvd start, then drive an app:
vd launch com.example.app
vd tree
vd tap 540 1200
vd type "hello"
vd screenshot /sdcard/shot.jpg
The same actions are available over HTTP on 127.0.0.1:3070, with GET /api/status, GET /api/dump_ui, and POST /api/action for combined observe/click/swipe/type/key/launch/wait steps. A live H.264 view is served at the root of the local web console. The source build is split across vd-tool-java/, vd-server-go/, agent-hook-apk/ and ksu-module/; each has its own build.sh, and ksu-module/pack.sh assembles the module zip.
Usage constraints
- Root and a hook framework are mandatory; without LSPosed the cross-display scheduling, input-method isolation and Status Bar capsule features will not work. Android 16 QPR2 / 17 users are directed to LSPosed 2.x or Vector.
- The gateway binds to
0.0.0.0:3070, so expose it only on trusted networks or firewall it; DSH console integration additionally assumes a local service on port 3080. - The LSPosed scopes include
system_server, and installation rewrites system paths, so a failed module can affect the whole device; keep a way to uninstall the module. - App launching via
--userrelies on multi-user/secondary-profile behavior and is only verified in the author's environment. - The README's 105-step drawing test is an author-reported demonstration on that test device, not a general performance guarantee.
- MIT licensed.
HotRepo's take
No ranking snapshots or discussion threads are recorded for this repository, so there is no trend data to read. The notable technical point is the virtual-display-plus-hook design: it keeps agent work off the physical screen and avoids soft-keyboard popups, which matters for continuous gesture streams. That design also makes the project inseparable from a rooted, LSPosed-hooked device, so the modest star count says little about adoption on stock Android.
The core deliverable is a prebuilt KernelSU zip and APK whose build relationship to the repository is described only by directory-level scripts, so users should confirm provenance before flashing.
Written by AI from the project's README, updated . The project's own docs are the source of truth.
You might also like
- kilolonion/agent-vscreenRun Android apps on an invisible virtual display so phone AI agents can operate them while you keep using your own screen.
- ybblab/DroidanceAn open-source Android agent that sees the screen and taps through apps for you in a background virtual display.
- guzhou079-arch/deepseek-harness-androidAn Android APK that bundles DeepSeek Harness with screen automation, virtual display, scheduled tasks and 27 skills, without Termux or root.
- minitap-ai/mobile-useAn LLM-driven agent that operates Android and iOS apps via natural language commands and returns structured data.
- droidrun/mobilerunAn LLM-agnostic framework that lets AI agents inspect, tap, swipe, and type on real Android and iOS devices via CLI or Python.