Find your next open-source project

Search by name, purpose, or keyword — or paste a GitHub repository URL

AcidGr/agent-mobile-use

AI Engineering

Runs Android apps on a background virtual display so an agent can operate them without touching the main screen.

124
13
Java
MIT
2 months old

What it does

agent-mobile-use is a Mobile Use backend for DeepSeek Harness (DSH). It creates a privileged virtual display on an Android device so an agent can launch apps, read the accessibility tree, send taps, swipes, key events and silent text input, and capture frames — all in a background display while the physical main screen remains usable.

It has four cooperating parts: a CLI bus installed at /system/bin/vd, a Java daemon (agent_vd.dex) that manages the virtual display and frame cache, an accessibility tool (agent_tools.dex) for UI dumps and keyboard-free text injection, and a statically linked ARM64 Go server on port 3070 exposing a REST API and an H.264 WebSocket stream. An LSPosed hook in system_server and com.android.systemui handles cross-display task focus, input-method isolation and Status Bar fluid-cloud filtering. A native SettingsActivity in agent_hook.apk provides status monitoring, DSH credentials and virtual-display switches.

It is aimed at people building or running DSH-based phone agents on rooted devices, not at ordinary app automation on stock Android.

Quick start

Prerequisites: a rooted Android device (KernelSU, APatch or Magisk), LSPosed or Vector 2.x installed, and DSH available locally if you want the console integration. The author's tested setup is a ColorOS 16 / Android 15–16 device with KernelSU and LSPosed.

  1. Download agent-mobile-use-ksu-v0.8.11.zip from the GitHub Releases page or the release/ directory.
  2. Copy the zip to the phone and install it through the KernelSU (or APatch/Magisk) module manager.
  3. The install script extracts agent_hook.apk, detects LSPosed/Vector databases and activates the system, android, com.android.systemui and com.agent.mobileuse scopes, and deploys vd to /system/bin/vd with links under the KSU and AP trees.
  4. Reboot so the LSPosed hook and system services load.
  5. Verify with vd status, start the virtual display with vd start, then drive an app:
vd launch com.example.app
vd tree
vd tap 540 1200
vd type "hello"
vd screenshot /sdcard/shot.jpg

The same actions are available over HTTP on 127.0.0.1:3070, with GET /api/status, GET /api/dump_ui, and POST /api/action for combined observe/click/swipe/type/key/launch/wait steps. A live H.264 view is served at the root of the local web console. The source build is split across vd-tool-java/, vd-server-go/, agent-hook-apk/ and ksu-module/; each has its own build.sh, and ksu-module/pack.sh assembles the module zip.

Usage constraints

  • Root and a hook framework are mandatory; without LSPosed the cross-display scheduling, input-method isolation and Status Bar capsule features will not work. Android 16 QPR2 / 17 users are directed to LSPosed 2.x or Vector.
  • The gateway binds to 0.0.0.0:3070, so expose it only on trusted networks or firewall it; DSH console integration additionally assumes a local service on port 3080.
  • The LSPosed scopes include system_server, and installation rewrites system paths, so a failed module can affect the whole device; keep a way to uninstall the module.
  • App launching via --user relies on multi-user/secondary-profile behavior and is only verified in the author's environment.
  • The README's 105-step drawing test is an author-reported demonstration on that test device, not a general performance guarantee.
  • MIT licensed.

HotRepo's take

No ranking snapshots or discussion threads are recorded for this repository, so there is no trend data to read. The notable technical point is the virtual-display-plus-hook design: it keeps agent work off the physical screen and avoids soft-keyboard popups, which matters for continuous gesture streams. That design also makes the project inseparable from a rooted, LSPosed-hooked device, so the modest star count says little about adoption on stock Android.

The core deliverable is a prebuilt KernelSU zip and APK whose build relationship to the repository is described only by directory-level scripts, so users should confirm provenance before flashing.

Written by AI from the project's README, updated . The project's own docs are the source of truth.

You might also like

  1. kilolonion/agent-vscreenRun Android apps on an invisible virtual display so phone AI agents can operate them while you keep using your own screen.Kotlin117
  2. ybblab/DroidanceAn open-source Android agent that sees the screen and taps through apps for you in a background virtual display.Kotlin151
  3. guzhou079-arch/deepseek-harness-androidAn Android APK that bundles DeepSeek Harness with screen automation, virtual display, scheduled tasks and 27 skills, without Termux or root.JavaScript110
  4. minitap-ai/mobile-useAn LLM-driven agent that operates Android and iOS apps via natural language commands and returns structured data.Python3,403
  5. droidrun/mobilerunAn LLM-agnostic framework that lets AI agents inspect, tap, swipe, and type on real Android and iOS devices via CLI or Python.Python10,059