Android, for AI coding agents
ShadowDroid is a CLI that lets Claude Code, Codex, Cursor, Gemini and any shell-capable agent drive, inspect and debug real Android apps, emulators and devices. Every answer is compact JSON with the next step built in.
brew install andriyo/tap/shadowdroid
Works with Claude CodeCodexCursorGeminiany shell
ui dump, parsed JSON, end to endapp apply, state keptThe loop
Three commands cover what an agent does all day on a device. No screenshots to squint at, no logcat spelunking.
Parsed elements with stable selectors and tap points, from a service that is already warm on the device.
shadowdroid ui dump{"screen_hash":"c50fd462de4304be","interaction_hash":"i:178f2b73ce0e1447", "snapshot_state":"consistent","viewport":{"w":1080,"h":2424}, "current_app":{"package":"io.github.andriyo.shadowdroid.sample"}, "element_count":39, "elements":[ {"id":3,"text":"Recover the\nsilent relay.","tap":[336,777]}, {"id":30,"rid":"nav_mission","tap":[403,2256],"clickable":true, "handle":"i:178f2b73ce0e1447/e:3"}]}
Tap by selector, wait for the screen to settle and check the destination, all in one round trip.
shadowdroid ui tap --rid nav_mission --expect-text "Night Shift: Signal Recovery"{"type":"action","cmd":"tap","ok":true, "matched_element":{"id":30,"rid":"nav_mission","tap":[403,2256]}, "action":"accessibility_click","settle_ms":1878,"screen_changed":true, "postcondition":{"kind":"text","expected":"Night Shift: Signal Recovery", "matched":true}, "postcondition_satisfied":true, "screen":{"screen_hash":"87332662a59bfb79","element_count":45}}
One bounded read turns a crash, the logs, the screen and failed requests into a verdict.
shadowdroid why{"type":"action","cmd":"why","ok":true,"verdict":"app_crashed", "explanation":"the app process crashed — see evidence.crash", "evidence":{"crash":{"kind":"java", "exception":"java.lang.RuntimeException", "stack":["…MainActivity.crashNow(MainActivity.kt:240)"]}}, "next_actions":["shadowdroid log --last 5m", "shadowdroid app start"]}
Every response ends with machine-readable next_actions. If the app crashed since the last command, the next response carries the parsed crash, so the crash finds the agent.
What your agent gets
ShadowDroid keeps a persistent service on the device and an expertise layer on the host. The agent gets the answer, not raw material to parse.
Tap, type, swipe and scroll by text, id, description or XPath. --expect-* proves the destination, stale-screen guards refuse to act on a changed screen, and taps on covered elements are refused. Compose and Android TV included.
A missed selector returns what the screen actually shows and the closest matches. why fuses crash, logs, screen and network into one verdict. log is app-scoped JSON with stack frames mapped to your source.
Breakpoints, stacks, variables, watches, eval, logpoints and coroutines, over JDWP through adb. No Android Studio needed, including breakpoints in startup code. Method calls only run when you opt in.
The ShadowDroid runtime attaches to any debuggable app for the layout tree, Compose recomposition counts with source locations, and live coroutine dumps. The same mechanism as Studio's Layout Inspector.
A built-in MITM proxy captures HTTP/2, SSE, streaming bodies and WebSocket frames. Intercept and edit requests in flight, write rewrite rules, inject faults, export HAR or curl, and replay recorded backends.
app apply hot-swaps the classes that changed and recomposes only the composables you edited, keeping app state. When that isn't safe it restarts the activity or reinstalls, and says why.
Per-device ports, reservations and sessions let several agents work at once on different devices and projects, without stepping on each other.
Watch the emulator live in Claude Desktop. Point at an element and say what to change, or record your taps and let Claude turn them into a UI test.
Why not just adb?
Keep adb, Android Studio and the android CLI for building and deploying, and Espresso for regression suites. ShadowDroid takes over once the app is running.
| Instead of | The gap in an agent loop | ShadowDroid |
|---|---|---|
uiautomator dump | A new process per read, raw XML on /sdcard to pull and parse | Warm, parsed JSON with selectors in ~65 ms |
input tap | Blind coordinates that break on any layout change | Selector actions with postconditions and stale-screen guards |
logcat | An unscoped text firehose with crashes buried in noise | App-scoped JSON, parsed crashes and ANRs, frames mapped to source |
| Appium, Maestro, Espresso | A server, DSL or compiled tests between the agent and the device | One CLI the agent already knows how to call |
How it works
The Rust CLI owns orchestration: act-and-observe fusion, log parsing, crash events, source mapping and recovery. The on-device service answers UI reads and performs actions over loopback HTTP. Host-side evidence keeps working even if the device service is down.
doctor --fix diagnoses and repairs the pipe2.0 Coming next
app apply builds incrementally, picks hot swap, activity restart or reinstall, and reports what changed on screen, crashes and logged errors.
Every debug verb works without Android Studio. With Studio open, ShadowDroid uses it, including native debugging.
Layout Inspector-class data and coroutine dumps from any debuggable app. The in-app library is no longer needed.
A live device viewer with annotations and tap-to-test recording, plus a lean skill that keeps agents fast and cheap.
Install
You also need Android Platform Tools (adb) on your PATH. On first connect, ShadowDroid installs a version-matched, checksum-verified service on the device.
brew install andriyo/tap/shadowdroidcurl --proto '=https' --tlsv1.2 -LsSf https://github.com/andriyo/ShadowDroid/releases/latest/download/shadowdroid-installer.sh | shscoop bucket add andriyo https://github.com/andriyo/scoop-bucket; scoop install shadowdroid# list devices and emulators
$ shadowdroid devices
# install the device service and verify it
$ shadowdroid connect
# read the screen as JSON
$ shadowdroid ui dump
# act by selector, prove the outcome
$ shadowdroid ui tap --text "Sign in" --expect-text "Welcome"
# when something surprises you
$ shadowdroid why
# teach your agent the loop
$ shadowdroid initInstall ShadowDroid, run shadowdroid init, and ask your agent to try the app.