looktap

<div align="center">

<img src="https://raw.githubusercontent.com/lakpriya1s/looktap/main/assets/looktap-logo.svg" alt="looktap — hands on the phone, for AI agents" width="560">

**Let AI agents drive iPhone and Android devices from the shell**<br>
Read the screen, tap, type, swipe, screenshot, and read logs and crashes. No server to start and no dependencies.

[![npm](https://img.shields.io/npm/v/@looktap/cli.svg?color=cb3837&logo=npm)](https://www.npmjs.com/package/@looktap/cli)
[![License: AGPL-3.0](https://img.shields.io/badge/License-AGPL--3.0-blue.svg)](https://github.com/lakpriya1s/looktap/blob/main/LICENSE)
[![Node ≥ 18](https://img.shields.io/badge/node-%E2%89%A518-339933.svg?logo=node.js&logoColor=white)](#requirements)
[![iOS simulator](https://img.shields.io/badge/iOS-simulator%20%7C%20device-000000.svg?logo=apple&logoColor=white)](#how-it-works)
[![Android](https://img.shields.io/badge/Android-emulator%20%7C%20device-3DDC84.svg?logo=android&logoColor=white)](#how-it-works)
[![Dependencies: 0](https://img.shields.io/badge/dependencies-0-16a34a.svg)](https://github.com/lakpriya1s/looktap/blob/main/package.json)

[Website](https://lakpriya1s.github.io/looktap/) · [Quick start](#quick-start) · [Commands](#commands) · [For AI agents](#for-ai-agents) · [Benchmark](BENCHMARK.md) · [Troubleshooting](#troubleshooting)

</div>

---

`looktap` is a small command-line tool for driving a mobile app on an iPhone or Android device, real or simulated.
Each command does one thing and prints a few lines of plain text:

<p align="center">
  <img src="https://raw.githubusercontent.com/lakpriya1s/looktap/main/assets/looktap-demo.gif" alt="An agent drives a demo app with looktap: it reads the sign-in screen as a list of labelled elements, taps, fills in an email address, and scrolls and zooms a map, while the phone beside the terminal reacts to each command" width="760">
</p>

That makes it a good fit for an AI coding agent (Claude Code, Cursor, Codex, opencode, …) that needs to **see and
use the app it's building**:
- **Small output.** A screen read is about 1 KB of text, where other agent tools return 4–125 KB. A screenshot is a
  file path, where most MCP servers put 40–270 KB of image into the context.
- **One call per step.** Add `--ls` to an action and its result comes back in the same call.
- **Fast.** Screen reads take about 0.1 s and taps 0.1–0.2 s on both platforms, and the first command comes back in
  under a second, with no session to set up. Against eight other agent tools on the same devices, it had the lowest
  agent time on both iPhone and Android ([How it compares](#how-it-compares)).

It's just as handy by hand, or in a shell script for screenshot sweeps and bug repros.

---

## Contents

- [What you get](#what-you-get): everything looktap does today
- [How it compares](#how-it-compares): the measured comparison with every other agent mobile tool
- [Quick start](#quick-start): install, boot a device, first commands
- [🤖 Kickstart with your AI agent](#-kickstart-with-your-ai-agent): one prompt and your agent takes it from there
- [Requirements](#requirements)
- [How it works](#how-it-works)
- [Examples](#examples): tap by label, forms, waiting, scrolling, debugging, bursts and video, several devices
- [For AI agents](#for-ai-agents)
- [Commands](#commands) · [Platform support](#platform-support) · [Options & config](#options-environment-variables-config-file)
- [Troubleshooting](#troubleshooting) · [FAQ](#faq) · [Roadmap](#roadmap)
- [Safety rules](#safety-rules) · [Contributing](#contributing) · [License](#license)

## What you get

| | What you get |
|---|---|
| 👀 | **Read the screen as text.** `ls` lists what's visible, one row per element: `@ref  type  x,y  w×h  label  = value`. It reads native screens, React Native screens and web content (Safari, in-app browsers, web views) |
| 👆 | **Tap by label.** Exact match first, then a prefix or substring that names exactly one label. An ambiguous match is an error, never a guess. Also by `@ref` from the last `ls`, or by `x y` coordinates |
| ⌨️ | **Type and fill.** `type` into the focused field, `fill` to clear a field and type, `key return\|escape\|delete\|tab\|selectall\|paste\|…` (the same names on both platforms), and `hidekb`, which never reports success while the keyboard is still up |
| 🧭 | **Navigate and gesture.** `swipe`, `scrollto <label>`, `back`, `home` (the Home button, the app keeps running), `open <deep link>`, `launch <app>`, `stop <app>`, `longpress`, and two-finger `zoom in\|out` |
| ⏱️ | **Wait instead of sleep.** `wait "Welcome"`, `wait "Loading" --gone`. Actions already wait up to 3 s for their target |
| 📸 | **Screenshots.** `shot` saves a PNG and prints its path, with `--scale 0.5` for fewer image tokens. On Android emulators it also captures `FLAG_SECURE` screens |
| 🎞️ | **Burst capture and video.** `burst` saves a frame each time the screen changes, to catch spinners, toasts and transitions. `record start` … `record stop` saves an MP4 of everything in between |
| 🪵 | **Logs.** `logs` shows the app's log since your last look. `--errors` and `--grep` narrow it, and on iOS Apple-framework chatter is filtered out |
| 💥 | **Crashes.** `crashes` reads iOS crash reports (exception, reason, top frames) and Android crashes, ANRs and **low-memory kills**, so "the app vanished" always comes with a reason |
| 🫆 | **Biometrics.** `bio match\|nomatch` simulates Face ID or a fingerprint |
| 🌍 | **Simulated conditions.** `location <lat> <lon>`, `appearance dark`, `textsize xxl` (Dynamic Type / font scale), `statusbar clean` (9:41 for screenshots), `rotate landscape` (`ls`, taps and screenshots follow the turned screen), Android `network offline`, and iOS `push` notifications |
| 🧹 | **App state for repros.** `install` a build, `reset` an app to a fresh install, `uninstall`, and `permission grant\|revoke\|reset camera\|location\|photos\|…`, with the same names on both platforms |
| 📱 | **Devices on demand.** `boot "iPhone 16 Pro"` or `boot Pixel_9a` starts a simulator or emulator by name and waits until it's ready, with a window or `--headless`; `shutdown` stops it. `devices` tells real phones from simulators and emulators (model, OS, USB or Wi-Fi), and says what to do for a phone that isn't ready yet |
| 🤖 | **Built for agents.** `--ls` on any action, `run` to batch steps, `--json`, `capabilities` to check what the device in use can and can't do, and a [Claude Code skill](#for-ai-agents) you install with one command |
| 🧩 | **Nothing to install on simulators, emulators or Android phones**, no server to run, no npm dependencies. The helpers start on first use. A real iPhone gets a small test runner, built for you |
| ⚠️ | **Honest failures.** A tap that's covered by a toast prints a warning, a stale `@ref` is an error rather than a wrong tap, a system prompt over the app is reported as one, a locked device is reported as locked (never typed into), and **an app that dies is named the moment looktap notices** |

## How it compares

Every agent mobile tool we could run on one Mac was given the same 12 steps in the Settings app, 10 runs each:
looktap (CLI and MCP), mobile-mcp, Maestro MCP, Appium MCP, raw Appium, ios-simulator-mcp, MobileBuildMCP, raw
idb and raw adb. A neutral checker decided whether each step worked. These are the totals over the steps every tool
can do:

| | **looktap CLI** | **looktap MCP** | Next best for an agent | Others |
|---|---|---|---|---|
| Agent time (iOS · Android)¹ | **21 s · 28 s** | 25 s · 37 s | 30 s Maestro MCP · 49 s Appium MCP | 36–63 s |
| Tool calls (iOS · Android) | **8** · 12 | **8** · 12 | 9 · **11** Maestro MCP | 10–27 |
| Text returned (iOS · Android) | **747 B · 1.2 KB** | 3.2 KB · 7.9 KB | 17 KB · 14 KB Maestro MCP | 17–154 KB |
| Steps passed | **100%** of what it can do | **100%** | | 87–100% |
| Setup before the first command | **under 1 s** | **under 1 s** | | 0.2–29 s |

¹ The device's reaction time plus 2 s per tool call for the model's round trip.

Where looktap loses:
- On raw device time, ios-simulator-mcp is faster in total on iOS (4.5 s against 5.2 s), with 20 calls to looktap's 8.
- The looktap MCP waits for the screen to settle after every action, which makes it slower per action than the CLI.
- looktap can't stop an app, and on iOS it has no home button.

Method, both platforms' tables and what else they show are in **[BENCHMARK.md](BENCHMARK.md)**.

**Where Appium is still the better choice:** it plugs into existing WebDriver test suites and CI grids, and supports
many more platforms. looktap isn't a test framework.

## Quick start

```bash
# 1. install (Node 18+; zero dependencies)
npm install -g @looktap/cli

# 2. check the toolchain
looktap doctor

# 3. boot a device: an iOS simulator or an Android emulator, by name
looktap devices --all                         # what's running, and what you can boot
looktap boot "iPhone 16 Pro"                  # or an AVD: looktap boot Pixel_9a

# 4. drive it
looktap launch com.apple.Preferences --ls     # (Android: com.android.settings)
looktap tap "General" --ls
looktap shot general
```

Or try it without installing anything: `npx @looktap/cli doctor` (any command works the same way).

To teach Claude Code how to use looktap, install the plugin. In Claude Code:

```
/plugin install looktap --marketplace lakpriya1s/looktap
```

It adds the looktap skill. Then run `/looktap:setup`: it installs the CLI, checks your
toolchain and gets a device ready. Or copy the skill in by hand:

```bash
looktap skill --install      # → ~/.claude/skills/looktap/SKILL.md
```

<details>
<summary><strong>Other ways to install</strong></summary>

The latest from GitHub, ahead of the npm release:

```bash
npm install -g github:lakpriya1s/looktap
```

From a checkout:

```bash
git clone https://github.com/lakpriya1s/looktap.git
cd looktap && npm install -g .
```

As a tarball you can hand to someone (it includes both prebuilt helpers):

```bash
npm pack