chrome-automation

# Chrome Browser Automation with Real-time Control

Control the user's Google Chrome browser using agent-browser connected via Chrome DevTools Protocol (CDP). All actions are visible in real-time in the user's browser window.

## ⚠️ CRITICAL: Automatic Installation & Dependency Check

**BEFORE using this skill, ALWAYS run the auto-installation check:**

This skill automatically checks and installs all required dependencies without user confirmation. The installation process verifies:
- agent-browser repository and binary
- Node.js and pnpm
- Playwright Chromium
- Rust toolchain (cargo)
- Native binary compilation

**Auto-installation workflow:**

```bash
# Mac: Run auto-install script
bash ~/.claude/skills/chrome-automation/scripts/auto-install-mac.sh

# Windows: Run auto-install script (PowerShell)
powershell -ExecutionPolicy Bypass -File "$HOME\.claude\skills\chrome-automation\scripts\auto-install-windows.ps1"
```

**The script will:**
1. Check if agent-browser binary exists and works
2. Install missing dependencies automatically (pnpm, Rust, etc.)
3. Clone repository if not present
4. Build native binary with proper Rust toolchain
5. Verify installation success

**Critical rules:**
- ❌ DO NOT fall back to using built-in browser tools (browser_use_desktop)
- ✅ MUST run auto-install script before any browser automation
- ✅ Only proceed with browser automation AFTER successful installation
- ✅ Script handles all dependency checks automatically

**Why agent-browser is required:**
- Provides real-time visual feedback in user's actual Chrome browser
- Allows user to see and interact with automation in progress
- Offers precise element selection and interaction capabilities
- Built-in browser tools are NOT a substitute for this skill's functionality

## Quick Start

### First-time setup (automatic)

**Run the auto-install script first:**

```bash
# Mac
bash ~/.claude/skills/chrome-automation/scripts/auto-install-mac.sh

# Windows (PowerShell)
powershell -ExecutionPolicy Bypass -File "$HOME\.claude\skills\chrome-automation\scripts\auto-install-windows.ps1"
```

The script automatically handles:
- Repository cloning
- Dependency installation (pnpm, Playwright)
- Rust toolchain setup
- Native binary compilation

### Connect and control Chrome

**IMPORTANT: Use the start-chrome script to preserve login states across sessions**

```bash
# Mac: Start Chrome with persistent profile (RECOMMENDED)
# This preserves your login states, cookies, and settings
bash ~/.claude/skills/chrome-automation/scripts/start-chrome-mac.sh

# Windows: Start Chrome with persistent profile (PowerShell)
powershell -ExecutionPolicy Bypass -File "$HOME\.claude\skills\chrome-automation\scripts\start-chrome-windows.ps1"

# Use agent-browser with --cdp flag
cd /Users/jyxc-dz-0100272/Documents/agent-browser
AGENT_BROWSER_HOME=/Users/jyxc-dz-0100272/Documents/agent-browser \
./bin/agent-browser --cdp 9222 snapshot -i
```

**What the start-chrome script does:**
1. First run: Creates a dedicated automation profile at `~/Library/Application Support/Google/Chrome-Automation`
2. First run: Imports your existing Chrome configuration (login states, cookies, settings)
3. Subsequent runs: Uses the existing automation profile - all your logins are preserved
4. You only need to log in once - it will be remembered for future sessions

## Core Workflow

**CRITICAL: Always show browser state to user after each significant action using screenshots.**

1. **Connect**: Ensure Chrome running with remote debugging port 9222
2. **Navigate**: Open pages using `--cdp 9222` flag
3. **Snapshot**: Get interactive elements with `snapshot -i`
4. **Interact**: Use element refs (@e1, @e2, etc.)
5. **Screenshot**: Capture state and show to user
6. **Report**: Describe what's visible in browser

### Standard operation pattern

```bash
# Set path variables
export AB_HOME=/Users/jyxc-dz-0100272/Documents/agent-browser
cd $AB_HOME

# Navigate
AGENT_BROWSER_HOME=$AB_HOME ./bin/agent-browser --cdp 9222 open https://example.com
sleep 2

# Get interactive elements
AGENT_BROWSER_HOME=$AB_HOME ./bin/agent-browser --cdp 9222 snapshot -i

# Screenshot to show user
AGENT_BROWSER_HOME=$AB_HOME ./bin/agent-browser --cdp 9222 screenshot /tmp/state.png

# Interact
AGENT_BROWSER_HOME=$AB_HOME ./bin/agent-browser --cdp 9222 fill @e1 "text"
AGENT_BROWSER_HOME=$AB_HOME ./bin/agent-browser --cdp 9222 press Enter

# Capture result
sleep 2
AGENT_BROWSER_HOME=$AB_HOME ./bin/agent-browser --cdp 9222 screenshot /tmp/result.png
```

## Installation & Setup

### Automatic Installation (RECOMMENDED)

**Always use the auto-install script - it handles all dependencies automatically:**

#### Mac Installation

```bash
# Run auto-install script
bash ~/.claude/skills/chrome-automation/scripts/auto-install-mac.sh
```

**What it does:**
1. Checks if agent-browser binary exists and works
2. Clones repository if missing
3. Installs Node.js dependencies (pnpm)
4. Installs Playwright Chromium
5. Checks/installs Rust toolchain (cargo, rustup)
6. Sets up stable Rust toolchain if needed
7. Builds native binary with `npm run build:native`
8. Verifies installation success

**Requirements:**
- Node.js (will notify if missing)
- Git (for cloning repository)
- Internet connection

#### Windows Installation

```powershell
# Run auto-install script (PowerShell)
powershell -ExecutionPolicy Bypass -File "$HOME\.claude\skills\chrome-automation\scripts\auto-install-windows.ps1"
```

**What it does:**
1. Checks if agent-browser binary exists and works
2. Clones repository if missing
3. Installs Node.js dependencies (pnpm)
4. Installs Playwright Chromium
5. Downloads and installs Rust toolchain if needed
6. Sets up stable Rust toolchain
7. Builds native binary with `npm run build:native`
8. Verifies installation success

**Requirements:**
- Node.js (will notify if missing - download from https://nodejs.org/)
- Git (for cloning repository)
- Internet connection

### Manual Installation (Advanced Users Only)

If you