Skip to main content
ClaudeWave
Skill240 repo starsupdated 20d ago

testdriver:find

Locate UI elements using natural language

Install in Claude Code
Copy
git clone --depth 1 https://github.com/testdriverai/testdriverai /tmp/testdriver-find && cp -r /tmp/testdriver-find/ai/skills/testdriver:find ~/.claude/skills/testdriver-find
Then start a new Claude Code session; the skill loads automatically.

SKILL.md

<!-- Generated from find.mdx. DO NOT EDIT. -->

## Overview

Locate UI elements on screen using AI-powered natural language descriptions. Returns an `Element` object that can be interacted with.

## Syntax

```javascript
const element = await testdriver.find(description)
const element = await testdriver.find(description, options)
```

## Parameters

<ParamField path="description" type="string" required>
  Natural language description of the element to find
</ParamField>

<ParamField path="options" type="object | number">
  Optional configuration for finding and caching
  
  <Expandable title="properties">
    <ParamField path="cacheKey" type="string">
      Custom cache key for storing element location. Use this to prevent cache pollution when using dynamic variables in prompts, or to share cache across tests.
    </ParamField>
    
    <ParamField path="cacheThreshold" type="number" default={0.05}>
      Similarity threshold (0-1) for cache matching. Lower values require more similarity. Set to -1 to disable cache.
    </ParamField>
    
    <ParamField path="timeout" type="number" default={10000}>
      Maximum time in milliseconds to poll for the element. Retries every 5 seconds until found or timeout expires. Defaults to `10000` (10 seconds). Set to `0` to disable polling and make a single attempt.
    </ParamField>
    
    <ParamField path="confidence" type="number">
      Minimum confidence threshold (0-1). If the AI's confidence score for the found element is below this value, the find will be treated as a failure (`element.found()` returns `false`). Useful for ensuring high-quality matches in critical test steps.
    </ParamField>
    
    <ParamField path="type" type="string">
      Element type hint that wraps the description for better matching. Accepted values:
      - `"text"` — Wraps the prompt as `The text "..."`
      - `"image"` — Wraps the prompt as `The image "..."`
      - `"ui"` — Wraps the prompt as `The UI element "..."`
      - `"any"` — No wrapping, uses the description as-is (default behavior)
    </ParamField>
    
    <ParamField path="zoom" type="boolean" default={false}>
      Two-phase zoom mode for better precision in crowded UIs with many similar elements. Disabled by default.
    </ParamField>
    
    <ParamField path="verify" type="boolean" default={false}>
      Enable AI verification of the located element. When `true`, a second AI call checks that the coordinates returned actually correspond to the requested element, catching hallucinated or incorrect positions. Disabled by default for lower latency. Defaults to the global `verify` option set on the SDK constructor when not specified per call.
    </ParamField>
    
    <ParamField path="ai" type="object">
      AI sampling configuration for this find call (overrides global `ai` config from constructor).
      
      <Expandable title="properties">
        <ParamField path="temperature" type="number">
          Controls randomness. `0` = deterministic. Default: `0` for find verification.
        </ParamField>
        
        <ParamField path="top" type="object">
          Sampling parameters
          
          <Expandable title="properties">
            <ParamField path="p" type="number">
              Top-P (nucleus sampling). Range: 0-1.
            </ParamField>
            
            <ParamField path="k" type="number">
              Top-K sampling. `1` = most deterministic.
            </ParamField>
          </Expandable>
        </ParamField>
      </Expandable>
    </ParamField>
  </Expandable>
</ParamField>

## Returns

`Promise<Element>` - Element instance that has been automatically located

## Examples

### Basic Element Finding

```javascript
// Find by role
const button = await testdriver.find('submit button');
const input = await testdriver.find('email input field');

// Find by text content
const link = await testdriver.find('Contact Us link');
const heading = await testdriver.find('Welcome heading');

// Find by visual appearance
const icon = await testdriver.find('red warning icon');
const image = await testdriver.find('company logo image');
```

### Finding with Context

```javascript
// Provide location context
const field = await testdriver.find('username input in the login form');
const button = await testdriver.find('delete button in the top right corner');

// Describe nearby elements
const input = await testdriver.find('input field below the email label');
const checkbox = await testdriver.find('checkbox next to "Remember me"');

// Describe visual position
const menu = await testdriver.find('hamburger menu icon in the top left');
```

### Interacting with Found Elements

```javascript
// Find and click
const submitBtn = await testdriver.find('submit button');
await submitBtn.click();

// Find and verify
const message = await testdriver.find('success message');
if (message.found()) {
  console.log('Success message appeared');
}

// Find and extract info
const price = await testdriver.find('product price');
console.log('Price location:', price.coordinates);
console.log('Price text:', price.text);
```

## Element Object

The returned `Element` object provides:

### Methods

- `found()` - Check if element was located
- `click(action)` - Click the element
- `hover()` - Hover over the element
- `doubleClick()` - Double-click the element
- `rightClick()` - Right-click the element
- `find(newDescription)` - Re-locate with optional new description

### Properties

- `coordinates` - Element position `{x, y, centerX, centerY}`
- `x`, `y` - Top-left coordinates
- `centerX`, `centerY` - Center coordinates
- `text` - Text content (if available)
- `screenshot` - Base64 screenshot (if available)
- `confidence` - AI confidence score
- `width`, `height` - Element dimensions
- `boundingBox` - Complete bounding box

See [Elements Reference](/v7/elements) for complete details.

### JSON Serialization

Elements can be safely serialized using `JSON.stringify()` for logging and debugging. Cir