Skip to main content
ClaudeWave
Skill240 repo starsupdated 20d ago

testdriver:elements

Locate and interact with UI elements using AI

Install in Claude Code
Copy
git clone --depth 1 https://github.com/testdriverai/testdriverai /tmp/testdriver-elements && cp -r /tmp/testdriver-elements/ai/skills/testdriver:elements ~/.claude/skills/testdriver-elements
Then start a new Claude Code session; the skill loads automatically.

SKILL.md

<!-- Generated from elements.mdx. DO NOT EDIT. -->

## Overview

TestDriver's element finding system uses AI to locate elements on screen using natural language descriptions. The `find()` method returns an `Element` object that you can interact with.

## Finding Elements

### find()

Locate an element on screen using a natural language description.

```javascript
const element = await testdriver.find(description)
```

**Parameters:**
- `description` (string) - Natural language description of the element to find

**Returns:** `Promise<Element>` - Element instance that has been located

**Example:**
```javascript
// Find a button
const submitButton = await testdriver.find('the submit button');

// Find an input field with context
const emailField = await testdriver.find('email input field in the login form');

// Find an element by visual characteristics
const redButton = await testdriver.find('red button in the top right corner');
```

<Tip>
  Be specific in your descriptions. Include visual details, location context, or nearby text to improve accuracy.
</Tip>

## Element Class

The `Element` class represents a located (or to-be-located) UI element. It provides methods for interaction and properties for element information.

### Methods

#### found()

Check if the element was successfully located.

```javascript
element.found()
```

**Returns:** `boolean` - True if element coordinates were found

**Example:**
```javascript
const element = await testdriver.find('login button');
if (element.found()) {
  await element.click();
} else {
  console.log('Element not found');
}
```

#### find()

Re-locate the element, optionally with a new description.

```javascript
await element.find(newDescription)
```

**Parameters:**
- `newDescription` (string, optional) - New description to search for

**Returns:** `Promise<Element>` - This element instance

**Example:**
```javascript
// Re-locate if the UI changed
const element = await testdriver.find('submit button');
// ... page updates ...
await element.find(); // Re-locate with same description

// Or update the description
await element.find('blue submit button'); // Now looking for blue button
```

#### click()

Click on the element.

```javascript
await element.click(action)
```

**Parameters:**
- `action` (string, optional) - Type of click: `'click'` (default), `'double-click'`, `'right-click'`, `'hover'`, `'mouseDown'`, `'mouseUp'`

**Returns:** `Promise<void>`

**Example:**
```javascript
const button = await testdriver.find('submit button');
await button.click(); // Regular click

const file = await testdriver.find('document.txt');
await file.click('double-click'); // Double-click

const menu = await testdriver.find('settings icon');
await menu.click('right-click'); // Right-click
```

<Note>
  The element must be found before clicking. The `find()` method automatically locates the element.
</Note>

#### hover()

Hover over the element without clicking.

```javascript
await element.hover()
```

**Returns:** `Promise<void>`

**Example:**
```javascript
const tooltip = await testdriver.find('info icon');
await tooltip.hover();
// Wait to see tooltip
await new Promise(resolve => setTimeout(resolve, 1000));
```

#### doubleClick()

Double-click on the element.

```javascript
await element.doubleClick()
```

**Returns:** `Promise<void>`

**Example:**
```javascript
const file = await testdriver.find('README.txt file icon');
await file.doubleClick();
```

#### rightClick()

Right-click on the element to open context menu.

```javascript
await element.rightClick()
```

**Returns:** `Promise<void>`

**Example:**
```javascript
const folder = await testdriver.find('Documents folder');
await folder.rightClick();
```

#### mouseDown() / mouseUp()

Press or release mouse button on the element (for drag operations).

```javascript
await element.mouseDown()
await element.mouseUp()
```

**Returns:** `Promise<void>`

**Example:**
```javascript
// Drag and drop
const item = await testdriver.find('draggable item');
await item.mouseDown();

// Move to drop target (using coordinates or another element)
const target = await testdriver.find('drop zone');
await target.hover();
await target.mouseUp();
```

### Properties

Element properties provide additional information about located elements. Properties are available after a successful `find()` call.

#### coordinates

Get the element's coordinates object containing all position information.

```javascript
const coords = element.getCoordinates()
// or access directly
element.coordinates
```

**Returns:** `Object | null` - Coordinate object with `{ x, y, centerX, centerY }`

**Example:**
```javascript
const button = await testdriver.find('submit button');
const coords = button.coordinates;

if (coords) {
  console.log(`Top-left: (${coords.x}, ${coords.y})`);
  console.log(`Center: (${coords.centerX}, ${coords.centerY})`);
}
```

#### x, y, centerX, centerY

Direct access to coordinate values. Always available after successful `find()`.

```javascript
element.x        // Top-left X coordinate (number)
element.y        // Top-left Y coordinate (number)
element.centerX  // Center X coordinate (number)
element.centerY  // Center Y coordinate (number)
```

**Example:**
```javascript
const button = await testdriver.find('submit button');
console.log(`Button at: (${button.x}, ${button.y})`);
console.log(`Button center: (${button.centerX}, ${button.centerY})`);

// Use for custom mouse operations
await testdriver.click(button.centerX, button.centerY);
```

#### width, height

Element dimensions in pixels. Available when AI detects element bounds.

```javascript
element.width   // Width in pixels (number | null)
element.height  // Height in pixels (number | null)
```

**Example:**
```javascript
const button = await testdriver.find('submit button');

if (button.width && button.height) {
  console.log(`Button size: ${button.width}x${button.height}px`);
  
  // Check if button is large enough
  if (button.width < 50) {
    console.wa