Skill240 repo starsupdated 20d ago
testdriver:elements
Locate and interact with UI elements using AI
Install in Claude Code
Copygit clone --depth 1 https://github.com/testdriverai/testdriverai /tmp/testdriver-elements && cp -r /tmp/testdriver-elements/ai/skills/testdriver:elements ~/.claude/skills/testdriver-elementsThen start a new Claude Code session; the skill loads automatically.
Definition
SKILL.md
<!-- Generated from elements.mdx. DO NOT EDIT. -->
## Overview
TestDriver's element finding system uses AI to locate elements on screen using natural language descriptions. The `find()` method returns an `Element` object that you can interact with.
## Finding Elements
### find()
Locate an element on screen using a natural language description.
```javascript
const element = await testdriver.find(description)
```
**Parameters:**
- `description` (string) - Natural language description of the element to find
**Returns:** `Promise<Element>` - Element instance that has been located
**Example:**
```javascript
// Find a button
const submitButton = await testdriver.find('the submit button');
// Find an input field with context
const emailField = await testdriver.find('email input field in the login form');
// Find an element by visual characteristics
const redButton = await testdriver.find('red button in the top right corner');
```
<Tip>
Be specific in your descriptions. Include visual details, location context, or nearby text to improve accuracy.
</Tip>
## Element Class
The `Element` class represents a located (or to-be-located) UI element. It provides methods for interaction and properties for element information.
### Methods
#### found()
Check if the element was successfully located.
```javascript
element.found()
```
**Returns:** `boolean` - True if element coordinates were found
**Example:**
```javascript
const element = await testdriver.find('login button');
if (element.found()) {
await element.click();
} else {
console.log('Element not found');
}
```
#### find()
Re-locate the element, optionally with a new description.
```javascript
await element.find(newDescription)
```
**Parameters:**
- `newDescription` (string, optional) - New description to search for
**Returns:** `Promise<Element>` - This element instance
**Example:**
```javascript
// Re-locate if the UI changed
const element = await testdriver.find('submit button');
// ... page updates ...
await element.find(); // Re-locate with same description
// Or update the description
await element.find('blue submit button'); // Now looking for blue button
```
#### click()
Click on the element.
```javascript
await element.click(action)
```
**Parameters:**
- `action` (string, optional) - Type of click: `'click'` (default), `'double-click'`, `'right-click'`, `'hover'`, `'mouseDown'`, `'mouseUp'`
**Returns:** `Promise<void>`
**Example:**
```javascript
const button = await testdriver.find('submit button');
await button.click(); // Regular click
const file = await testdriver.find('document.txt');
await file.click('double-click'); // Double-click
const menu = await testdriver.find('settings icon');
await menu.click('right-click'); // Right-click
```
<Note>
The element must be found before clicking. The `find()` method automatically locates the element.
</Note>
#### hover()
Hover over the element without clicking.
```javascript
await element.hover()
```
**Returns:** `Promise<void>`
**Example:**
```javascript
const tooltip = await testdriver.find('info icon');
await tooltip.hover();
// Wait to see tooltip
await new Promise(resolve => setTimeout(resolve, 1000));
```
#### doubleClick()
Double-click on the element.
```javascript
await element.doubleClick()
```
**Returns:** `Promise<void>`
**Example:**
```javascript
const file = await testdriver.find('README.txt file icon');
await file.doubleClick();
```
#### rightClick()
Right-click on the element to open context menu.
```javascript
await element.rightClick()
```
**Returns:** `Promise<void>`
**Example:**
```javascript
const folder = await testdriver.find('Documents folder');
await folder.rightClick();
```
#### mouseDown() / mouseUp()
Press or release mouse button on the element (for drag operations).
```javascript
await element.mouseDown()
await element.mouseUp()
```
**Returns:** `Promise<void>`
**Example:**
```javascript
// Drag and drop
const item = await testdriver.find('draggable item');
await item.mouseDown();
// Move to drop target (using coordinates or another element)
const target = await testdriver.find('drop zone');
await target.hover();
await target.mouseUp();
```
### Properties
Element properties provide additional information about located elements. Properties are available after a successful `find()` call.
#### coordinates
Get the element's coordinates object containing all position information.
```javascript
const coords = element.getCoordinates()
// or access directly
element.coordinates
```
**Returns:** `Object | null` - Coordinate object with `{ x, y, centerX, centerY }`
**Example:**
```javascript
const button = await testdriver.find('submit button');
const coords = button.coordinates;
if (coords) {
console.log(`Top-left: (${coords.x}, ${coords.y})`);
console.log(`Center: (${coords.centerX}, ${coords.centerY})`);
}
```
#### x, y, centerX, centerY
Direct access to coordinate values. Always available after successful `find()`.
```javascript
element.x // Top-left X coordinate (number)
element.y // Top-left Y coordinate (number)
element.centerX // Center X coordinate (number)
element.centerY // Center Y coordinate (number)
```
**Example:**
```javascript
const button = await testdriver.find('submit button');
console.log(`Button at: (${button.x}, ${button.y})`);
console.log(`Button center: (${button.centerX}, ${button.centerY})`);
// Use for custom mouse operations
await testdriver.click(button.centerX, button.centerY);
```
#### width, height
Element dimensions in pixels. Available when AI detects element bounds.
```javascript
element.width // Width in pixels (number | null)
element.height // Height in pixels (number | null)
```
**Example:**
```javascript
const button = await testdriver.find('submit button');
if (button.width && button.height) {
console.log(`Button size: ${button.width}x${button.height}px`);
// Check if button is large enough
if (button.width < 50) {
console.waMore from this repository
testdriver:aiSkill
Execute natural language tasks using AI
testdriver:assertSkill
Make AI-powered assertions about screen state
testdriver:aws-setupSkill
Deploy TestDriver on your AWS infrastructure using CloudFormation
testdriver:cacheSkill
Speed up tests with screenshot-based caching
testdriver:cachingSkill
1.7x faster test execution with intelligent caching and optimization
testdriver:captchaSkill
Solve captchas using 2captcha service
testdriver:ci-cdSkill
Run TestDriver tests in CI/CD with parallel execution and cross-platform support
testdriver:clickSkill
Click at specific coordinates or on elements