Skill240 repo starsupdated 20d ago
testdriver:find
Locate UI elements using natural language
Install in Claude Code
Copygit clone --depth 1 https://github.com/testdriverai/testdriverai /tmp/testdriver-find && cp -r /tmp/testdriver-find/ai/skills/testdriver:find ~/.claude/skills/testdriver-findThen start a new Claude Code session; the skill loads automatically.
Definition
SKILL.md
<!-- Generated from find.mdx. DO NOT EDIT. -->
## Overview
Locate UI elements on screen using AI-powered natural language descriptions. Returns an `Element` object that can be interacted with.
## Syntax
```javascript
const element = await testdriver.find(description)
const element = await testdriver.find(description, options)
```
## Parameters
<ParamField path="description" type="string" required>
Natural language description of the element to find
</ParamField>
<ParamField path="options" type="object | number">
Optional configuration for finding and caching
<Expandable title="properties">
<ParamField path="cacheKey" type="string">
Custom cache key for storing element location. Use this to prevent cache pollution when using dynamic variables in prompts, or to share cache across tests.
</ParamField>
<ParamField path="cacheThreshold" type="number" default={0.05}>
Similarity threshold (0-1) for cache matching. Lower values require more similarity. Set to -1 to disable cache.
</ParamField>
<ParamField path="timeout" type="number" default={10000}>
Maximum time in milliseconds to poll for the element. Retries every 5 seconds until found or timeout expires. Defaults to `10000` (10 seconds). Set to `0` to disable polling and make a single attempt.
</ParamField>
<ParamField path="confidence" type="number">
Minimum confidence threshold (0-1). If the AI's confidence score for the found element is below this value, the find will be treated as a failure (`element.found()` returns `false`). Useful for ensuring high-quality matches in critical test steps.
</ParamField>
<ParamField path="type" type="string">
Element type hint that wraps the description for better matching. Accepted values:
- `"text"` — Wraps the prompt as `The text "..."`
- `"image"` — Wraps the prompt as `The image "..."`
- `"ui"` — Wraps the prompt as `The UI element "..."`
- `"any"` — No wrapping, uses the description as-is (default behavior)
</ParamField>
<ParamField path="zoom" type="boolean" default={false}>
Two-phase zoom mode for better precision in crowded UIs with many similar elements. Disabled by default.
</ParamField>
<ParamField path="verify" type="boolean" default={false}>
Enable AI verification of the located element. When `true`, a second AI call checks that the coordinates returned actually correspond to the requested element, catching hallucinated or incorrect positions. Disabled by default for lower latency. Defaults to the global `verify` option set on the SDK constructor when not specified per call.
</ParamField>
<ParamField path="ai" type="object">
AI sampling configuration for this find call (overrides global `ai` config from constructor).
<Expandable title="properties">
<ParamField path="temperature" type="number">
Controls randomness. `0` = deterministic. Default: `0` for find verification.
</ParamField>
<ParamField path="top" type="object">
Sampling parameters
<Expandable title="properties">
<ParamField path="p" type="number">
Top-P (nucleus sampling). Range: 0-1.
</ParamField>
<ParamField path="k" type="number">
Top-K sampling. `1` = most deterministic.
</ParamField>
</Expandable>
</ParamField>
</Expandable>
</ParamField>
</Expandable>
</ParamField>
## Returns
`Promise<Element>` - Element instance that has been automatically located
## Examples
### Basic Element Finding
```javascript
// Find by role
const button = await testdriver.find('submit button');
const input = await testdriver.find('email input field');
// Find by text content
const link = await testdriver.find('Contact Us link');
const heading = await testdriver.find('Welcome heading');
// Find by visual appearance
const icon = await testdriver.find('red warning icon');
const image = await testdriver.find('company logo image');
```
### Finding with Context
```javascript
// Provide location context
const field = await testdriver.find('username input in the login form');
const button = await testdriver.find('delete button in the top right corner');
// Describe nearby elements
const input = await testdriver.find('input field below the email label');
const checkbox = await testdriver.find('checkbox next to "Remember me"');
// Describe visual position
const menu = await testdriver.find('hamburger menu icon in the top left');
```
### Interacting with Found Elements
```javascript
// Find and click
const submitBtn = await testdriver.find('submit button');
await submitBtn.click();
// Find and verify
const message = await testdriver.find('success message');
if (message.found()) {
console.log('Success message appeared');
}
// Find and extract info
const price = await testdriver.find('product price');
console.log('Price location:', price.coordinates);
console.log('Price text:', price.text);
```
## Element Object
The returned `Element` object provides:
### Methods
- `found()` - Check if element was located
- `click(action)` - Click the element
- `hover()` - Hover over the element
- `doubleClick()` - Double-click the element
- `rightClick()` - Right-click the element
- `find(newDescription)` - Re-locate with optional new description
### Properties
- `coordinates` - Element position `{x, y, centerX, centerY}`
- `x`, `y` - Top-left coordinates
- `centerX`, `centerY` - Center coordinates
- `text` - Text content (if available)
- `screenshot` - Base64 screenshot (if available)
- `confidence` - AI confidence score
- `width`, `height` - Element dimensions
- `boundingBox` - Complete bounding box
See [Elements Reference](/v7/elements) for complete details.
### JSON Serialization
Elements can be safely serialized using `JSON.stringify()` for logging and debugging. CirMore from this repository
testdriver:aiSkill
Execute natural language tasks using AI
testdriver:assertSkill
Make AI-powered assertions about screen state
testdriver:aws-setupSkill
Deploy TestDriver on your AWS infrastructure using CloudFormation
testdriver:cacheSkill
Speed up tests with screenshot-based caching
testdriver:cachingSkill
1.7x faster test execution with intelligent caching and optimization
testdriver:captchaSkill
Solve captchas using 2captcha service
testdriver:ci-cdSkill
Run TestDriver tests in CI/CD with parallel execution and cross-platform support
testdriver:clickSkill
Click at specific coordinates or on elements