screenpipe-api

# Screenpipe API

Local REST API at `http://localhost:3030`.

## Authentication

**Every request needs auth.** `$SCREENPIPE_LOCAL_API_KEY` is already in your env; without it you get 403.

```bash
curl -H "Authorization: Bearer $SCREENPIPE_LOCAL_API_KEY" "http://localhost:3030/..."
```

No-auth endpoints: `/health`, `/ws/health`, `/audio/device/status`, `/connections/oauth/callback`, `/frames/*`, `/notify`, `/pipes/store/*`.

## Context Window Protection

Responses can be large. Write curl output to a file (`-o /tmp/sp.json`), check size (`wc -c`), and if over ~5KB read only the first 50-100 lines / extract with `jq`. Never dump full large responses into context.

Cut tokens at the source on list endpoints (`/search`, `/elements`): add `&format=csv` (or `tsv`) for a columnar table (column names written once instead of per-row keys — ~70% cheaper on uniform rows like elements), and `&fields=a,b,c` for only the columns you need (dotted paths like `content.text`). Text-heavy `ocr`/`audio` barely benefit — use `fields` + `max_content_length` there instead.

---

## 1. Activity Summary — `GET /activity-summary`

Default broad-context call. Bundles apps, windows, key_texts, audio, edited_files, recording health, top memories, deduped screen+audio snippets, and a `data_status`/`query_status`/`guidance` triple.

```bash
curl -H "Authorization: Bearer $SCREENPIPE_LOCAL_API_KEY" "http://localhost:3030/activity-summary?start_time=30m%20ago&end_time=now"
```

Required: `start_time`, `end_time`. Optional: `app_name`, `q` (filters memories+snippets, drives `query_status`); `include_recording|memories|snippets|guidance=false` to slim (each defaults true); `max_snippets`, `max_snippet_chars`, `max_memories`. For a lean time-tracking sweep also set `include_key_texts=false` (biggest win), `include_apps=false`, `include_windows=false` — `total_active_minutes` + per-app/window `minutes` + the status triple still return.

- `data_status` ∈ `ok|empty_but_recording|no_capture_in_range|not_recording` — check before claiming "no activity".
- `query_status` ∈ `not_requested|matched|no_query_matches`; `guidance.next_best_query` is a ready hint when empty.
- Escalate to `/search` only for verbatim quotes / frame_ids.

---

## 2. Search — `GET /search`

Use when `/activity-summary` says `ok` but you need verbatim quotes, media paths, frame IDs, or a specific match.

```bash
curl -H "Authorization: Bearer $SCREENPIPE_LOCAL_API_KEY" "http://localhost:3030/search?q=QUERY&content_type=all&limit=10&start_time=1h%20ago"
```

| Parameter | Required | Description |
|-----------|----------|-------------|
| `q` | No | Keywords. Avoid for audio — transcriptions are noisy, `q` over-filters. |
| `content_type` | No | `all` (default), `accessibility`, `audio`, `input`, `ocr`, `memory`. Screen text is primarily the accessibility tree; OCR is the fallback for apps without it (videos, games, remote desktops). |
| `limit` | No | Default 20. Keep ≤20 to protect context. |
| `offset` | No | Pagination. Default 0. |
| `start_time` | **Yes** | ISO 8601 or relative (`16h ago`, `2d ago`, `30m ago`). |
| `end_time` | No | Defaults to now (`now`, `1h ago`). |
| `app_name` | No | Substring, e.g. "Google Chrome", "Slack". |
| `window_name` | No | Window title substring. |
| `speaker_name` | No | Filter audio by speaker (case-insensitive partial). |
| `focused` | No | Only focused windows. |
| `tags` | No | Comma-separated; returns items carrying ALL of them (`person:ada,project:atlas`). Exact match. |
| `include_related` | No | With `tags`, also return a `related` map of co-occurring tags (people/projects/workflows), most-frequent first. |
| `max_content_length` | No | Middle-truncate each result's text. |
| `format` | No | `json` (default), `csv`, `tsv`/`table`. CSV is lossless; TSV collapses newlines. |
| `fields` | No | Column allowlist of dotted paths, e.g. `type,content.app_name,content.text`. |

**Critical rules:** always include `start_time` (unbounded queries timeout) · "recent" = 30 min, "today" = since midnight, "yesterday" = yesterday's range · if `/search` is empty, fall back to `/activity-summary` and check `data_status` before saying "no data" · on timeout, narrow the range.

**Tags** link people/projects/topics across screen, audio, and memories under one namespace (`person:ada`, `project:atlas`, `topic:pricing`). Add to a frame/audio: `POST /tags/vision/{frame_id}` or `POST /tags/audio/{chunk_id}` body `{"tags":["person:ada"]}`; to a memory: `tags` in `POST /memories`. Retrieve: `GET /search?tags=person:ada&start_time=30d%20ago` (add `content_type=memory` for memories). Frames are pruned by retention — tag a **memory** for durable links (memories carry `created_at` + a `frame_id` back to the moment). `include_related=true` returns co-occurring tags grouped by namespace, replacing 2-3 follow-up calls.

Response: `{"data": [{"type":"OCR","content":{"frame_id":...,"text":...,"app_name":...}}, {"type":"Audio","content":{"chunk_id":...,"transcription":...,"speaker":{"name":...}}}, {"type":"Input","content":{...}}], "pagination":{"limit":10,"offset":0,"total":42}}`.

---

## 3. Elements — `GET /elements`

Lightweight FTS over UI elements (~100-500 bytes each vs 5-20KB from `/search`). Uniform rows, so `format=csv` pays off most.

```bash
curl -H "Authorization: Bearer $SCREENPIPE_LOCAL_API_KEY" "http://localhost:3030/elements?frame_id=12345&format=csv&fields=role,text,bounds.left,bounds.top"
```

Params: `q`, `frame_id`, `source` (`accessibility`|`ocr`), `role`, `start_time`, `end_time`, `app_name`, `limit`, `offset`, `format`, `fields`.

Frame context (accessibility text, parsed nodes, extracted URLs): `GET /frames/{id}/context`.

**Roles are not normalized across platforms** — use the right one for the user's OS:

| Concept | macOS | Windows | Linux |
|---------|-------|---------|-------|
| Button | `AXButton` | `Button` | `Button` |
| Static text | `AXStaticText` | `Text` | `Label` |
| Link | `AXLink` | `Hyperlink` | `Link` |
| T