terminal-use

# terminal-use

An [MCP](https://modelcontextprotocol.io) server that lets an AI agent use a real terminal the way a person does: type, press keys, click, read the screen, take a screenshot.

![An agent opening vim, pasting a program, saving it, running it, and taking a screenshot — each step labelled with the tool call that made it](docs/demo.gif)

*Every frame above was drawn by terminal-use's own screenshot renderer; the caption on each is the tool call that produced it.*

Most agent shell tools run a command and hand back its output. That breaks down for anything interactive — `vim`, `htop`, a REPL, an installer asking questions, `git rebase -i`, an SSH session. terminal-use gives the agent a shell on a real pseudo-terminal, rendered by a real terminal emulator, so full-screen and interactive programs work and the agent sees what you would see.

- **Real PTY, real emulator** — colors, cursor movement and the alternate screen are interpreted, not passed along as escape codes.
- **Text and pixels** — read the screen as plain text, or as a PNG when layout and color matter.
- **Waits properly** — block until a command actually finishes, or until some text appears.
- **Watch along** — attach your own terminal to any session and type alongside the agent.

## Quick start

Requires Node.js 20.19 or newer, on macOS, Linux or Windows.

**Claude Code**

```bash
claude mcp add terminal-use --scope user -- npx -y terminal-use
```

**Other MCP clients** (Claude Desktop, Cursor, and anything else that takes a command):

```json
{
  "mcpServers": {
    "terminal-use": {
      "command": "npx",
      "args": ["-y", "terminal-use"]
    }
  }
}
```

Start a new session in your client and ask it to do something in a terminal, for example: *"Open vim, write a haiku into /tmp/haiku.txt, save and quit, then show me a screenshot of `cat`-ing it."*

Server options go after the command: `npx -y terminal-use --cols 100 --rows 40`.

| Option | Default | |
|---|---|---|
| `--shell <path>` | `$SHELL` or `/bin/bash`; PowerShell on Windows | Shell to run in new sessions |
| `--cwd <path>` | where the server was started | Working directory for new sessions |
| `--cols <n>` / `--rows <n>` | `120` / `30` | Terminal size |
| `--scrollback <n>` | `5000` | Lines of history kept |
| `--login` | off | Start shells as login shells (see below) |

### "command not found" inside a session

If programs that work in your own terminal (`node`, `brew`, `pyenv`…) are missing inside a session, the server probably inherited a bare environment. That happens when the MCP client is started from the Dock or a launcher instead of a terminal: your `PATH` is set up by your shell's profile files (`~/.zprofile`, `~/.bash_profile`, `~/.profile`), and nothing has read them.

A login shell reads those files. Turn it on for every session with the `--login` server flag, or for one session with `login: true` on `terminal_create`. It is off by default because it makes each shell slower to start and runs whatever your profile runs. It has no effect in PowerShell or `cmd.exe`, which have no login mode.

## Tools

Every tool except `terminal_create` and `terminal_list` takes the `sessionId` that `terminal_create` returns.

| Tool | What it does |
|---|---|
| `terminal_create` | Start a session: an interactive shell, or one program with `command`. Optional `label`, `cols`, `rows`, `shell`, `cwd`, `env`, `login`, `scrollback`, `theme`. |
| `terminal_list` | List sessions. |
| `terminal_destroy` | End a session. |
| `terminal_type` | Type text. `\n` presses Enter. `paste: true` sends it as one paste. |
| `terminal_press` | Press a key or combination: `Enter`, `Ctrl+C`, `ArrowUp`, `Shift+Tab`, `Alt+Enter`, `F5`… |
| `terminal_wait` | Wait for the running command to finish, for a regex to appear, or for output to go quiet. |
| `terminal_read` | Read the screen or scrollback as text. |
| `terminal_screenshot` | Render the screen as a PNG. |
| `terminal_click` | Left-click a cell, with a preview step. |
| `terminal_scroll` | Turn the mouse wheel over a cell. |
| `terminal_batch` | Send several inputs in one call and get the screen back once. |
| `terminal_resize` | Change the terminal size. |
| `terminal_reset` | Clear the screen and scrollback, or restart the shell with `hardReset: true`. |

## Docker

The repository has a [Dockerfile](Dockerfile) for running the server in a container:

```bash
docker build -t terminal-use .
docker run -i --rm -v "$PWD":/workspace terminal-use
```

In an MCP client, use `docker` as the command with `run -i --rm terminal-use` as its arguments. The terminals the agent gets are then shells inside the container, not on your machine: it sees only what you mount, which is the point if you want it boxed in. Attaching from your own terminal is not available in this setup.

## The skill

terminal-use comes with an [Agent Skill](https://agentskills.io): a short guide for the agent on how to use these tools well — when to use a command session, how to wait, how to read what is selected — plus recipes for vim, pagers, REPLs, menu-driven programs, ssh prompts and end-to-end testing of a TUI. It lives in [`skills/terminal-use`](skills/terminal-use).

- **Hosts that load skills from MCP servers** get it automatically. The server implements the MCP Skills extension (`io.modelcontextprotocol/skills`): the skill is listed by `skills/list` and its files are served as `skill://terminal-use/...` resources. Few hosts support this yet.
- **Hosts that load skills from disk** can install the same files. For Claude Code:

  ```bash
  cp -r "$(npx -y terminal-use skills-dir)/terminal-use" ~/.claude/skills/
  ```

The skill is optional. Without it the agent still has the tool descriptions and the server's built-in instructions.

## Shell sessions and command sessions

By default a session is an interactive shell: type commands into it as you would at a prompt.

Pass `command` to `terminal_create` to run one program in the terminal instead:

```json
{"command": "npm test -- --watch", "cwd": "/path/to/project", "env": {"CI": "1"}}
```

The command goes through the session's shell (`sh -c`, PowerShell `-Command`, or `cmd /c`), so quoting, pipes and redirection work as they do at that shell's prompt. Input goes straight to the program. When it exits:

- its exit status is reported (by the call that was in progress, by `terminal_wait`, and in `terminal_list`);
- the final screen stays readable with `terminal_read` and `terminal_screenshot`;
- input tools return an error with the exit status rather than restarting anything;
- `terminal_reset` with `hardReset: true` runs it again, and `terminal_destroy` removes it.

This is the mode for testing a CLI or TUI: launch it, drive it, check how it ended.

## Watching and typing along

`terminal_create` returns a command you can run in your own terminal to join the session:

```
node /path/to/terminal-use/bin/terminal-use.js attach 3 --socket /tmp/terminal-use-501/41234-3.sock
```

You see what the agent sees and can type into the same shell. It works like a shared `tmux` session: several people can attach at once, and **Ctrl+]** detaches.

- You get the current screen and recent scrollback on connect, not a blank terminal.
- `--resize` makes the session follow your window size. By default the session keeps its own.
- `--socket` picks the server. Each MCP client runs its own terminal-use, and they all number sessions from 1; without `--socket`, `attach <id>` works when only one running server has that id and lists the candidates otherwise.
- On Windows the session is reached through a named pipe instead of a socket file; the command you are given works the same way.
- Sockets are per-user (`0600`, inside a `0700` directory under the system temp dir). Anyone who can connect gets a shell as you, so they are not exposed any further than that.

## How it works

```
agent ──MCP──▶ terminal-use ──▶ node-pty ──▶ your shell
                  │
                  └─ @xterm/headless  ◀── everything the shell prints
                        │
                        ├─ terminal_read        (text)
                        └─ terminal_screenshot  (PNG)
```

[`node-pty`](https://github.com/microsoft/node-pty) runs the shell on a pseudo-terminal. Everything it prints is fed to a headless [xterm.js](https://xtermjs.org) emulator, and that emulator's buffer is the single source of truth: reads and screenshots both come from it, so the agent gets the rendered screen rather than a stream of escape codes.

## Details

### Waiting for things

`terminal_type`, `terminal_press` and `terminal_click` return once output has been quiet for a moment (`idleMs`, default 200 ms) and never wait longer than 10 seconds. That suits keystrokes. It doesn't suit a build, which can be silent for a while long before it is done. For anything slow, follow up with `terminal_wait`:

- **Default — wait for the command to finish.** terminal-use asks the kernel which process owns the terminal's foreground. A shell hands the terminal to each command it runs and takes it back afterwards, so when the shell owns it again, the prompt is back. This needs no shell integration or prompt parsing, and works for commands that print nothing.
- **`pattern` — wait for a regex to match the screen.** For things that never exit (`Listening on port`), REPL prompts, or a particular state of a TUI. `^` and `$` match at line starts and ends; a leading `(?i)` or `(?s)` sets further flags.
- **`until: "quiet"` — wait for output to stop** for `quietMs` (default 1 s). The same rule the typing tools use, without the 10-second cap.

In a command session, the default mode waits for the program to exit and reports its exit status.

`timeoutMs` defaults to 30 seconds (maximum 10 minutes). A timeout isn't an error: the response says the command is still running, and you can wait again.

On Windows there is no way to ask who owns the terminal, so in a shell session the default mode waits for two seconds of silence ins