pyjab-mcp

# pyjab-mcp

An MCP server that lets an AI agent drive **Java desktop applications** — Swing and AWT
— through [pyjab](https://pypi.org/project/pyjab/), which reads the JVM's own
accessibility tree via Java Access Bridge. **JavaFX is not supported**: it publishes its
accessibility through UI Automation rather than Java Access Bridge, so a JavaFX window is
not a Java window as far as JAB is concerned
([pyjab#227](https://github.com/gaozhao1989/pyjab/issues/227)).

<!-- The MCP Registry proves ownership of the PyPI package by finding this exact line in the
     README *of the published package*, so it cannot be added after a release. Keep it. -->
mcp-name: io.github.gaozhao1989/pyjab-mcp

**It never modifies, restarts or injects into the target application.** That constraint is
the point: launch parameters, classpaths and dependency JARs stay exactly as they are,
which is what makes this usable against a production Java client that you cannot change.

**The alternative is to put an agent inside the target JVM.** An agent is attached to a JVM
that is already running, or named on the command line with `-javaagent` when the application
starts, and either way the tool's code sits inside someone else's process — a JVM it did not
launch and does not control. This server does neither: it loads nothing into the application
and reads the accessibility tree through Java Access Bridge from outside it, which is how it
can attach to a process it did not launch.

The dynamic half of that — attaching an agent to a JVM that is already running — is the half
the JDK has a direction on, with no release date attached.
[JEP 451](https://openjdk.org/jeps/451) shipped in JDK 21 as the preparation stage: loading an
agent into an already-running JVM now warns. The JEP states its aim as *"prepare users for a
future release which disallows the dynamic loading of agents by default"*, a release that has
not been named; its own non-goals leave agents named with `-javaagent` at startup, and any use
of the Attach API that is not loading an agent, outside that direction.

## Status

**Released on PyPI** — 16 tools: `java_diagnostics`, `java_list_windows`,
`java_attach_window`, `java_snapshot`, `java_find`, `java_get_element`, `java_get_text`,
`java_get_table`, `java_click`, `java_set_text`, `java_select`, `java_fill_form`,
`java_send_keys`, `java_focus_window`, `java_wait_for`, and `java_screenshot`. It needs
`pyjab>=1.11.0`: 1.9.0 brought the tree walk a snapshot is built on, 1.10.0 brought the public
window listing and `JABDriver.detach()`, which releases a driver's binding without terminating
the process — this server has no tool that releases a session, so a binding lasts for the life
of the process — and 1.11.0 brought `JABDriver.send_keys`, so `java_send_keys` sends a real
chord, and `JABDriver.focus()`, so `java_focus_window` can bring a window forward.

`java_send_keys` sends a chord to whichever window has the keyboard focus, and **attaching does
not give a window focus** ([pyjab#180](https://github.com/gaozhao1989/pyjab/issues/180)).
`java_focus_window` is the missing half: it brings the window forward and, because *asked for
focus* and *got focus* are different facts, it reports **whether the window actually came
forward** rather than assuming it did. It probes for the API rather than assuming it, so an
install below the pin is told which release has it instead of failing with an `AttributeError`.

**Two elements can share a name**, which is why a handle carries where an element *is* as well as
what it is called: searching by name returns every match, each gets its own handle, and the
end-to-end suite checks that clicking one acts on that one and not the other.

**Nothing is acted on until it has been checked.** Every action re-finds the element and
compares role, name, position among siblings and bounds against what the snapshot recorded,
then acts on the element it just checked — and if anything differs, it does nothing and
reports the difference.

`java_get_table` is where this substrate earns its place: it reads a `JTable` through the
JVM's own `AccessibleTable` interface — row and column counts, then cell text, a page of rows
at a time — rather than through a platform bridge that does not carry that information at all.

Live behaviour is verified on a hosted Windows runner by `windows-gui.yml`: the end-to-end
suite attaches to a real Swing application, snapshots it, resolves a handle against the real
element, and checks that a window opened *after* the bridge had been idle is still seen. That
last one is the reason the layer exists — a message pump on the wrong thread passes every
other test in this repository. What it still cannot cover is your application: the run drives
pyjab's test app, and the one that matters is the one you care about.

It needs `pyjab>=1.11.0`, which brought the APIs this project spent M1 and M2 waiting on:
`list_java_windows()`, so the window list comes from the bridge directly rather than through
pyjab's CLI; `JABDriver.detach()`, so a bound driver can be released **without terminating the
application** — the only teardown before it killed the bound process; and, in 1.11.0,
`JABDriver.send_keys()`, so `java_send_keys` sends a real chord, and `JABDriver.focus()`, which
`java_focus_window` uses to bring a window forward and report whether it came forward. That
return value is why the tool exists at all: what a caller needs to know before sending a chord
is where the keyboard focus actually is, not that focus was requested.

A handle from `java_snapshot` is a position, not a reference. A later call re-finds the
element and checks its role, name, position and bounds before doing anything, so a window
that moved on produces a message rather than an action on the wrong control.

The one thing that was already settled is the question the whole direction rested on.
Measured on a real Java window, through the same `UIAutomationCore` that generic Windows
automation uses:

| | elements | named | depth |
|---|---|---|---|
| generic UI Automation, the Java window | **6** | 5 | 3 |
| pyjab, the same window | **601** | 513 | 12 |
| *control: UI Automation, the whole desktop* | *147* | *99* | *finished* |

The control is what makes the six mean anything: the *same* measurement over the whole
desktop finishes, so the measurement works and the Java window is the hard case.

## Where to read first

| file | what |
|---|---|
| `docs/ARCHITECTURE.md` | how the server is put together, the constraints that shape it, and what each guard enforces |
| `docs/CLIENTS.md` | connecting Claude Desktop, Claude Code, Cursor and VS Code |
| `docs/RELEASING.md` | how a release happens, and what has to exist before one |
| `CHANGELOG.md` | what changed, and why |

Maintainer working notes are kept out of this repository deliberately. Everything a user or a
contributor needs is above, and the rules that are not prose are enforced by the guards in
`tools/` — which run in CI on every push.

## Requirements

* Windows
* Python 3.10 or newer
* A JDK with **Java Access Bridge** enabled — `"%JAVA_HOME%\bin\jabswitch" -enable`, then
  restart the Java application — for the application being driven
* `pyjab`, which brings the rest

## Install

```console
pip install pyjab-mcp
```

Working on the server itself is `pip install -e ".[dev]"` — see [Working on it](#working-on-it).

### Or install the bundle: one file, no build step

v0.1.0–v0.1.2 predate the bundle and carry no assets; every release cut since carries one. It is
`pyjab-mcp-<version>.mcpb` — the built wheel and its dependency closure, installed together into
one tree per supported CPython minor, plus one entry script, in a single file. Each tree carries
the package itself, so the file holds one copy of `pyjab_mcp` per minor rather than one wheel
beside the trees. Download it from the
[releases page](https://github.com/gaozhao1989/pyjab-mcp/releases) and hand it to the client:
Claude Desktop installs a `.mcpb` by opening it, and [docs/CLIENTS.md](docs/CLIENTS.md) has the
details.

**The bundle changes none of the requirements above, and it narrows one of them to the
interpreters it actually carries.** It bundles no Python interpreter and no JDK: this server
drives Java Access Bridge, so **64-bit Windows, 64-bit CPython 3.10-3.13, and a JDK with Java
Access Bridge enabled** are still what the machine has to provide — and the Java application it
drives still has to be running before you attach to it. That range is narrower than `pip
install`, which has no upper bound, and it is not a detail: the bundle is tens of megabytes
because it carries one dependency tree **per supported minor**, since `pydantic-core`, `cffi`,
`rpds-py`, `cryptography` and `pywin32` ship compiled wheels tagged for one minor each. Started
on any other interpreter it stops with a sentence naming the interpreter it found and the ones
the file carries, rather than an `ImportError` from a native module.

**The interpreter has to be the `python` on `PATH`.** The manifest declares no `user_config`
fallback, so the client runs whatever `python` resolves to there — and a machine with CPython
installed but not on `PATH` gets a client that shows no tools, with nothing in it that names the
cause: no interpreter ever starts, so the bundle's own refusal cannot run. `where.exe python`
prints the interpreter that will be used; if it prints nothing, put the installation's directory
on `PATH`.

**MCPB enforces no permissions.** There is no manifest-level sandbox — a bundled server runs
with the full privileges of the user who installed it, exactly as it would if you had started
it by hand. Read the source before installing a `.mcpb` from anyone, this one included.

## Use

The console script the package installs:

```json
{
  "mcpServers": {
    "pyjab-mcp": { "command": "python", "args": ["-m", "pyjab_mcp"] }
  }
}
```

Claude Desktop reads `%APPDATA%\Claude\claude_desktop_config.json`, and takes this (double
the backslashes — JSON n