vllm-ops-mcp

# vllm-ops-mcp

[![License: MIT](https://img.shields.io/badge/license-MIT-green)](LICENSE)
[![tests](https://img.shields.io/badge/tests-102%20%2895%20passing%2C%207%20live--skipped%29-brightgreen)](tests)
[![CI](https://github.com/jaimenbell/vllm-ops-mcp/actions/workflows/ci.yml/badge.svg)](https://github.com/jaimenbell/vllm-ops-mcp/actions/workflows/ci.yml)

Read-only ops/health MCP server for a local, WSL2/systemd-managed vLLM
server -- liveness vs. real-completion health tiers, GPU/VRAM status,
systemd service status, and live serve-flag introspection. Built to the same
standard as [mcp-factory](https://github.com/jaimenbell/mcp-factory) and
[desktop-mcp](https://github.com/jaimenbell/desktop-mcp), and other local
tooling in this operator's portfolio: own pyproject, own fastmcp server,
honest README, real test suite, no product code before a spec was confirmed.

This is **not** a container-lifecycle manager. It assumes vLLM already runs
as a bare-metal `systemd` unit inside WSL2 (this operator's actual setup --
see [vllm-autostart](../vllm-autostart)), not inside Docker/Podman.

## Quickstart

```jsonc
// ~/.claude.json (or any MCP-host stdio client config)
{
  "mcpServers": {
    "vllm-ops-mcp": {
      "type": "stdio",
      "command": "C:\\Users\\jaime\\projects\\vllm-ops-mcp\\.venv\\Scripts\\python.exe",
      "args": ["C:\\Users\\jaime\\projects\\vllm-ops-mcp\\run_server.py"]
    }
  }
}
```

## Tools (Phase 1 -- all read-only, no gating required beyond a throughput cap)

| Tool | What it does | What it can't do |
|---|---|---|
| `check_health(deep: bool = False)` | `deep=False`: GET `/v1/models` liveness check (same signal as `llm_router.LocalLLM.is_healthy()`). `deep=True`: additionally fires one minimal real `/v1/chat/completions` call (mirrors `check-vllm.cmd`'s two-stage probe) to confirm the server actually *generates*, not just *reports a model loaded*. | Cannot fix a degraded server -- reports ground truth only. `deep=True` is rate-limited (default 20/min, see below). |
| `list_models` | `/v1/models` passthrough. | No detail beyond what vLLM's API itself exposes. |
| `test_completion(prompt, max_tokens=16)` | On-demand real completion with a caller-supplied prompt, for manual sanity checks. | Rate-limited (same bucket as `check_health(deep=True)`); refuses if the liveness check fails first (no point firing a completion at a server that isn't even listening). `max_tokens` is clamped server-side to 1024 (`config.MAX_TEST_COMPLETION_TOKENS`) and the prompt is capped at 8000 chars (`config.MAX_TEST_COMPLETION_PROMPT_CHARS`, oversized prompts are rejected, not truncated) -- the rate limiter only bounds call *frequency*, not the cost of a single call. |
| `get_gpu_status` | `nvidia-smi` wrapper: per-GPU VRAM used/total + utilization%, plus per-process VRAM via `--query-compute-apps`. | **Per-process VRAM attribution does not work on this operator's actual WSL2 setup** -- live-verified empty even while the vLLM process holds 13.7GB (see Limitations). Treat `processes` as best-effort/often-empty, not a reliable per-PID breakdown. |
| `get_service_status` | `systemctl show <unit> -p ...` (structured property output, not free-text parsing) via `wsl -d <distro>` from the Windows host, or native `systemctl` if this server itself runs inside WSL2/Linux. Returns load/active/sub state, restart count, main PID, last-active timestamp. | Read-only -- cannot start/stop/restart the unit. No computed uptime duration (deliberately -- see Limitations, clock-discipline note). |
| `get_serve_config` | Read-only launch-flag introspection. **Prefers the LIVE process's actual argv** (`/proc/<pid>/cmdline` of the systemd unit's `MainPID` -- ground truth of what's running right now), falling back to the static exec script (local mirror via `VLLM_OPS_MCP_SERVE_CONFIG_PATH`, else a live `cat` of the WSL-side exec script) only when the process isn't up. | vLLM doesn't expose its own launch flags over the API, so this never queries the server itself -- it's always shelling out. Exec-script fallback parsing is best-effort shell-text parsing (less precise than the live-argv path). |

### Future extension: `restart_service` (Phase 2, not built)

Deliberately **not implemented in this release**. Per the spec: vLLM is
shared infra other bots/tools depend on (options-bot, `qwen_cli.py`
callers, DeerFlow fallback), so a restart tool needs its own gating design
(off by default, `VLLM_OPS_MCP_ENABLE_RESTART=1`, refuse to touch a healthy
instance unless `force=True`, rate-capped) before it ships -- not
retrofitted onto a read-only MVP. Tracked as a v2 idea, not a current
capability. If built, it should wrap `run-vllm.cmd`'s exact recipe (`systemctl
start vllm` only, never blind `stop`/`restart`) rather than reinvent it.

## The differentiator: probe HTTP, not the listener

Every health signal in this server comes from an actual HTTP request to the
OpenAI-compatible API (`urllib.request`, not a socket/connect check). This
is deliberate, not incidental: `Get-NetTCPConnection` and other
listener-based checks **miss WSL2-mirrored-networking-relayed ports
entirely** -- a genuinely healthy `127.0.0.1:8000` can be invisible to a
listener probe run from the Windows host. `check-vllm.cmd` already embodies
this lesson; this server generalizes it into two tiers (liveness vs. real
generation) that nothing else in this operator's fleet exposes as MCP tools.

## Security

Read this before you register vllm-ops-mcp. Honesty about blast radius is
the point.

- **This server shells out to `nvidia-smi`, `systemctl` (via `wsl -d`), and
  reads `/proc/<pid>/cmdline`.** All six tools are read-only -- nothing here
  starts, stops, restarts, or kills a process, and no tool accepts an
  argument that gets passed to a shell unescaped (unit/distro names come
  from server-side config, not caller input; `shlex.quote` wraps the one
  caller-adjacent path that does reach a shell string, the unit name in
  `systemctl show`).
- **`check_health(deep=True)` and `test_completion` fire a real inference
  call against a shared GPU server.** Rate-limited (default 20 calls/min,
  `VLLM_OPS_MCP_DEEP_RATE_LIMIT_PER_MIN`) specifically because this vLLM
  instance is shared infra other bots/tools depend on -- a caller hammering
  this MCP could starve or slow other consumers even though every call
  individually looks "read-only."
- **`get_serve_config`'s live-process path reads `/proc/<pid>/cmdline`,**
  which can include secrets if any were ever passed as CLI flags (none are,
  in this operator's actual setup -- the model path and flags are all
  non-secret; auth, if any, is environment-injected inside the exec script,
  not passed as an argv flag). Defense-in-depth: any argv entry whose
  preceding flag name matches a known-sensitive pattern (`token`, `key`,
  `secret`, `password`, `passwd`, `credential` -- `config.
  SENSITIVE_ARGV_FLAG_PATTERNS`) has its **value** redacted to
  `[REDACTED]` before being returned, in both the structured `argv`/`flags`
  fields and `raw_text`; the flag name itself is left visible so the shape
  is still diagnosable. This applies across all three sources
  (`live_process`, `exec_script`, `local_config_mirror`).
- **No tool has any write/mutate capability in this release.** There is no
  env var that turns one on -- `restart_service` doesn't exist in this
  codebase yet (see Future extension above), so there's nothing to
  accidentally enable.

## Honest-capabilities table

| Claim | Implementation | Verified by |
|---|---|---|
| Liveness health tier (`/v1/models`) | `vllm_ops_mcp/probes.py::_probe_models`, `check_health` | `tests/test_probes.py::TestCheckHealthLiveness`; live: `tests/test_live_smoke.py::test_live_check_health_liveness` |
| Deep/real-completion health tier | `vllm_ops_mcp/probes.py::_probe_completion`, `check_health(deep=True)` | `tests/test_probes.py::TestCheckHealthDeep`; live: `test_live_check_health_deep` |
| Model list passthrough | `probes.py::list_models` | `tests/test_probes.py::TestListModels`; live: `test_live_list_models` |
| On-demand completion sanity check | `probes.py::test_completion` | `tests/test_probes.py::TestTestCompletion`; live: `test_live_test_completion` |
| GPU/VRAM status via nvidia-smi | `probes.py::get_gpu_status` | `tests/test_probes.py::TestGetGpuStatus`; live: `test_live_get_gpu_status` |
| Per-process VRAM attribution | `probes.py::get_gpu_status` (`--query-compute-apps`) | Unit-tested with a mocked non-empty response; **live-verified to return EMPTY on this operator's actual WSL2 setup** -- see Limitations |
| systemd unit status via structured `systemctl show` | `probes.py::get_service_status` | `tests/test_probes.py::TestGetServiceStatus`; live: `test_live_get_service_status` |
| Live-process launch-flag introspection via `/proc/<pid>/cmdline` | `probes.py::get_serve_config`, `parse_launch_flags` | `tests/test_probes.py::TestGetServeConfig`, `TestParseLaunchFlags`; live: `test_live_get_serve_config` -- live-confirmed it recovers flags (`--enable-auto-tool-choice --tool-call-parser qwen3_xml`) that were **not** in this repo's own grounding spec, proving the live-argv approach beats trusting a static doc |
| Exec-script fallback when the unit is down | `probes.py::get_serve_config`, `_parse_exec_script_text` | `tests/test_probes.py::TestGetServeConfig::test_falls_back_to_exec_script_when_service_down` |
| Local config-mirror override path | `probes.py::get_serve_config` + `config.serve_config_override_path` | `tests/test_probes.py::TestGetServeConfig::test_uses_local_override_file`, `test_override_file_missing_reports_error` |
| Sensitive-flag-value redaction in `get_serve_config` | `probes.py::redact_argv`, `redact_raw_text`, `config.SENSITIVE_ARGV_FLAG_PATTERNS` | `tests/test_probes.py::TestGetServeConfig::test_live_process_redacts_sensitive_flag_values`, `test_exec_script_raw_text_redacts_sensitive_flag_values`, `test_local_override_file_redacts_sensitive_flag_values` |
| Deep-inference rat