kafka-mcp

# kafka-mcp

[![License](https://img.shields.io/github/license/denizgursoy/kafka-mcp?color=blue&style=flat-square)](https://raw.githubusercontent.com/denizgursoy/kafka-mcp/main/LICENSE)
[![Coverage](https://img.shields.io/sonar/coverage/denizgursoy_kafka-mcp?logo=sonarcloud&server=https%3A%2F%2Fsonarcloud.io&style=flat-square)](https://sonarcloud.io/summary/overall?id=denizgursoy_kafka-mcp)
[![Web](https://img.shields.io/badge/web-document-blueviolet?style=flat-square)](https://denizgursoy.github.io/kafka-mcp/)
[![Release](https://img.shields.io/github/v/release/denizgursoy/kafka-mcp?style=flat-square)](https://github.com/denizgursoy/kafka-mcp/releases/latest)
[![Kafka MCP server – quality and maintenance score on Glama](https://glama.ai/mcp/servers/denizgursoy/kafka-mcp/badges/score.svg)](https://glama.ai/mcp/servers/denizgursoy/kafka-mcp)

An MCP server that exposes Kafka debugging as tools an LLM can call. It speaks
MCP over stdio by default, optionally serves streamable HTTP, and talks to Kafka
with [franz-go](https://github.com/twmb/franz-go).

One process can connect to several Kafka clusters. In stdio mode, one endpoint
is selected for the session. In HTTP mode, each endpoint has its own path, so a
session is bound to one cluster by how it connects rather than by a parameter a
caller could forget to send.

## Install

Every release ships the same server four ways:

- **Binary**: `kafka-mcp_<Os>_<arch>` archives on the
  [releases page](https://github.com/denizgursoy/kafka-mcp/releases/latest).
  Put `kafka-mcp` on your `PATH`.
- **MCP Bundle**: `kafka-mcp_<version>_<os>_<arch>.mcpb` on the same page,
  for macOS on Apple Silicon, Linux (amd64, arm64) and Windows (amd64). Open
  it in a client that installs bundles, such as Claude Desktop. It asks for
  your configuration file and, when that file has several endpoints, which
  one to serve. Intel Macs use the binary or the image.
- **Container image**: `ghcr.io/denizgursoy/kafka-mcp:<tag>`. It starts with
  `--server`; for stdio, mount the file and pass `--server=false`:

  ```sh
  docker run -i --rm \
    --mount type=bind,src=$PWD/kafka-mcp.yaml,dst=/config/kafka-mcp.yaml,readonly \
    -e CONFIG_FILE=/config/kafka-mcp.yaml \
    ghcr.io/denizgursoy/kafka-mcp:latest --server=false --endpoint local
  ```

- **MCP Registry**: listed as `io.github.denizgursoy/kafka-mcp` in the
  [official registry](https://registry.modelcontextprotocol.io/v0.1/servers?search=io.github.denizgursoy/kafka-mcp),
  with the image and the bundles, so a client that reads the registry can
  install it from there.

How releases reach the registry and the other catalogs is in
[PUBLISHING.md](PUBLISHING.md).

## Configuration

`kafka-mcp.{toml,yaml,yml,json}` in the working directory, `~/.config/kafka-mcp/` or `/etc` configures the server. The `http` block is used only with `--server`.

```yaml
http:
  address: ":8090"
  base_path: /kafka-mcp # optional; prefixes every endpoint and /healthz
output_dir: /var/tmp/kafka-mcp
clusters:
  prod:
    brokers:
      - kafka-1:9093
      - kafka-2:9093
    security:
      tls:
        enabled: true
        ca_file: /etc/kafka/ca.pem
        # cert_file: /etc/kafka/client.pem # optional mTLS; requires key_file
        # key_file: /run/secrets/client.key
      sasl:
        - scram:
            enabled: true
            algorithm: SCRAM-SHA-256 # or SCRAM-SHA-512
            user: kafka-mcp-readonly
            pass: "{env:KAFKA_PASSWORD}" # or password_file: /run/secrets/kafka
  preprod:
    brokers: kafka-preprod:9093
endpoints:
  prod-read:
    cluster: prod
    path: /mcp
    description: Production investigation and debugging
    read_only: true
  prod-write:
    cluster: prod
    path: /mcp/rw
    description: Approved production changes
    tools:
      copy_message: false
  preprod:
    cluster: preprod
    path: /mcp/preprod
```

`clusters` owns Kafka connection details: brokers, TLS and SASL. `endpoints`
owns the policy and, in HTTP mode, the MCP route. Stdio selects an endpoint by
name and does not use its path. Several endpoints may reference one cluster, so
the example reuses one production connection at `/kafka-mcp/mcp` in read-only
mode and `/kafka-mcp/mcp/rw` in writable mode. Paths are exact: `/mcp` does not
capture `/mcp/rw`. `description` is optional and is reported by `server_config`
so a caller knows what the endpoint is intended for.

`http.base_path` prefixes endpoint paths and the liveness route. A leading or
trailing slash on an endpoint path is optional. Paths must be unique and may
not be `/`, `/healthz`, or contain a query or fragment. When `path` is omitted,
it defaults to `/mcp/<endpoint-name>`.

For compatibility, a file with no `endpoints` block still creates one endpoint
per cluster at `/mcp/<cluster-name>`. Existing cluster-level `read_only` and
`tools` values continue to apply as a lower bound during migration; an explicit
endpoint cannot widen them. New configurations should put both fields under
`endpoints`.

An endpoint may switch individual tools off, by name:

```yaml
endpoints:
  prod-read:
    cluster: prod
    path: /mcp
    read_only: true
    tools:
      create_topic: false
      commit_offset: false
```

Cross-cluster writes are granted per endpoint. `copy_message` and
`produce_message` may name a `destination_cluster`, and that cluster must be
listed in the calling endpoint's `destinations`:

```yaml
endpoints:
  prod-read:
    cluster: prod
    read_only: true
    destinations: [preprod]   # this session may copy into preprod, nowhere else
```

An endpoint with no `destinations` writes to no other cluster. A listed cluster
configured `read_only` still refuses, and listing the endpoint's own cluster is
an error, because that is governed by `read_only`. Names are checked at
startup. A file with no `endpoints` block keeps its old behaviour: each
cluster's endpoint lists every other cluster. Clusters that no endpoint serves
are invisible to `list_clusters`, `compare_clusters` and `destination_cluster`.

A tool the map does not mention stays exposed, so the file states only what it
withholds rather than relisting every tool and silently losing whatever is added
later. Names are exact and lowercase, as listed by `server_config`. They are
checked at startup: a name that is not a tool stops the
server, because a typo would leave the tool it was meant to withhold exposed.
`server_config` cannot be switched off, since it is how a session learns which
cluster and policy it reached and which tools that endpoint has. The `tools`
map only narrows an endpoint: `read_only: true` still withholds the writing
tools regardless of what the map says.

A minimal local file:

```yaml
clusters:
  local:
    brokers: localhost:19092
    schema_registry:
      urls: ["http://localhost:18081"]
endpoints:
  local:
    cluster: local
    path: /mcp/local
```

Run a custom configuration over stdio with
`CONFIG_FILE=/path/to/config.yaml go run ./cmd/server`. If it defines several
endpoints, select one with `--endpoint <name>`. Add `--server` to serve every
configured endpoint over HTTP instead; `--endpoint` is not used in HTTP mode.
Without `CONFIG_FILE`, chu discovers `kafka-mcp.{toml,yaml,yml,json}` first in
the working directory, then in the operating system's user config directory
(`~/.config/kafka-mcp/` on Linux), and finally in `/etc`. It uses the first
matching file rather than merging files. Its standard loader order is defaults,
file, HTTP, then environment; environment overrides use the `KAFKA_MCP_` prefix (for example,
`KAFKA_MCP_HTTP_ADDRESS=:9000` or
`KAFKA_MCP_HTTP_BASE_PATH=/kafka-mcp`). Logging can be configured with
`LOG_LEVEL` and `LOG_PRETTY`.

At least one cluster with a broker is required. `http.address` defaults to
`:8090`, `http.base_path` defaults to the HTTP root, and `output_dir` defaults
to the system temp directory. Exports are confined to that directory:
`output_file` takes a new file name, never a path; an existing name is refused
rather than overwritten.

`brokers` accepts either one address as a scalar or several addresses as a YAML
list. Each address is passed to Kafka as a separate seed broker.

Keep secrets out of the file with `{env:VAR}` or `password_file`. Unknown fields
are ignored by chu.

TLS is configured with `twmb/tlscfg`, using its TLS 1.2 minimum and recommended
cipher suites. TLS uses the system trust store;
`ca_file` adds a custom CA. For mTLS, supply both `cert_file` and `key_file`.
`security.sasl` is a preference-ordered list: each entry enables either `scram`
or `plain`, or uses `oauth` as shown below. For PLAIN, use `plain: {enabled: true, user: alice, pass: "{env:KAFKA_PASSWORD}"}`.
Both accept optional `zid` (authorization identity) and `password_file` instead
of `pass`; SCRAM also accepts `is_token: true` for delegation tokens. Algorithms
are case-insensitive. Disabled entries are ignored; repeated mechanisms and
entries enabling multiple mechanisms are rejected. Fallback negotiates a
broker-supported mechanism; it does not retry invalid credentials.
All Kafka connections, including message-reading sessions, use these settings.

Legacy per-cluster `tls` and `sasl: {mechanism, user, password, password_file}`
remain supported, but cannot be combined with `security` on the same cluster.

For OAUTHBEARER client credentials, use this cluster security block:

```yaml
security:
  tls:
    enabled: true
  sasl:
    - oauth:
        enabled: true
        token_url: https://identity.example.com/realms/apps/protocol/openid-connect/token
        client_id: kafka-mcp
        client_secret: "{env:KAFKA_CLIENT_SECRET}"
        scopes: [kafka]
        timeout: 10s
        # zid: optional-authorization-id
        # extensions: {tenant: example}
```

Alternatively, use `oauth: {enabled: true, token: "{env:KAFKA_TOKEN}"}` for a
static token. Configure exactly one mode: static `token`, client credentials, or
`gcp`. Client-credentials tokens are fetched
on authentication, cached per cluster ac