agentgateway

Agentgateway is an open-source reverse proxy written in Rust that sits between AI agents, LLMs, and tools to provide centralized security, observability, and routing across agent-to-LLM, agent-to-tool, and agent-to-agent communication. It operates as an MCP gateway supporting stdio, HTTP, SSE, and Streamable HTTP transports, and can federate multiple MCP servers behind a single endpoint with OpenAPI integration and OAuth authentication. An LLM gateway layer routes traffic to providers including Anthropic, OpenAI, Gemini, and Bedrock through a unified OpenAI-compatible API with budget controls, load balancing, and failover. Guardrails span regex filtering, OpenAI moderation, AWS Bedrock Guardrails, and Google Model Armor. Authorization uses JWT, API keys, OAuth, and a CEL-based RBAC policy engine, while observability runs through OpenTelemetry metrics, logs, and tracing. A Kubernetes controller with Gateway API support handles cluster deployments, including inference routing based on GPU utilization and KV cache metrics. Platform engineers, MLOps teams, and developers building multi-agent systems with Claude or other LLMs benefit most from this project.

What people ask about agentgateway