MCP Gateway | LLMS3

Definition

What it is

A specialized, state-aware reverse proxy purpose-built for the **Model Context Protocol** — managing bidirectional Server-Sent-Events streams, multiplexing tools across many backend MCP servers, applying semantic caching to LLM tool calls, enforcing per-tool authorization policies, and providing observability (token-cost telemetry, per-tool latency histograms, audit logs) at the boundary between agentic clients and the federated MCP server fleet.

Why it exists

Traditional API gateways (Kong, Apigee, AWS API Gateway in REST mode, NGINX in passive proxy mode) are architecturally incompatible with MCP traffic. They were built for stateless request-response: route by URL + method, apply policy to the request, return the response, forget. MCP traffic is the opposite — a single client POST opens a persistent SSE stream over which many JSON-RPC events flow in both directions; routing decisions must consider session state, tool schemas are discovered dynamically at runtime (not at gateway-config time), and policies must apply *per-event* within the stream. Native MCP Gateways are the architectural fix.

Recent developments

Latest signals

Bifrost shipped as an open-source native MCP Gateway. Provides federated MCP server registration, semantic caching, token-cost telemetry, and OAuth 2.1 / Cross-App Access enforcement. Per the Bifrost repo (maximhq).
Tyk introduced an MCP Gateway product. Tyk's positioning argues MCP Gateways are the natural successor generation to traditional API gateways. Per Tyk — Why MCP Gateways Are the Next Evolution.
AWS launched Amazon API Gateway MCP proxy capability. Translation-layer mode — wraps REST APIs and exposes them as MCP-compatible endpoints without backend changes, with semantic-search-based tool discovery over the wrapped REST endpoints. Per AWS — Amazon API Gateway MCP proxy support.
The New Stack + Tyk + others document the gateway divergence formally. "MCP Gateways and API Gateways are not interchangeable" became the consensus position; legacy REST gateways treating SSE streams as opaque passthrough lose all the observability + governance that production MCP requires. Per The New Stack — MCP vs API Gateways: Not Interchangeable.