Definition

What it is

An open-source **AI gateway** (MIT-licensed) sitting between the agent runtime and foundation models. Provides observability (per-call traces persisted to S3), cost analytics, semantic caching, and unified routing across LLM providers. Repository: [github.com/Helicone/ai-gateway](https://github.com/Helicone/ai-gateway). The Helicone Gateway launch (June 2025) marked the open-source side of the model-gateway category catching up to managed-product alternatives.

Why it exists

AI workloads at production scale need observability that doesn't exist in the underlying provider APIs — per-request latency, cost attribution, prompt/response logging, semantic-cache hit rates, multi-provider failover decisions. Helicone wraps the gateway pattern with first-class observability that streams to durable S3-backed storage for downstream lakehouse analysis.

Primary use cases

Per-call LLM observability traces persisted to S3, cost attribution across multi-tenant agentic deployments, semantic caching with S3 backends, prompt/response auditing for compliance pipelines, model-routing decisions based on real-time cost/latency signals.