Technology

Redpanda

A Kafka-compatible streaming platform written in C++ that provides a single binary deployment with built-in Tiered Storage to S3, eliminating the need for ZooKeeper/KRaft and JVM tuning.

9 connections3 resources2 posts

Summary

What it is

A Kafka-compatible streaming platform written in C++ that provides a single binary deployment with built-in Tiered Storage to S3, eliminating the need for ZooKeeper/KRaft and JVM tuning.

Where it fits

Redpanda replaces Apache Kafka in S3-centric architectures where operational simplicity and deterministic performance matter. Its native Tiered Storage writes log segments directly to S3, making it a natural fit for streaming ingestion into lakehouses.

Misconceptions / Traps
  • Kafka-compatible does not mean identical. Some Kafka features (exactly-once semantics across topics, specific Connect plugins) may behave differently or have limitations.
  • Redpanda's Tiered Storage to S3 is not the same as Kafka's KIP-405. The implementation, configuration, and recovery semantics differ.
  • Redpanda removes JVM complexity but introduces its own operational model. Capacity planning must account for Redpanda's thread-per-core architecture.
Key Connections
  • scoped_to S3, Object Storage — native S3 tiered storage
  • alternative_to Kafka Tiered Storage — Kafka-compatible alternative with built-in S3 offload
  • enables Event-Driven Ingestion — streaming platform for lakehouse ingestion
  • enables CDC into Lakehouse — transports CDC events to S3-based sinks

Definition

What it is

A Kafka-compatible streaming data platform written in C++, designed for lower latency and simpler operations. Supports tiered storage to S3 for cost-effective long retention.

Why it exists

Kafka's JVM-based architecture introduces operational complexity (ZooKeeper/KRaft, JVM tuning, garbage collection pauses). Redpanda provides wire-compatible Kafka APIs with a single binary, no JVM, and built-in S3 tiered storage for simplified streaming infrastructure.

Primary use cases

Kafka-compatible streaming with S3 tiered storage, low-latency event ingestion for lakehouse pipelines, simplified streaming infrastructure.

Recent developments

Latest signals

Source mix note: post-acquisition Redpanda corpus is dominated by third-party comparison and review content rather than primary engineering posts. The bullets below cite multiple independent surveys to triangulate concrete pricing and positioning data.

  • Latest release: v26.2.1 (current as of August 2026), succeeding v26.1.13 (July 2026); v25.3.x remains a supported stable line through March 2027. Note: v25.1 reached end-of-support April 7, 2026 — deployments on 25.1 or earlier should upgrade. Per Redpanda releases.

  • Redpanda Iceberg Topics auto-materialize Kafka topics as Apache Iceberg tables, no connector or ETL. Brokers persist topic data directly into Iceberg-format Parquet in object storage, making streams instantly queryable in Snowflake, Databricks, ClickHouse, and Redshift; Iceberg Topics reached GA in the Redpanda 25.1 release (April 7, 2025). Per Redpanda Announces the General Availability of Apache Iceberg Topics for the Enterprise.

  • Redpanda Streaming 26.1 named its engine: Redpanda One (R1), launched March 31, 2026. Billed as the industry's first adaptable streaming engine, R1 is a single multi-modal platform that lets enterprises tune performance, safety, and efficiency at the per-topic level — combining Write Caching, Tiered Storage, Iceberg Topics, and Cloud Topics — building on the zero-ETL Iceberg Topics foundation. Per Redpanda Streaming Delivers Industry's First Adaptable Streaming Engine (BusinessWire) and Redpanda launches streaming engine optimized for AI (TechTarget).

  • Cloud Topics' write pass-through: payloads go straight to object storage, Raft consensus stays on local NVMe. The mechanism behind R1's cost math — heavy message payloads stream directly to S3/GCS while only metadata and the Raft consensus path touch local high-performance disks, cutting cross-availability-zone replication traffic by up to 90% without the metadata lag of fully stateless brokers. This is the Agentic Data Plane pattern applied to streaming: the broker keeps coordination local and pushes bulk state to the storage tier. Per Redpanda Cloud Topics.

  • Redpanda Serverless reached general availability (February 2026). Zero-ops, scale-to-zero streaming on AWS with PrivateLink support — a consumption-priced entry point below the node-based enterprise licensing tier. Per Redpanda Serverless now Generally Available (Redpanda blog) and What's New in Redpanda Cloud (Redpanda docs).

  • Tiered Storage cost claims hold up under independent analysis. Per third-party cost analysis, Redpanda's S3 tiered storage delivers reported savings of $70K–$1.2M depending on retention window and throughput. The 3-6× cost-effectiveness vs Kafka cited by the vendor is roughly consistent with third-party calculators, though it requires Enterprise licensing for the full feature set (Tiered Storage, Remote Read Replicas, SSO, RBAC).

  • Confluent vs Redpanda pricing positioning — Redpanda cheaper at smaller scale. Per the modern-datatools comparison (March 2026), Redpanda Cloud is priced at ~$0.08/partition-hour vs Confluent's ~$0.012, but Redpanda needs fewer nodes for the same throughput — net cheaper for most workloads, though Confluent's managed connector catalog can shift the math for connector-heavy use cases. Gartner Peer Insights (May 2026) notes Redpanda pricing was 4× higher than competitors when last evaluated; enterprise license requirements for premium features impact TCO.

Connections9

Outbound7
Inbound2

Resources3

Featured in