Databricks
A unified data + AI platform built on Apache Spark and Delta Lake, with a managed lakehouse covering data engineering, SQL analytics, ML/AI, and (as of 2026) operational application data via **Lakebase**. Originated the lakehouse architecture pattern.
Summary
A unified data + AI platform built on Apache Spark and Delta Lake, with a managed lakehouse covering data engineering, SQL analytics, ML/AI, and (as of 2026) operational application data via **Lakebase**. Originated the lakehouse architecture pattern.
Databricks sits as the commercial lakehouse-platform layer above S3-compatible object storage. The platform bundles cluster management, Delta Lake transactions, Unity Catalog governance, and a managed runtime, so operators can treat S3 as the system of record without managing the substrate. With Lakebase (May 2026 GA), the platform also serves operational app data colocated with the analytical lakehouse — a single product class that erases the operational/analytical wall.
- "Open formats = portable" is partially true. Delta Lake and Iceberg are open, but Photon engine, Unity Catalog deep integration, and Lakebase are platform-stickiness layers. Migration cost is real even with open table formats.
- Lakebase is not a relational DB replacement; it's an OLTP-shaped surface layered over the analytical lakehouse. Don't expect classical RDBMS features (referential integrity enforced by FKs, complex stored procedures, etc.).
- Databricks pricing is consumption-based (DBUs); poorly-tuned workloads or always-on clusters can cost more than self-managed Spark + S3 if not carefully managed.
scoped_toLakehouse, S3implementsLakehouse Architecture — coined the patterndepends_onApache Spark, Delta Lake, Unity Catalogcompetes_withSnowflake — the dominant platform-war axis in 2026
Definition
A unified data + AI platform built on Apache Spark and Delta Lake, with a managed lakehouse architecture spanning data engineering, SQL analytics, ML/AI, and (as of 2026) operational application data via **Lakebase**. Originated the lakehouse pattern. The platform is deeply integrated with cloud object stores (AWS S3, Azure Blob, GCS) — every Databricks workspace fundamentally reads/writes data on S3-compatible buckets.
Enterprises wanted SQL warehouse ergonomics with data-lake economics — running Spark and Hive on raw S3 was operationally heavy and lacked transaction guarantees. Databricks bundled cluster management, Delta Lake transactions, **Unity Catalog** governance, and a managed runtime so operators could treat S3 as the system of record without managing the substrate.
Production lakehouse pipelines, large-scale ETL with Delta Lake, ML model training on S3-resident data, SQL warehouse workloads with cost-efficient compute, real-time streaming with Spark Structured Streaming, and (with Lakebase) operational application backends colocated with the analytical lakehouse.
Recent developments
- Lakebase GA + operational/analytical convergence. Lakebase reached GA (per Databricks on AWS January 2026 release notes) with autoscaling and provisioned-unified compute, eliminating the boundary between operational and analytical systems — OLTP-shaped reads/writes for application data flow into the analytical lakehouse with no separate ETL. The May 2026 wave added Lakehouse Sync (Lakebase CDC) in Public Preview. Per Databricks AWS release notes (Jan/May 2026).
- May 2026 platform wave. Query-based connectors for Lakeflow Connect GA, Databricks Apps horizontal scaling (Beta), Lakeflow Designer updates (bidirectional AI descriptions, N-way Combine, custom joins), the sidebar's "SQL" section renamed "Lakehouse," Catalog Commits GA (keeps Unity Catalog consistent), and Anthropic Claude Opus 4.8 added as a Databricks-hosted model. Per Azure Databricks May 2026 release notes.
- Runtime and hosted-model cadence. January 2026 shipped Databricks Runtime 18.0 GA, serverless workspaces GA, Knowledge Assistant GA, and OpenAI GPT-5.2 Codex as a hosted model; April 2026 added Data Classification GA, a 5X-Large SQL warehouse (Public Preview), Unity Catalog function tagging, and Databricks Runtime 18.2 Beta. Per Databricks AWS release notes (Jan/Apr 2026).
- FabCon / Data + AI Summit 2026 — pricing and agentic push. FabCon 2026 introduced a Lakeflow Connect Free Tier (100 free DBUs/day, ~100M records) plus Lakebase GA and deep Microsoft (Power BI/Excel/Teams/Azure OpenAI) integration; the Summit framed the platform around agentic data/governance anchored under Unity Catalog and the new Unity AI Gateway.
- Unity Catalog open-catalog strategy holds. Databricks' open-sourced Unity Catalog (June 2024) ships Iceberg REST Catalog API support — external read GA, write in Public Preview — directly answering Snowflake's Apache Polaris. Combined with the Tabular acquisition (June 2025), Databricks influences both Delta Lake and Iceberg and argues for format convergence. Per Iceberg vs Delta 2026 analysis.
Connections 9
Outbound 9
implements1depends_on4Resources 3
Official Databricks platform site — overview of the unified data + AI platform, Lakebase, Delta Lake, Unity Catalog, Mosaic AI offerings.
Official Databricks documentation covering workspace administration, runtime versions, Delta Lake operations, Unity Catalog governance, and the Lakebase OLTP-shaped surface.
Lakebase-specific community articles including the launch announcement and operational/analytical convergence patterns covered in 2026.