Unity Catalog

Latest OSS release: v0.5.1 (July 2026). The open-source Unity Catalog server (unitycatalog/unitycatalog) — v0.5.0 (GA 2026-06-18) added the UC Delta API for managing Delta tables, per-Spark-version artifacts, and credential-scoped filesystems by default. Distinct from the Databricks-managed UC service described below. Per unitycatalog releases.

Governance primitives reach GA across all three clouds. Per the April 2026 Databricks on Google Cloud release notes, Data Classification and Governed Tags both went GA in April 2026 — the two surface features most organizations bisect "is Unity Catalog production-ready?" against. The May 2026 Databricks on AWS release notes ship Catalog commits to GA, plus app telemetry persistence into Unity Catalog tables (observability becomes a first-class governed dataset, not a side channel). The cumulative effect: UC has crossed the "feature-incomplete" → "default expectation" threshold faster than the open-source counterparts (Apache Polaris, Iceberg REST Catalog) have shipped equivalents.

MANAGE permissions auto-propagate to materialized views + streaming tables. Per the Lakeflow / Spark Declarative Pipelines 2026 release notes, as of March 2026 Unity Catalog auto-propagates MANAGE permissions from base tables to derived MVs and streaming tables — eliminating a real class of permission-drift incidents that previously left downstream artifacts with stale grants. Pipeline settings can now also live inside Unity Catalog table properties rather than as external YAML, collapsing the catalog-vs-config split that's been a quiet operational drag.

CRTAS optimized writes default + Python UDTF custom-dependency support. Per the Databricks SQL 2026 release notes, optimized writes for CRTAS (CREATE OR REPLACE TABLE AS SELECT) on Unity Catalog tables flipped to default in March 2026 — a quiet but meaningful perf improvement for any pipeline that does idempotent table rebuilds. Unity Catalog Python UDTFs now support custom dependencies, removing the "you can only use what's in the workspace runtime" constraint that pushed teams toward external compute for non-standard libraries.

Data + AI Summit (July 20) consolidated the multi-format story — and added cross-engine ABAC. The summit packaged Managed Iceberg GA and Iceberg v3 GA (detailed below) together with attribute-based access control that follows the data across engines — the governance boundary extending beyond Databricks' own runtime to any engine reading through the REST catalog. Per the Data + AI Summit 2026 catalog roundup (dataengineerhub, aggregator); primary Databricks confirmation of the ABAC scope pending.

Full Iceberg GA plus managed MCP servers make Unity Catalog an agent-facing control plane. Databricks moved Managed Iceberg, Foreign Iceberg, and Iceberg v3 to general availability, with Unity Catalog implementing the Iceberg REST Catalog API so external engines read and write managed Iceberg tables through the open standard — UC stops being a Delta-first catalog and becomes a genuinely multi-format one. In parallel, Databricks' managed MCP servers expose Unity Catalog tables, functions, and Vector Search indexes natively to AI agents, so the same governance boundary that scopes human access now scopes agent access. The catalog, not the engine, becomes the place both people and agents are granted (or denied) data. Per Databricks — Unity Catalog and the next era of Apache Iceberg and Databricks — Expanded interoperability with Unity Catalog Open APIs.

Unity Catalog now publishes to Microsoft Fabric (Public Preview, June 2026), extending interoperability beyond the Iceberg REST Catalog to Databricks' primary BI competitor's catalog surface. Personal-access-token auto-scoping also reached GA in the same release. Per Azure Databricks — June 2026 release notes.

Lakehouse Sync brings CDC from Lakebase (Databricks' operational Postgres) into Unity Catalog Delta tables (Public Preview, May 2026), closing the loop between OLTP and the governed lakehouse without a separate ETL pipeline. Per Azure Databricks — May 2026 release notes.

Predictive Optimization automatically tunes Unity Catalog-managed tables — AI-driven file-size optimization, query-pattern-based clustering, and automatic statistics collection for data skipping and join planning, without manual tuning. Per InvGate — Databricks Unity Catalog (tertiary).

The governance gap: deep inside Databricks compute, thin outside it. Unity Catalog's AI-integrated governance suite — lineage, ABAC, Predictive Optimization — is most complete when the compute reading it is Databricks' own; achieving comparable governance across non-Databricks engines and other clouds still leans on external supplementary tools rather than native UC coverage, the structural trade-off against Polaris's and Lakekeeper's engine-agnostic-by-design posture. Per Iceberg REST Catalog, Actually Explained (Datastrato).

The openness gap, quantified. Community analysis puts Unity Catalog OSS's org-concentration index at 0.99 — over 99% of commits from Databricks — in the Linux Foundation's loose sandbox tier, and the managed catalog implements the Iceberg REST API one-directionally: external engines can read Unity-governed tables, but Unity does not accept vended short-lived credentials to read or write foreign Iceberg tables, so federating outward still means managing static cloud credentials. Per LakeOps — best catalog for Apache Iceberg and Security Data Works — the catalog decision. Sources: Databricks — Unity Catalog and the next era of Apache Iceberg · Databricks — Expanded interoperability with Unity Catalog Open APIs · April 2026 release notes — Databricks on Google Cloud (Data Classification + Governed Tags GA) · May 2026 release notes — Databricks on AWS (Catalog commits GA) · Lakeflow Spark Declarative Pipelines 2026 release notes · Databricks SQL 2026 release notes · TPC-DS sample dataset for Unity Catalog workspaces (Databricks docs)