Architecture

Interoperability Patterns

Architectural strategies for enabling multiple table formats (Iceberg, Delta, Hudi), query engines (Spark, Trino, Flink), and catalogs to operate on the same S3-stored data without requiring format conversion or data duplication.

6 connections 3 resources

Summary

What it is

Where it fits

Interoperability patterns address the fragmentation challenge in the S3 lakehouse ecosystem. Tools like Apache XTable, Delta UniForm, and Iceberg REST Catalog enable organizations to avoid lock-in to a single table format or query engine while keeping a single copy of data on S3.

Misconceptions / Traps

Format interoperability (XTable, UniForm) adds a metadata translation layer. Each format's metadata must be kept in sync, and features unique to one format (e.g., Iceberg's hidden partitioning) may not translate cleanly.
"Read interoperability" is easier than "write interoperability." Multiple engines reading Iceberg tables is well-supported; multiple engines writing concurrently to the same Iceberg table requires careful commit conflict resolution.
Catalog interoperability is as important as format interoperability. An engine that reads Iceberg but uses HMS while another uses Glue will see different table states unless catalogs are synchronized.

Key Connections

scoped_to Table Formats, Lakehouse — cross-format and cross-engine compatibility
enabled_by Apache XTable, Delta UniForm — format translation tools
enabled_by Iceberg REST Catalog Spec — standardized catalog interface
solves Vendor Lock-In — reduces dependence on a single format or engine

Definition

What it is

Architectural patterns that enable multiple table formats (Iceberg, Delta, Hudi), query engines, and catalogs to coexist on the same S3 data — including format translation (Apache XTable, Delta UniForm) and multi-catalog federation.

Why it exists

Organizations rarely standardize on a single table format or query engine. Interoperability patterns enable teams to use the best tool for each job while sharing the same physical data on S3, avoiding costly format migrations and data duplication.

Primary use cases

Multi-format lakehouse environments, incremental migration from Delta to Iceberg, cross-engine analytics on shared S3 data.

Connections 6

Outbound 6

scoped_to2

S3 Table Formats

depends_on2

Apache XTable Delta UniForm

solves2

Vendor Lock-In S3 Compatibility Drift

Resources 3

DocsHigh

xtable.apache.org/

Apache XTable documentation for cross-format interoperability between Iceberg, Delta Lake, and Hudi without data duplication.

DocsHigh

docs.databricks.com/aws/en/delta/uniform

Delta UniForm documentation enabling Delta tables to be read as Iceberg or Hudi, the key vendor-led interoperability mechanism.

DocsHigh

iceberg.apache.org/docs/latest/rest-catalog/

Iceberg REST Catalog specification enabling vendor-neutral catalog access, a foundation for cross-engine interoperability on S3.