Iceberg REST Catalog Spec
An open REST API specification for Apache Iceberg catalog operations — namespace/table listing, metadata load, commit, snapshot management — enabling multi-engine interoperability through a standardized HTTP-based catalog interface. Extended in practice with **credential vending**, where the catalog mints prefix-scoped, short-lived S3 credentials at table-load time.
Summary
An open REST API specification for Apache Iceberg catalog operations — namespace/table listing, metadata load, commit, snapshot management — enabling multi-engine interoperability through a standardized HTTP-based catalog interface. Extended in practice with **credential vending**, where the catalog mints prefix-scoped, short-lived S3 credentials at table-load time.
The REST Catalog Spec solves the catalog fragmentation problem in the Iceberg ecosystem. Instead of every engine needing native support for Hive Metastore, Glue, Nessie, etc., any catalog that implements the REST spec becomes accessible to all REST-capable engines. This is also the wire that lets a local-first engine like DuckDB attach directly to an Amazon S3 Tables bucket (`ATTACH '<arn>' AS cat (TYPE iceberg, ENDPOINT_TYPE s3_tables)`) or metadata-clone via `iceberg_to_ducklake(...)` — bypassing heavy distributed compute for interactive querying of multi-terabyte remote tables.
- The REST Catalog Spec defines the API contract, not the catalog implementation. Performance, consistency, and feature completeness depend on the catalog server behind the API.
- Not all Iceberg catalog operations may be supported by every REST catalog implementation. Check compatibility for advanced features like branching, tagging, and view support.
- Credential vending is not part of the base spec — it's a widely-adopted extension (Apache Polaris, Unity Catalog, S3 Tables). Check whether your client understands the vended-credential response shape before assuming it "just works."
scoped_toIceberg Table Spec, Table Formats — standardizes catalog access for Icebergused_byDuckDB — the direct-attach path for local-first analytics over S3 TablessolvesVendor Lock-In — engine-agnostic catalog accesssolvesMetadata Overhead at Scale — enables centralized catalog management
Definition
An open specification defining a RESTful HTTP API for Iceberg catalog operations — listing namespaces and tables, loading table metadata, committing updates, and managing snapshots — independent of any specific catalog backend. Extended in practice with **credential vending**, where the catalog mints short-lived, prefix-scoped S3 credentials at table-load time so clients never hold broad long-lived keys.
Iceberg catalogs were historically tied to specific implementations (Hive Metastore, AWS Glue, Nessie). The REST Catalog Spec decouples catalog clients from catalog backends, enabling multi-engine, multi-language interoperability through a universal HTTP interface.
Multi-engine Iceberg catalog access, vendor-neutral catalog interoperability, cloud-managed Iceberg catalogs, cross-language catalog clients, local-first analytics (DuckDB `ATTACH '<s3_tables_arn>' AS cat (TYPE iceberg, ENDPOINT_TYPE s3_tables)`), DuckLake metadata-only cloning (`CALL iceberg_to_ducklake(...)`) for interactive querying of multi-terabyte remote tables.
Connections 11
Outbound 5
Resources 6
OpenAPI specification for the Iceberg REST Catalog, the standard API for multi-engine catalog interoperability.
Iceberg catalog concepts documentation explaining catalog types, REST catalog design, and configuration.
Introduction to the Iceberg REST Catalog from Tabular (founded by Iceberg creators) explaining design motivations and architecture.
DuckDB Iceberg extension documentation showing the `ATTACH ... TYPE iceberg ENDPOINT_TYPE s3_tables` pattern for local-first analytics over remote S3 Tables buckets.
Reference for DuckDB's REST Catalog support and the `iceberg_to_ducklake` metadata-only clone path for interactive querying of multi-terabyte remote tables.
AWS walkthrough of DuckDB → S3 Tables via REST Catalog credential vending, including predicate pushdown and partition pruning behavior.