Amazon S3 Metadata
An AWS feature that automatically generates queryable metadata tables (in Apache Iceberg format) over S3 objects, enabling SQL-based discovery and governance of object metadata.
Summary
An AWS feature that automatically generates queryable metadata tables (in Apache Iceberg format) over S3 objects, enabling SQL-based discovery and governance of object metadata.
S3 Metadata bridges the gap between S3's minimal per-object metadata and the rich, queryable metadata that data governance requires. It automatically creates Iceberg tables from object metadata, queryable via Athena or Spark.
- Not the same as user-defined S3 tags or custom metadata headers. S3 Metadata creates actual Iceberg tables containing system-generated metadata that can be queried with SQL.
- Metadata tables are generated asynchronously. There is a delay between object creation and metadata availability in the Iceberg table.
solvesObject Listing Performance — SQL queries replace expensive LIST operationsscoped_toMetadata-First Object Storage — the AWS implementation of metadata-first designscoped_toMetadata Management — automated metadata generation and querying
Definition
AWS feature that automatically generates and maintains a queryable metadata table (Apache Iceberg format) for all objects in a bucket, making object metadata SQL-queryable.
S3 stores billions of objects but provides limited metadata queryability. S3 Metadata surfaces system and custom metadata as queryable Iceberg tables, enabling SQL-based discovery and governance at scale.
Data governance at scale, SQL-based object discovery, automated metadata-driven lifecycle management, compliance auditing.
Connections 7
Outbound 7
implements1constrained_by1Resources 3
S3 Metadata feature overview for automated metadata discovery, enrichment, and querying across S3 buckets.
Official S3 Metadata user guide covering table bucket integration, schema configuration, and query patterns.
Launch blog explaining how S3 Metadata enables automatic metadata capture and SQL-based querying of object properties.