Credential Vending
A security architecture where a control plane issues short-lived, narrowly scoped S3 credentials at query time rather than relying on long-lived IAM roles or bucket policies for data access.
Summary
A security architecture where a control plane issues short-lived, narrowly scoped S3 credentials at query time rather than relying on long-lived IAM roles or bucket policies for data access.
The mechanism behind Iceberg REST Catalog's credential vending endpoint, Unity Catalog's external-location tokens, and Apache Polaris's scoped access. This is how modern lakehouse catalogs enforce fine-grained (table/row/column) access control over data stored in S3 without requiring consumers to have direct bucket access.
- Not the same as pre-signed URLs — credential vending issues full STS tokens scoped to specific S3 prefixes, supporting reads, writes, and listing.
- Requires a catalog that understands table-level metadata to scope tokens correctly. Without a catalog, credential vending degrades to bucket-level access.
- Token refresh and caching are performance-critical. Poorly implemented vending can add 50–200ms per query.
- Core enabler for Apache Polaris, Unity Catalog, and Iceberg REST Catalog Spec security models.
- Solves Policy Sprawl by centralizing access decisions in the catalog rather than in IAM.
- Enables Tenant Isolation without per-tenant buckets or prefix-based IAM policies.
Definition
An access control architecture where a central service issues short-lived, scoped credentials (typically STS tokens or signed URLs) to data consumers at query time, rather than granting persistent IAM roles or long-lived keys to storage buckets.
Traditional S3 access patterns use long-lived IAM credentials or bucket policies that grant broad access. Credential vending creates a control plane that evaluates policies (table-level, row-level, column-level) and mints temporary credentials scoped to exactly the S3 prefixes and objects a query needs. This is the mechanism behind Iceberg REST Catalog credential vending and how Unity Catalog and Apache Polaris enforce fine-grained access.
Multi-tenant lakehouse deployments, cross-engine S3 access with uniform security, row/column-level access control enforcement, zero-trust data lake architectures.
Connections 6
Outbound 6
scoped_to2enables2Resources 3
Iceberg REST Catalog specification including the credential vending endpoint that enables scoped S3 token issuance per table/namespace.
Unity Catalog open-source documentation covering external-location credential vending for Delta Lake tables on S3.
AWS Lake Formation credential vending documentation — the AWS-native implementation of scoped temporary credentials for data lake access.