Standard

S3 Directory Bucket

A specialized S3 bucket type with a hierarchical directory namespace — forward slash is a true directory boundary, not a delimiter — optimized for high-request-rate workloads. Required for S3 Express One Zone. Bucket names carry an AZ suffix (`--x-s`). ETags are random alphanumeric strings, not MD5. `ListObjectsV2` abandons lexicographical sorting for raw throughput. Multipart uploads require consecutive part numbers. Object tags are unsupported.

9 connections 5 resources 1 post

Summary

What it is

A specialized S3 bucket type with a hierarchical directory namespace — forward slash is a true directory boundary, not a delimiter — optimized for high-request-rate workloads. Required for S3 Express One Zone. Bucket names carry an AZ suffix (`--x-s`). ETags are random alphanumeric strings, not MD5. `ListObjectsV2` abandons lexicographical sorting for raw throughput. Multipart uploads require consecutive part numbers. Object tags are unsupported.

Where it fits

Traditional S3 uses a flat namespace where prefix-based listing scans all matching keys. Directory buckets use actual directories with fast LIST operations and materialize `dir1/dir2/` paths during PutObject, eliminating the listing bottleneck for workloads with millions of objects in deep hierarchies. This is the infrastructure layer beneath S3 Express One Zone's single-digit millisecond latency.

Misconceptions / Traps
  • Directory buckets are not general-purpose S3 buckets. No object tags means tag-driven lifecycle rules don't apply. No MD5 in ETags breaks integrity checks that assume the standard ETag shape.
  • Zonal endpoints demand AZ affinity. Cross-AZ compute access causes severe latency outliers — empirically up to 7,570ms for PUT operations in misconfigured Next.js CDN deployments.
  • Single-AZ only. Not appropriate for primary storage without replication.
  • Deleting an object recursively removes empty parent directories, unlike flat-namespace buckets where virtual prefixes persist after deletion. Don't rely on prefix presence as a sentinel.
Key Connections
  • enables S3 Express One Zone — required bucket type
  • solves Object Listing Performance — hierarchical namespace eliminates prefix scanning
  • solves Directory Namespace / Listing Bottlenecks — real directory primitives, not virtual prefixes
  • alternative_to S3 API — departs from the flat-namespace contract in several places
  • constrained_by S3 Compatibility Drift — behavior diverges from standard buckets in ways that break some clients

Definition

What it is

A specialized S3 bucket type that uses a hierarchical directory-based namespace instead of S3's traditional flat key-value namespace. The forward slash operates as a functional directory boundary rather than a delimiter character — uploading `dir1/dir2/file1.txt` materializes real `dir1/` and `dir2/` directory constructs during PutObject. Required for S3 Express One Zone and optimized for millions of requests per second per bucket.

Why it exists

S3's flat namespace uses prefix-based listing that scales poorly for workloads with millions of objects in deep hierarchies. Directory buckets provide actual directory structure with fast listing, eliminating the prefix-scan bottleneck for high-throughput analytical and AI workloads.

Primary use cases

High-performance storage for S3 Express One Zone, latency-sensitive ML checkpointing, Spark/Trino cache tier with fast metadata operations, GPU training checkpoint synchronization across thousands of nodes.

Connections 9

Outbound 8
Inbound 1

Resources 5

Featured in