Technology

AWS S3

Amazon's fully managed object storage service — the origin and reference implementation of the S3 API. As of December 2025, the maximum object size is 50 TB (up from 5 TB).

36 connections 4 resources

Summary

What it is

Amazon's fully managed object storage service — the origin and reference implementation of the S3 API. As of December 2025, the maximum object size is 50 TB (up from 5 TB).

Where it fits

AWS S3 is the gravitational center of the ecosystem. It defined the API that became the de-facto standard, and most tools in this index were built to work with AWS S3 first and other providers second. The 50 TB object limit shift means massive AI training datasets and 8K video can now be stored as single atomic objects rather than complex multi-part sequences.

Misconceptions / Traps

AWS S3 is now strongly consistent (read-after-write), but code written against the old eventual consistency model may still contain unnecessary workarounds.
S3 storage is cheap; S3 API calls and egress are not. Cost optimization requires understanding request pricing and transfer charges, not just storage GB.
The 50 TB limit applies to individual objects; existing tooling that splits datasets at the old 5 TB boundary may need updating.

Key Connections

implements S3 API — the reference implementation of the standard
enables Lakehouse Architecture — provides the storage layer for lakehouses
enables Separation of Storage and Compute — foundational to the pattern
used_by Medallion Architecture — each layer stores data on S3
constrained_by Object Listing Performance, Lack of Atomic Rename, Egress Cost — key operational limitations

Definition

What it is

Amazon's fully managed object storage service. The original implementation that defined the S3 API and established object storage as a category. As of December 2025, supports individual objects up to 50 TB (increased from 5 TB), enabling massive AI training datasets and high-resolution media to be stored as single atomic objects.

Why it exists

To provide scalable, durable, low-cost storage accessible over HTTP, decoupled from any specific compute or filesystem.

Primary use cases

Data lake storage, static asset hosting, backup and archival, analytics data staging, ML training data storage, large-object AI dataset ingest (50 TB single-object).

Recent developments

Latest signals

S3 Files GA — April 7, 2026. First and only cloud object store providing fully-featured high-performance file system access; caches active data for low-latency access; up to multi-TB/sec aggregate read throughput. AWS's bet against Mountpoint for Amazon S3 + s3fs-fuse. Per AWS What's New — S3 Files.
S3 Tables in AWS GovCloud (February 2026). S3 Tables (managed Apache Iceberg) now available in AWS GovCloud (US-East + US-West) — extends the managed-Iceberg story into the FedRAMP / DoD-compliant tier. 10× higher TPS vs Iceberg tables in general-purpose buckets. Per AWS What's New — S3 Tables in GovCloud.
S3 Tables 90% compaction cost cut (July 2025). Compaction processing fees reduced significantly: per-object price 50% lower; per-byte 90% lower for binpack, 80% for sort/z-order. Took effect July 1 2025, automatic in all S3 Tables regions. Per AWS — S3 Tables features.
50TB single-object limit since Dec 2025. Individual object size limit raised from 5TB → 50TB enables ingesting massive AI training datasets and high-resolution media as single atomic objects (previously required multi-part upload + reassembly). Per AWS — S3 Features.
23× per-GB cost gap between storage classes. S3 Standard vs Glacier Deep Archive is a 23× price ratio per GB-month; Express One Zone vs Standard is 5× the other direction (more expensive). The 2026 storage-class menu is wider than 2024. Per Cloud Burn — 23× cost gap.
Intelligent-Tiering remains the cost-optimization default for irregular-access workloads. Automatically moves data to most cost-effective access tier based on access frequency, with sub-second latency on the hot tier. IA tier saves up to 40% on storage; Archive Instant tier saves up to 68%. Per AWS — Object Storage Classes.