Topic
S3
Amazon's Simple Storage Service and the broader ecosystem of S3-compatible object storage. The root concept of this entire index.
239 connections
4 resources
3 posts
Summary
What it is
Amazon's Simple Storage Service and the broader ecosystem of S3-compatible object storage. The root concept of this entire index.
Where it fits
Every node in the index answers the question "How does this relate to S3?" S3 is not just a product — it is the API, the paradigm, and the ecosystem that the rest of the map is built around.
Misconceptions / Traps
- S3 is not a filesystem. It has no directories, no atomic rename, and no POSIX semantics. Treating it like a filesystem causes subtle bugs.
- "S3-compatible" does not mean identical. Consistency guarantees, performance characteristics, and feature coverage vary across providers.
Key Connections
- Root topic — all other Topics connect inward via
scoped_to - Object Storage
scoped_toS3 — S3 is the dominant implementation of the object storage paradigm - S3 API
scoped_toS3 — the HTTP interface that defines the ecosystem - AWS S3
scoped_toS3 — the origin and reference implementation
Definition
What it is
The conceptual root of this index. Encompasses Amazon S3, the ecosystem of S3-compatible object storage, the technologies built on top of it, and the engineering problems that arise at scale.
Why it exists
S3 created the object storage paradigm that underpins modern data infrastructure. Every node in this index answers the question: "How does this relate to S3?"
Recent developments
Latest signals
- S3 turned 20 in March 2026 — AWS's reframing positions S3 as the AI-era foundation. AWS Weekly Roundup (March 16, 2026) celebrated S3's 20th anniversary with a coordinated set of AI-era launches. The narrative: S3 isn't just storage anymore; it's the substrate the modern AI stack runs on. Per AWS — AWS Weekly Roundup: Amazon S3 Turns 20 (March 16, 2026) and GeekWire — Amazon Revamps S3 Cloud Storage for the AI Era.
- S3 Files GA in 34 regions (April 2026) — buckets accessible as file systems. S3 Files delivers a shared file system that connects any AWS compute resource directly with S3 data. File-based applications + agents + teams access + work with S3 data as a file system using existing POSIX tools. Per AWS — Announcing Amazon S3 Files (April 2026).
- S3 Vectors GA December 2025 — first cloud object storage with native vector indexing. Up to 2B vectors per index, query latency <100ms, up to 90% cost reduction vs dedicated vector DBs. The "object storage with semantic-search built in" architectural primitive. Per Medium — The 2026 AWS Evolution: How S3 is Reclaiming the Heart of the AI Era.
- Individual object size limit raised from 5 TB → 50 TB. Specifically designed for foundation-model training datasets — the previous 5 TB cap forced workarounds for petabyte-scale dataset shards. The 10× increase signals AWS sizing for AI-training reality, not legacy enterprise patterns. Per Medium — 2026 AWS Evolution: S3 Reclaiming AI Era.
- Account-level Regional Namespaces (March 2026). New account-level namespaces let bucket names match across accounts — simplifies IaC + predictable naming across regions. Closes a long-standing pain point in multi-account architectures. Per AWS — AWS Weekly Roundup March 16, 2026.
- SSE-C disabled by default for new buckets (April 2026). Response to the Codefinger SSE-C ransomware campaign — new general-purpose buckets reject SSE-C write requests by default. Defensive default that closes the ransomware vector at the platform level. Per AWS — S3 Starts Rolling Out New Security Best Practice by Default (April 2026).
Connections 239
Inbound 239
scoped_to233Topic21
Object StorageData LakeTable FormatsVector Indexing on Object StorageLLM-Assisted Data SystemsMetadata ManagementData VersioningDirectory Buckets / Hot Object StorageObject Storage for AI Data PipelinesKubernetes Object Provisioning & PolicyMetadata-First Object StorageGeo / Edge Object StorageTime TravelSovereign StorageAI Memory InfrastructureRetrieval EngineeringInference LocalityAI Runtime InfrastructureAI Memory GovernanceGPU + Object Storage ConvergenceDistributed Context Systems
Technology102
AWS S3MinIOpgsty/minio ForkVersity S3 GatewayCephApache OzoneDuckDBSpice.aiTrinoClickHouseApache SparkLanceDBOpenSearchStarRocksApache FlinkS3 Express One ZoneAmazon S3 TablesAmazon S3 VectorsAmazon S3 MetadataAWS LambdaAmazon S3 FilesSeaweedFSCloudflare R2Backblaze B2IDrive e2HexabyteOVHcloud Object StorageAliyun OSSTencent COSHuawei OBSGoogle Cloud StorageVAST DataDell ECSNetApp StorageGRIDPure Storage FlashBladeHitachi VantaraHPE Alletra Storage MP X10000GarageAlluxioOpenDALlakeFSRookGeeseFSApache PolarisApache GravitinoUnity CatalogApache XTableDelta UniFormApache PaimonFlink CDCEstuary FlowAlarikRustFSMarquezApache RangerS3 Bucket KeyWarpStreamApache DorisAWS Glue CatalogHive MetastoreDremioDatabricksAthenaDebeziumDataFusionPolarsKafka Tiered StorageRedpandaProject NessieAirbyteSpark Structured StreamingVeloxdltOpenMetadataDataHubApache AtlasrcloneTailscaleMem0ZepGraphitiLMCacheSGLangMooncakeVestigeLangGraphLiteLLMHelicone AI GatewayTraefik AI GatewayNVIDIA BlueField-4Inference Context Memory Storage (ICMS)NVIDIA cuObjectResticAliyun CPFS + OSS HybridCacheyHS5AWS CLIBoto3S3cmdminikvDigitalOcean AI-Native CloudDataKit (Guance Cloud)
Standard16
Architecture51
Separation of Storage and ComputeHybrid S3 + Vector IndexOffline Embedding PipelineLocal Inference StackWrite-Audit-PublishTiered StorageGeo-Dispersed Erasure CodingNVMe-backed Object TierGPU-Direct Storage PipelineCache-Fronted Object StorageCheckpoint/Artifact Lake on Object StorageTraining Data Streaming from Object StorageFeature/Embedding Store on Object StorageOnline Embedding Refresh PipelineMulti-Site ReplicationActive-Active Multi-Site Object ReplicationEdge-to-Core Object AggregationImmutable Backup Repository on Object StorageRansomware-Resilient Object Backup ArchitectureDeletion VectorLSM-tree on S3CompactionCDC into LakehouseRow / Column SecurityEncryption / KMSTenant IsolationRAG over Structured DataClustering / Sort OrderFile Sizing StrategyAudit TrailsPII TokenizationBatch vs StreamingEvent-Driven IngestionManifest PruningCompliance-Aware ArchitecturesBranching / TaggingAI-Safe ViewsStructured ChunkingBenchmarking MethodologyCapacity PlanningHybrid Metadata PatternsInteroperability PatternsNon-Blocking Concurrency ControlDecoupled Vector SearchPartitioningCredential VendingObject Lifecycle ManagementLakehouse for AI WorkflowsMultimodal Object StorageRedaction LayersObjectCache
Pain Point34
Small Files ProblemCold Scan LatencyLegacy Ingestion BottlenecksHigh Cloud Inference CostData Loading BottleneckObject Listing PerformancePartition Pruning ComplexityVendor Lock-InAGPL Licensing RiskEgress CostLack of Atomic RenameS3 Compatibility DriftDirectory Namespace / Listing BottlenecksGeo-Replication Conflict / DivergenceRetention Governance FrictionPolicy SprawlCold Retrieval LatencySmall Files AmplificationRequest Pricing ModelsCompression EconomicsData ResidencyCLOUD Act Data AccessChina Data LocalizationRequest AmplificationCross-Region ConsistencyRead / Write AmplificationCache ROIPerformance-per-DollarZero-Egress EconomicsSSE-C Encryption HijackingDatacenter Power ShortfallDatacenter Water ConsumptionGPU StarvationTail Latency on Object Storage
stores_in1Resources 4
DocsHigh
docs.aws.amazon.com/AmazonS3/latest/userguide/Welcome.html
The single authoritative reference for all S3 concepts, features, and operational guidance.
DocsHigh
docs.aws.amazon.com/AmazonS3/latest/API/Welcome.html
The S3 REST API Reference defines every API operation, request/response format, and error code — the ground truth for any S3 integration.
DocsHigh
docs.aws.amazon.com/AmazonS3/latest/userguide/optimizing-per...
AWS's own performance design patterns covering partitioning, request rate scaling, and throughput optimization for production S3 workloads.
DocsHigh
aws.amazon.com/s3/
The S3 product landing page provides pricing tiers, storage class overviews, and feature announcements in a single consolidated view.