Topic

Object Storage

The storage paradigm of flat-namespace, HTTP-accessible binary objects with metadata. Data is addressed by bucket and key, not by filesystem path.

120 connections 3 resources 2 posts

Summary

What it is

The storage paradigm of flat-namespace, HTTP-accessible binary objects with metadata. Data is addressed by bucket and key, not by filesystem path.

Where it fits

Object storage is the foundational layer beneath everything in this index. S3 is the dominant API; all technologies, table formats, and architectures in the map operate on top of object storage.

Misconceptions / Traps
  • Object storage has no native directory hierarchy. Prefixes simulate folders but LIST operations scan linearly — not like ls on a filesystem.
  • Durability (11 9s) is not the same as availability or performance. Data is safe but access can be slow or throttled.
Key Connections
  • scoped_to S3 — S3 is the dominant object storage API
  • Lakehouse scoped_to Object Storage — lakehouses are built on object storage
  • AWS S3, MinIO, Ceph, Apache Ozone scoped_to Object Storage — concrete implementations
  • Separation of Storage and Compute scoped_to Object Storage — the pattern that decouples compute from data

Definition

What it is

The storage paradigm of flat-namespace, HTTP-accessible binary objects with metadata. Data is addressed by bucket and key, not by filesystem path.

Why it exists

Traditional filesystems and block storage do not scale to billions of objects across distributed infrastructure. Object storage trades POSIX semantics for horizontal scalability, durability, and HTTP accessibility.

Recent developments

Latest signals
  • Hitachi Vantara named a Leader in the 2026 GigaOm Radar for Object Storage. Per Hitachi Vantara's GigaOm Radar announcement, Hitachi was recognized for AI-optimized data paths that reduce read/write bottlenecks for ML workloads. The 2026 GigaOm Radar field includes Hitachi, Pure Storage FlashBlade, NetApp StorageGRID, Cloudian HyperStore, and VAST Data — the enterprise on-prem object storage segment has hardened into a 4-5 vendor field with each anchored to AI-data-pipeline workloads.
  • AI-supercomputer-grade object storage entering the conversation. Per IBM's 2026 Storage Scale roadmap, IBM's "Blue Vela" purpose-built on-prem supercomputer for AI model training uses IBM Scale System 6000 as its object-storage substrate — running continuous training cycles for IBM's Granite LLM family. For organizations sizing internal training infrastructure, the IBM reference shape is now a real comparable to the NVIDIA + VAST and the Meta + 3FS architectures.
  • S3-compatible provider field continues to expand. Per Mixpeek's 2026 S3-compatible providers roundup, the credible S3-compat field now spans MinIO (self-hosted, archived), Storj (decentralized at $4/TB/mo), Vultr Object Storage ($5/250GB, 17 global locations), and the existing hyperscaler + EU-sovereign field. The expansion is driven by post-MinIO migration urgency on the self-hosted side and continued egress-cost pressure on the cloud side.

Connections 120

Outbound 1
scoped_to1
S3
Inbound 119click to expand
scoped_to118
Topic17
LakehouseVector Indexing on Object StorageMetadata ManagementData VersioningDirectory Buckets / Hot Object StorageObject Storage for AI Data PipelinesKubernetes Object Provisioning & PolicyMetadata-First Object StorageGeo / Edge Object StorageSovereign StorageAI Memory InfrastructureRetrieval EngineeringInference LocalityAI Runtime InfrastructureAI Memory GovernanceGPU + Object Storage ConvergenceDistributed Context Systems
Technology42
AWS S3MinIOpgsty/minio ForkVersity S3 GatewayCephApache OzoneS3 Express One ZoneAmazon S3 FilesSeaweedFSCloudflare R2Backblaze B2WasabiWasabi AiRIDrive e2HexabyteOVHcloud Object StorageAliyun OSSTencent COSHuawei OBSVAST DataDell ECSNetApp StorageGRIDPure Storage FlashBladeHitachi VantaraHPE Alletra Storage MP X10000GarageAlluxioOpenDALRookGeeseFSJuiceFSAlarikRustFSS3 Bucket KeyWarpStreamInfinidatSoftIronKafka Tiered StorageRedpandarcloneTigris DataNVIDIA GPUDirect RDMA for S3
Standard8
Container Object Storage Interface (COSI)NVMe-oF / NVMe over TCPNFS v4.1RDMA (RoCE v2 / InfiniBand)Zoned Namespace (ZNS) SSDObject Lock / WORM SemanticsCRDTS3 Directory Bucket
Architecture20
Lakehouse ArchitectureSeparation of Storage and ComputeTiered StorageGeo-Dispersed Erasure CodingNVMe-backed Object TierRDMA-Accelerated Object AccessCache-Fronted Object StorageMulti-Site ReplicationActive-Active Multi-Site Object ReplicationEdge-to-Core Object AggregationImmutable Backup Repository on Object StorageRansomware-Resilient Object Backup ArchitectureEncryption / KMSAudit TrailsEast Data West ComputingBenchmarking MethodologyCapacity PlanningObject Lifecycle ManagementMultimodal Object StorageReal-Time AI Lakehouse
Pain Point31
Small Files ProblemCold Scan LatencyData Loading BottleneckObject Listing PerformanceVendor Lock-InAGPL Licensing RiskEgress CostS3 Consistency Model VarianceS3 Compatibility DriftDirectory Namespace / Listing BottlenecksRebuild Window RiskRepair Bandwidth SaturationGeo-Replication Conflict / DivergenceRetention Governance FrictionPolicy SprawlCold Retrieval LatencySmall Files AmplificationRequest Pricing ModelsCompression EconomicsData ResidencyCLOUD Act Data AccessChina Data LocalizationRequest AmplificationCross-Region ConsistencyRead / Write AmplificationCache ROIPerformance-per-DollarZero-Egress EconomicsDatacenter Power ShortfallDatacenter Water ConsumptionTail Latency on Object Storage
is_a1
Data Lake

Resources 3

DocsHigh
aws.amazon.com/what-is/object-storage/

AWS's conceptual explainer defines object storage versus block and file storage, covering the fundamental paradigm shift that underpins S3.

DocsHigh
www.redhat.com/en/topics/data-storage/what-is-object-storage

Red Hat's vendor-neutral overview explains object storage architecture, metadata handling, and use cases from an infrastructure perspective.

DocsHigh
min.io/product/overview

MinIO is the most widely deployed S3-compatible open-source object store; its architecture docs illuminate how the S3 API is implemented outside AWS.

Featured in

2026-05-16 When AI Memory Became an Architecture: KV-Cache Persistence, MCP, and the Night S3 Got Its Memory Tier Stateful agents needed stateful storage. Tonight 33 new nodes joined the index: Mem0, Zep, LMCache, SGLang, Mooncake, MCP, cuObject, BlueField-4, Animesis CMA. The architectural thesis: AI memory infrastructure crystallized around object storage in 2026, and the persistence layer the industry settled on is S3. 2026-03-21 The Shift to Local Intelligence: How Local S3 and Edge Inference Are Replacing Cloud Latency The economics and architecture behind the migration from cloud inference to local-first AI — covering the Local S3 semantic storage pattern, CXL memory expansion, specialized inference silicon, and why 80% of AI inference is moving to the edge.