Technology

SeaweedFS

An open-source distributed storage system with an S3-compatible API, architecturally optimized for billions of small and large files with O(1) disk seek for any object lookup.

7 connections3 resources6 posts

Summary

What it is

An open-source distributed storage system with an S3-compatible API, architecturally optimized for billions of small and large files with O(1) disk seek for any object lookup.

Where it fits

SeaweedFS addresses the small files problem at the storage engine level. Its architecture separates metadata (master) from data (volume servers), enabling fast lookup regardless of namespace size — a design well-suited for workloads with billions of small objects.

Misconceptions / Traps
  • Not just another MinIO clone. SeaweedFS has a fundamentally different architecture that separates metadata from data storage, providing fast lookup regardless of namespace size.
  • Community and ecosystem are smaller than MinIO or Ceph. Evaluate support and integration availability for your stack.
Key Connections
  • implements S3 API — S3-compatible interface
  • solves Small Files Problem — O(1) lookup architecture handles billions of small files
  • solves Vendor Lock-In — self-hosted S3-compatible alternative

Definition

What it is

An open-source, distributed storage system with S3 API compatibility, FUSE mount, and optimized handling of both small and large files. Designed for billions of files with fast lookup.

Why it exists

Traditional object stores struggle with billions of small files. SeaweedFS separates file metadata from content storage, enabling O(1) file lookup regardless of namespace size, making it ideal for workloads that S3's flat namespace struggles with.

Primary use cases

Image and media storage at scale, small file-intensive workloads, S3-compatible development environments, on-premise object storage.

Recent developments

Latest signals

  • SeaweedFS Enterprise pricing is public and flat: $2/TB/month. The commercial edition is licensed at $2/TB/month (or $20/TB/year) with no API-access or egress fees, and is free for development and test deployments under 25TB. The vendor's own comparison point is Amazon S3 Standard at roughly $23/TB/month before request and egress charges — positioning Enterprise as the supported path for the same post-MinIO self-hosters the open-source project already serves. Per SeaweedFS Enterprise pricing (seaweedfs.com).

  • Latest release: v4.42 (August 2026) — heartbeat protocol and master hot-path fixes. 4.42 optimizes the volume-heartbeat protocol and removes extensive volume-allocation work from the master's hot paths, resolving stubborn out-of-memory edge cases when digesting very large topologies (a failure mode tracked for deployments past ~2M volumes). Per seaweedfs/seaweedfs releases.

  • "Sealed Directories" shrink the metadata store 18–58×. Cold directory entries are packed into compressed, erasure-coded volume chunks instead of living individually in the filer's metadata store — directly targeting the OOM failures common in massive FUSE mounts. Combined with customizable erasure-coding ratios (a 20+4 layout gives full failure protection at 1.2× storage overhead versus 3× for traditional triple-replication), this is the mechanism behind the project's petabyte-scale cost claims. Per seaweedfs.com.

  • SeaweedFS 4.12 feature-testing — S3 API + WebDAV + filer all in scope. Per simonw/research's SeaweedFS 4.12 feature evaluation (February 13, 2026), the version 4.12 release was independently tested on Linux x86_64 — S3 API fully interoperates with AWS CLI, TTL volumes expire and are efficiently deleted, collections and namespace isolation work as documented, image resizing is in place, and the all-in-one deployment via weed mini simplifies single-node setup. For teams evaluating SeaweedFS against Ceph or Garage, this independent feature-testing dataset is the most current reference point.

  • Post-MinIO positioning crystallizing in the self-hosted S3 landscape. Per the LLMS3 Post-MinIO Landscape guide, SeaweedFS occupies the "billion-files plus broad-ecosystem-integration" niche in the post-MinIO self-hosted S3 fragmentation — distinct from RustFS's MinIO-binary-replacement framing, Ceph's enterprise complexity, and Garage's edge/homelab specialization.

  • Two CVEs disclosed against pre-4.34 releases. CVE-2026-55874 (HIGH): path traversal in the S3 gateway via the X-Amz-Copy-Source header, affecting versions prior to 4.34. CVE-2026-55873 (MEDIUM): improper authorization in the S3Tables / Iceberg REST management API, affecting versions 4.08-4.33. Both predate the current 4.41 line. Per CVE-2026-55874 / CVE-2026-55873 (tertiary source, no corroborating advisory found).

  • Iceberg REST Catalog support added alongside S3/WebDAV/FUSE. SeaweedFS now ships an Iceberg REST Catalog and Iceberg table maintenance features, extending it beyond blob/object/file storage into lakehouse table-format territory. Per seaweedfs/seaweedfs GitHub repository.

  • SeaweedFS's core mechanism is Facebook's 2010 Haystack design: volumes pack many files together so lookup + disk seek is O(1) no matter how large the dataset gets. Each volume server keeps just 16 bytes of metadata per blob entirely in RAM — a billion files needs roughly 16GB of index RAM — so reading any file is one in-memory lookup followed by exactly one disk seek, with no directory traversal or inode table involved. This is the architectural reason SeaweedFS holds up at billions-of-files scale where traditional filesystems degrade, and it's the piece of "why O(1) lookup" this node names as a capability without previously explaining the mechanism. Per SeaweedFS: The Sovereign Storage Deep-Dive.

  • Kubeflow Pipelines switched its default embedded object store from MinIO to SeaweedFS as of KFP 2.15.2. MinIO remains fully supported for teams that want it, but new installs now provision SeaweedFS by default — a concrete, named adoption by a major ML-pipeline project in the post-MinIO-license fragmentation, distinct from the general "billion-files niche" positioning already on this node. Per Kubeflow AI Reference Platform 1.11 Release Announcement. Sources: SeaweedFS 4.12 Feature Testing — simonw/research (February 2026) · The Post-MinIO Landscape (LLMS3) · seaweedfs/seaweedfs GitHub repository

Connections7

Outbound6
scoped_to2
S3Object Storage
implements1
S3 API
solves3
Small Files ProblemDirectory Namespace / Listing BottlenecksVendor Lock-In
Inbound1
integrates_with1
txn2/mcp-s3

Resources3

GitHubHigh
github.com/seaweedfs/seaweedfs

Primary SeaweedFS repository for this distributed storage system supporting S3 API, FUSE mount, and Hadoop compatibility.

DocsHigh
github.com/seaweedfs/seaweedfs/wiki

SeaweedFS documentation wiki covering architecture, deployment, replication, and tiered storage configuration.

DocsHigh
github.com/seaweedfs/seaweedfs/wiki/Amazon-S3-API

SeaweedFS S3 API compatibility reference detailing supported S3 operations and configuration.

Featured in

2026-08-24The Storage Tier Is the New Context WindowA vendor published the break-even arithmetic for persisting LLM inference state to object storage: one cache reuse every 94 hours justifies the spend. Between that number, NVMe object tiers priced like commodities, and streaming engines that write payloads straight to S3, the quiet reclassification of object storage — from cold archive to active memory tier — now has receipts.2026-06-16The Whole Stack Went Open — Weights, Storage, and SovereigntyIn the same quarter that a 1.6-trillion-parameter open-weights model landed on top of last year's closed frontier — and Anthropic's Claude Fable 5 promptly sprinted ahead again — the storage layer underneath it went open too: DeepSeek open-sourced its file system, Europe stood up sovereign S3, and the post-MinIO self-hosted stack matured. The frontier and the floor are pulling apart, and the pattern underneath all of it is the oldest pain point this index maps: vendor lock-in.2026-05-08When the Index Gained 13 Nodes: A Field Guide to the May 2026 WaveOn May 7 the index added 13 new nodes — five technologies, six pain points, one architecture, one standard. Each answers a specific question engineers building on object storage are asking right now: what is China's S3 stack, why are GPUs starving, what replaced MinIO, what are the three data gravity wells, and a few more. This post walks through them by question rather than by node type, because that's how the questions actually arrive in production.2026-05-07When DeepSeek Made Storage Strategic: The China Direction Reshaping the S3 EcosystemDeepSeek's $5.6M training run didn't just reset model architecture. It made object storage performance-critical, made silicon export controls a storage decision, and pushed Aliyun OSS, Tencent COS, and Huawei OBS from regional curiosities into the global S3 vocabulary. This index added 13 nodes to track what changed.2026-05-01What Survived MinIO: The Post-Archive S3 Server Landscape, May 2026On April 25, 2026, MinIO archived its main GitHub repository. This index has tracked Vendor Lock-In as a first-class pain point since launch. Here is what the post-MinIO Apache-2.0 server set actually looks like — Alarik, RustFS, SeaweedFS, Garage — and what the fragmentation tells us about what we believed open-source meant.2026-03-21The Shift to Local Intelligence: How Local S3 and Edge Inference Are Replacing Cloud LatencyThe economics and architecture behind the migration from cloud inference to local-first AI — covering the Local S3 semantic storage pattern, CXL memory expansion, specialized inference silicon, and why 80% of AI inference is moving to the edge.