Technology
Concrete tools, systems, or platforms with version histories and maintainers.
193 nodesAn open-source Go MCP server that exposes S3 (and any S3-compatible store) as governed tools to AI agents — browse buckets, read/w…
MinIO's native MCP integration connecting LLM clients directly to AIStor clusters for cluster admin, object analysis, and policy m…
An MCP server that lets agents discover, query, and reason over managed Apache Iceberg tables and S3 Metadata inventory tables in …
A scalable persistence architecture for LLM reinforcement-learning workloads in the verl framework — hot data in RAM, cold in loca…
A flash-native context-memory store embedded in the AI storage tier, exposing petabytes of NVMe to GPU pods as shared KV cache ove…
Amazon's fully managed object storage service — the origin and reference implementation of the S3 API. As of December 2025, the ma…
An open-source, S3-compatible object storage server designed for high performance and self-hosted deployment. As of February 2026,…
A community-maintained AGPL v3 fork of MinIO created after the upstream repository was archived in February 2026 and permanently r…
An open-source (Apache 2.0) S3-compatible gateway that translates S3 API calls into POSIX filesystem operations. A thin translatio…
A distributed storage system providing object, block, and file storage in a unified platform. S3 compatibility via its RADOS Gatew…
A scalable, distributed object storage system in the Hadoop ecosystem with an S3-compatible interface.
An open table format for large analytic datasets. Manages metadata, snapshots, and schema evolution for collections of data files …
An open table format and storage layer providing ACID transactions, scalable metadata, and schema enforcement on data stored in ob…
A table format and data management framework optimized for incremental data processing — upserts, deletes, and change data capture…
A lakehouse metadata format that stores table metadata in an embedded SQL database (DuckDB) instead of file-based manifests on S3.…
An in-process analytical database engine (like SQLite for analytics) that reads Parquet, Iceberg, and other formats directly from …
A PostgreSQL extension streaming heap tables into DuckLake columnar tables on S3 via WAL-based CDC — the transactional-to-analytic…
A federated AI/data runtime that combines embedded DuckDB compute with native delegation to Amazon S3 Vectors for similarity searc…
A distributed SQL query engine for federated analytics across heterogeneous data sources, with deep support for S3-backed data lak…
A column-oriented DBMS designed for real-time analytical queries, with native support for reading from and writing to S3.
A distributed compute engine for large-scale data processing — batch ETL, streaming, SQL, and machine learning — over S3-stored da…
A vector database that stores data in the Lance columnar format directly on object storage. Designed for serverless vector search …
An open-source vector database with hybrid search combining BM25 keyword matching and vector similarity in a single query, plus mu…
A Rust-based vector search engine with native payload filtering and a custom HNSW index implementation that applies metadata filte…
A commercial vector database launched by Actian in April 2026, multi-cloud (AWS/Azure/GCP), built on FAISS + OnDiskIVF indices wit…
A distributed vector database built for billion-scale similarity search, using a microservices architecture with SSD caching for h…
The de facto open-source PostgreSQL extension for vector similarity search. Adds a `vector` data type plus indexed nearest-neighbo…
A high-performance PostgreSQL extension for vector similarity search, positioned as a **drop-in replacement for pgvector** with or…
An open-source distributed search + analytics engine forked from Elasticsearch in 2021, now governed by the **OpenSearch Software …
An MPP analytical database with native lakehouse capabilities, able to directly query S3 data in Parquet, ORC, and Iceberg formats…
A distributed stream processing framework that processes data in real-time, with S3 as checkpoint store, state backend, and output…
An AWS S3 storage class delivering single-digit millisecond latency for frequently accessed data, using Directory Buckets in a sin…
An AWS-managed feature providing native Apache Iceberg tables as a built-in S3 capability with automated Binpack / Sort / Auto com…
Native vector storage and similarity search built into S3, operating under a dedicated `s3vectors` AWS service namespace with its …
An AWS feature that automatically generates queryable metadata tables (in Apache Iceberg format) over S3 objects, enabling SQL-bas…
An AWS feature (launched June 2026) letting users attach custom, structured metadata directly to individual S3 objects at scale — …
AWS's serverless compute service — pay-per-invocation function execution with managed runtime, no server provisioning. **Now mount…
A POSIX file-system interface over general-purpose S3 buckets, launched April 7, 2026. Any bucket can be mounted as an NFS v4.1 or…
An open-source distributed storage system with an S3-compatible API, architecturally optimized for billions of small and large fil…
An S3-compatible object storage service from Cloudflare with zero egress fees, integrated with the Cloudflare global edge network.
A low-cost S3-compatible cloud storage service with free egress to CDN partners through the Bandwidth Alliance, designed for cost-…
An S3-compatible cloud storage service with a fixed pricing model — no egress fees, no API request fees, approximately $5–7/TB/mon…
Wasabi Technologies' AI-augmented object storage tier — facial recognition, speech-to-text, OCR, and logo detection run inline as …
A budget-tier S3-compatible cloud object storage from IDrive (the established backup vendor), priced at **~$5/TB/month** with **ze…
A Sweden-headquartered S3-compatible object storage provider, **launched May 2026**, priced at **€5/TB/month** with zero egress fe…
**OVHcloud's** S3-compatible object storage service from France's largest cloud provider. Three storage classes — **Standard (~$5/…
Alibaba Cloud's S3-compatible Object Storage Service — the dominant object store across mainland China. Standard bucket/key data m…
Tencent Cloud's Cloud Object Storage — S3-compatible, the storage backbone for Tencent's gaming, video, fintech, and Hunyuan AI tr…
Huawei Cloud's Object Storage Service — S3-compatible, tightly co-engineered with Huawei's domestic AI accelerator (Ascend 910B/91…
Google Cloud Storage (GCS) is Google's fully-managed object storage service — buckets with per-object storage classes (Standard, N…
A disaggregated all-flash data platform providing unified access via S3, NFS, and SMB protocols, optimized for AI and deep learnin…
WEKA is an AI-native, software-defined parallel storage platform (WekaFS) that presents a single namespace across NVMe flash with …
DDN's software-defined S3-compatible object storage for AI, sold alongside the AI400X3 appliance line — holder of the strongest ML…
An enterprise-grade software-defined object storage platform from Dell with S3-compatible API, designed for on-premise and hybrid …
Dell's Kubernetes-native S3-compatible object storage platform — a container-architecture rebuild of the Dell ECS codebase — and, …
A software-defined S3-compatible object storage system with policy-driven information lifecycle management (ILM), designed for ent…
An all-flash unified file and object storage platform from Pure Storage with S3-compatible API, designed for AI, analytics, and mo…
Enterprise-grade software-defined object storage from Hitachi, S3-compatible, with native Iceberg-aware S3 Tables functionality an…
Hewlett Packard Enterprise's enterprise scale-out object storage platform, S3-compatible, with native data-intelligence services b…
A lightweight, self-hosted, geo-distributed S3-compatible object storage system designed for small distributed clusters, edge depl…
An open-source distributed data caching and orchestration layer between S3-compatible object storage and compute (Spark, Trino, Py…
**Fire-Flyer File System** — DeepSeek's high-performance distributed file system purpose-built for AI training and inference, **op…
A unified data access layer providing a single API for accessing 40+ storage backends including S3, GCS, Azure Blob, HDFS, and loc…
A Git-like version control system for data lakes on S3, providing branching, committing, merging, and rollback for datasets stored…
A Kubernetes storage orchestrator that deploys and manages Ceph clusters on Kubernetes, providing K8s-native S3-compatible object …
A high-performance FUSE-based filesystem that provides POSIX-compatible access to S3-compatible object storage, optimized for AI/M…
A POSIX-compliant distributed filesystem that uses S3-compatible object storage as its data backend and a separate metadata engine…
An open-source REST catalog for Apache Iceberg with centralized RBAC, originally developed by Snowflake and donated to Apache.
A unified metadata lake — "catalog of catalogs" — that federates Iceberg, Hive, Kafka, and file-based data sources into a single g…
An open-source, multi-format data catalog by Databricks (Linux Foundation), supporting Iceberg, Delta Lake, Hudi, and unstructured…
Lakekeeper is an open-source (Apache-2.0), **Rust-native Apache Iceberg REST Catalog**. Where the incumbent catalogs grew out of J…
A zero-copy metadata translator (Apache incubating, formerly OneTable) that converts between Iceberg, Delta Lake, and Hudi metadat…
A Delta Lake feature that automatically generates Iceberg and Hudi metadata for Delta tables, enabling cross-format reads without …
An Apache top-level streaming lakehouse table format built on LSM-tree architecture, designed for high-frequency real-time writes …
Apache Flink connectors for reading database change logs (MySQL binlog, PostgreSQL WAL) and streaming them directly into lakehouse…
A managed real-time data integration platform with exactly-once connectors for streaming data from databases and SaaS APIs into S3…
A Python-native stream processing framework built on a Rust-based Timely Dataflow engine, designed for real-time data transformati…
A platform for programmatically authoring, scheduling, and monitoring workflows as directed acyclic graphs (DAGs) written in Pytho…
A high-performance, S3-compatible object storage server written in Swift on SwiftNIO, distributed under Apache 2.0. Uses ARC (Auto…
A high-performance, Rust-based, S3-compatible object storage server positioned as a truly open-source alternative to MinIO.
A CLI-first, plan-then-execute object storage migration and sync tool purpose-built for large-scale S3-to-S3 transfers — marketed …
The reference implementation for OpenLineage — an open-source metadata and lineage service with a web UI for visualizing data flow…
A framework for fine-grained security and centralized auditing across the Hadoop and lakehouse ecosystem, providing column-level a…
An S3 feature that reduces KMS API calls by up to 99% by caching encryption key material at the bucket level rather than making in…
A stateless, S3-native data streaming platform with Kafka protocol compatibility. No local disks, no brokers to manage — all data …
A real-time analytical database with native lakehouse capabilities, querying Iceberg, Hudi, and Paimon tables on S3 directly. Late…
An enterprise storage platform with S3-compatible object storage, delivering hardware-defined performance guarantees at petabyte s…
A purpose-built, hardware-defined storage appliance providing S3-compatible object storage on Ceph with auditable supply-chain man…
AWS's fully managed metadata catalog service that stores table definitions, partition information, and schema metadata for data st…
The original metadata catalog service from the Apache Hive project that stores table schemas, partition mappings, and storage loca…
A lakehouse query engine that provides SQL analytics directly on S3-stored data with integrated Iceberg table management, data ref…
A unified data + AI platform built on Apache Spark and Delta Lake, with a managed lakehouse covering data engineering, SQL analyti…
AWS's serverless, pay-per-query SQL engine that runs queries directly against data stored in S3 without requiring infrastructure p…
An open-source distributed platform for change data capture (CDC) that streams row-level changes from databases (PostgreSQL, MySQL…
An extensible query execution framework written in Rust, built on Apache Arrow, that provides a SQL query planner and execution en…
A high-performance DataFrame library written in Rust with Python and Node.js bindings, designed for fast columnar analytics with l…
An Apache Kafka feature (KIP-405) that offloads older log segments from broker-local disks to S3-compatible object storage, extend…
A Kafka-compatible streaming platform written in C++ that provides a single binary deployment with built-in Tiered Storage to S3, …
An open-source transactional catalog for data lakes that provides Git-like branching, tagging, and commit semantics for Iceberg ta…
An open-source data integration platform that provides pre-built connectors for extracting data from hundreds of sources (APIs, da…
Apache Spark's stream processing API that enables continuous, micro-batch, or near-real-time ingestion of data streams into S3-bac…
A C++ vectorized execution engine developed by Meta that provides a unified, high-performance data processing backend usable by mu…
A Python library for declarative data loading (data load tool) that simplifies building data pipelines to extract from APIs and lo…
An open-source metadata platform providing a centralized catalog for data discovery, quality, lineage, and governance across S3-ba…
An open-source metadata platform originally developed at LinkedIn that provides data discovery, lineage tracking, governance, and …
An open-source metadata management and governance framework originally built for the Hadoop ecosystem, providing classification, l…
A command-line program that synchronizes files and directories to and from cloud storage, supporting **70+ backends** through a si…
A multimodal vector store (MVS) testing and benchmarking platform that evaluates S3-compatible providers for AI/ML workloads — fee…
An S3-compatible, globally distributed object storage platform engineered to optimize small-object workloads through metadata inli…
NVIDIA's client/server library stack released November 2025 that moves S3-compatible object data directly from storage-node memory…
A WireGuard-based secure mesh-networking platform. In April 2026, Tailscale added an S3-compatible export for log and telemetry da…
An open-source universal memory layer for AI agents, distributed under Apache 2.0. Provides persistent semantic memory backed by S…
An open-source AI memory platform (Apache 2.0) built around the **Graphiti** temporal-knowledge-graph engine. Zep stores semantic …
The open-source temporal knowledge-graph engine that powers Zep. Real-time knowledge-graph construction for AI agents — stores ent…
A high-performance distributed **KV-cache offloading** layer for LLM inference, written to maximize prefix-reuse across vLLM and o…
An open-source LLM serving engine optimized for structured generation and prefix sharing. Distributed under Apache 2.0. The **Radi…
The open-source LLM serving platform for **Kimi**, Moonshot AI's leading LLM product. Repository: [github.com/kvcache-ai/Mooncake]…
A cognitive-memory system for AI agents, distributed as a single ~22MB Rust binary that doubles as an **MCP server** for Claude, C…
An open-source agent-runtime framework built on top of LangChain that models agentic workflows as **state machines** — supervisor/…
An open-source **model gateway** that abstracts the complexity of calling hundreds of different LLM endpoints behind a unified, Op…
An open-source **AI gateway** (MIT-licensed) sitting between the agent runtime and foundation models. Provides observability (per-…
**Traefik Labs**'s commercial AI gateway, layered on the Traefik reverse proxy heritage. In December 2025, Traefik joined the **HP…
NVIDIA's fourth-generation **Data Processing Unit (DPU)**, announced in 2026 as the substrate for a new class of **AI-native stora…
A new storage tier — also referred to as **Context Memory eXtension (CMX)** — sitting between traditional NVMe SSDs and cold S3 bu…
NVIDIA's library coordinating the highly orchestrated data movement between storage tiers, GPUs, and inference engines. NIXL provi…
A commercial **memory orchestration** platform for AI workloads, providing software-defined coordination of CXL-attached memory po…
NVIDIA's CUDA library extending **GPUDirect Storage (GDS)** semantics to S3-compatible object storage. Where the original GDS targ…
NVIDIA's InfiniBand XDR platform — Q3400-RA switch plus ConnectX-8 SuperNICs — doubling per-port bandwidth to 800 Gbps with 144 po…
Fast secure backup tool with S3 support.
Database AI lakehouse with in-DB vector retrieval, graph compute, and inference
Aliyun POSIX cache over OSS for AI training — admission of object storage limitations
1.82x TTFT speedup at 200K context.
Predict N+1, N+2 tokens simultaneously for denser gradients.
Read-through cache for S3-compatible storage (Rust, hybrid memory+disk)
Fast single-node S3-compatible storage in C++ (LMDB-based, MinIO replacement)
Official AWS command-line interface.
Official Python SDK for AWS.
CLI tool for S3 bucket and file management.
GQA → MLA migration without retraining from scratch.
Distributed KV + S3-compatible object store in Rust (Raft, multi-tenant)
Embedded OLAP SQL engine powered by ClickHouse. In-process analytical database for Python, Go, Rust, Node.js. Queries Parquet, Arr…
Open-source local-LLM runtime that lets developers run hundreds of language models — including DeepSeek-R1, Llama 3.1, Gemma 4, Qw…
Managed serverless vector database with a storage-compute separation architecture built directly on Amazon S3 (and equivalent obje…
Open-source AI-native search infrastructure with a client-server architecture and pluggable storage backends. In embedded mode run…
Globally-distributed S3-compatible object storage service that automatically replicates objects close to the regions writing them …
Decentralized S3-compatible object storage built on a network of 30,000+ independent storage nodes worldwide. Files are encrypted …
Enterprise-grade scale-out object + file storage software from Scality, built around the RING distributed architecture. Supports f…
Fully managed S3-compatible object storage from CoreWeave, purpose-built for AI workloads (training datasets, model weights, check…
Geo-distributed, multi-tenant S3-compatible object storage built around a "Swarm" architecture — encrypted shards distributed acro…
S3-compatible object storage from German hosting provider Hetzner, served from EU data centers in Falkenstein, Nuremberg, and Hels…
S3-compatible object storage from Akamai's developer cloud (formerly Linode, acquired by Akamai 2022). Globally distributed across…
S3-compatible cloud object storage from Yandex Cloud. Replicates data across multiple availability zones with a **99.98% SLA**, su…
GPU-first AI cloud platform with S3-compatible object storage as one tier of a unified AI infrastructure stack (GPU droplets, mana…
Full-stack AI cloud platform launched by DigitalOcean at **Deploy 2026** (April 2026), explicitly built end-to-end for the inferen…
Initial community fork of [MinIO](/node/minio) created in May 2025 to restore the management UI and admin features that MinIO Inc.…
SAP HANA Cloud's data-lake tier — extends the in-memory HANA database with **virtual tables that provide read-only access to Apach…
Open-source unified data-collection agent for the **Guance Cloud** observability platform. Supports Linux / Windows / macOS hosts …
A bucket-level Amazon S3 feature that preserves every version of every object — every PUT or DELETE creates a new version rather t…
Amazon S3 feature for automatically replicating objects + metadata + tags from a source bucket to one or more destination buckets …
Write-once-read-many (WORM) feature for Amazon S3 buckets — once configured, an object version cannot be deleted or overwritten fo…
Family of three S3 cold-storage tiers, all under the `S3 Glacier` brand but with structurally different retrieval-latency profiles…
Open-source FUSE-style file client from AWS that mounts an S3 bucket as a local POSIX filesystem on a compute instance. Built on t…
AWS-managed, serverless **Apache Cassandra–compatible** wide-column NoSQL database service. Zero infrastructure management, pay-pe…
An open-source LLM serving engine originally developed at UC Berkeley (Sky Computing Lab) that introduced **PagedAttention** — a p…
NVIDIA's optimized LLM inference framework built on TensorRT, providing hand-tuned CUDA kernels, in-flight batching, paged KV-cach…
A Gemma-4-specific architectural feature — exposed in HuggingFace `transformers` as the `num_kv_shared_layers` config field — that…
A hybrid kernel formulation for DeepSeek-style Multi-head Latent Attention (MLA) introduced in 2026 that interleaves the *naive* (…
An FP8-native quantization scheme for MLA latent KV-cache, introduced 2026, that quantizes the *latent* tensor (the compressed sha…
A streaming KV-cache compression and transmission system from researchers at the University of Chicago that treats the KV-cache as…
A privacy-first, locally-hosted persistent-memory server (developed by mem0ai under the CaviraOSS open-source line) that speaks th…
An open-source **durable runtime** for AI agents from ZenML, designed as the "outer harness" that sits underneath any agent SDK (P…
An open-source **OS-style memory management framework** for LLM agents (formerly **MemGPT**), built on the analogy that the LLM's …
An open-source **persistent agent-memory framework** that builds a hybrid graph-plus-vector memory layer for LLM agents. Cognee in…
A managed-SaaS **memory layer for LLM applications** focused on developer ergonomics — a few-line SDK that ingests user/conversati…
AWS's managed **stateful agent runtime** for the Bedrock platform, providing isolated **microVMs** (Firecracker-style lightweight …
On-prem, S3-compatible, exabyte-scale object storage whose 8.2.6 release is NVIDIA-Certified and supports S3 over RDMA for direct …
A managed Apache Pinot platform that serves sub-second, high-concurrency analytics directly on Apache Iceberg and Parquet tables i…
A UK-operated (RCS Technologies) S3-compatible object storage service with flat per-GB pricing, no API-request fees, and no inboun…
An object-storage-native vector and full-text search engine where S3/GCS is the durable source of truth and SSD/RAM are caches, bu…
A dedicated, standalone catalog engine for large data systems that replaces general-purpose stores and table-format manifest trees…
The single, tenant-wide data lake under Microsoft Fabric, built on ADLS Gen2, storing tables as Delta by default and exposing them…
Cloudflare's managed AI-agent persistent-memory service (private beta, June 2026) — extraction/retrieval architecture behind an op…
Open-source (Apache 2.0), self-hostable agent memory framework launched June 2026 — multimodal ingestion, self-evolving memory, ag…
Agent-first storage platform exposed as an MCP server (June 2026): 19 consolidated tools, 50 GB free persistent storage per agent,…
A Rust-based CDC platform that moves data from a database WAL to a committed Iceberg table on S3 in one process — WAL → Arrow → Pa…
A PostgreSQL-compatible streaming SQL database — continuously-maintained materialized views over streaming/CDC sources, served fro…
Apache-2.0 YAML-driven LLM fine-tuning framework (v0.18.0, July 2026) covering LoRA/QLoRA, full fine-tuning, QAT, preference tunin…
Hugging Face's Apache-2.0 post-training library — SFT, DPO, GRPO, KTO, and reward-model trainers on top of Transformers; v1.9.2 as…
A WebUI + CLI fine-tuning framework (Apache-2.0, ~73.7k stars) covering 100+ LLMs/VLMs with full, LoRA/QLoRA, and preference-optim…
PyTorch's native LLM post-training library (BSD-3-Clause) — readable, modifiable recipes for full/LoRA/QLoRA SFT, DPO, PPO, GRPO, …
Memory-efficient fine-tuning/RL library (custom Triton kernels; core Apache-2.0, pip 2026.8.1) with a published consumer-GPU floor…