Architecture

GPU-Direct Storage Pipeline

An architecture that streams data directly from storage devices to GPU memory, bypassing the CPU and system memory entirely. Uses technologies like NVIDIA GPUDirect Storage (GDS).

3 connections 3 resources

Summary

What it is

An architecture that streams data directly from storage devices to GPU memory, bypassing the CPU and system memory entirely. Uses technologies like NVIDIA GPUDirect Storage (GDS).

Where it fits

GPU-Direct Storage eliminates the CPU bottleneck in AI/ML training data loading. Instead of CPU reading from storage, copying to system memory, then transferring to GPU memory, data flows directly from NVMe/RDMA storage to GPU — increasing training throughput.

Misconceptions / Traps
  • GPU-Direct Storage requires specific hardware support: compatible GPUs, NVMe drives, and RDMA-capable NICs. It does not work with arbitrary storage backends or network configurations.
  • Not all data formats benefit equally. GPU-Direct Storage is most effective with large, sequential reads (training batches). Random small-file access patterns see less improvement.
Key Connections
  • depends_on RDMA (RoCE v2 / InfiniBand) — requires RDMA for direct data path
  • solves Cold Scan Latency — eliminates CPU-mediated data loading latency
  • scoped_to Object Storage for AI Data Pipelines — optimizing GPU training data flow

Definition

What it is

An architecture that streams data directly from NVMe or S3-compatible storage into GPU memory using NVIDIA GPUDirect Storage (GDS), bypassing CPU and system RAM to eliminate data copy overhead.

Why it exists

AI/ML training is GPU-bound, and data loading is often the bottleneck. GPU-Direct Storage removes the CPU from the data path, enabling GPUs to pull training data directly from storage at maximum throughput.

Primary use cases

AI/ML training data loading, high-throughput inference data streaming, GPU-accelerated data processing pipelines.

Connections 3

Outbound 3
scoped_to2
S3Object Storage for AI Data Pipelines
solves1
Cold Scan Latency

Resources 3

DocsHigh
developer.nvidia.com/gpudirect-storage

NVIDIA GPUDirect Storage overview enabling direct data transfer between storage and GPU memory, bypassing the CPU.

DocsHigh
docs.nvidia.com/gpudirect-storage/

GPUDirect Storage documentation covering architecture, API reference, and integration with distributed storage systems.

BlogHigh
developer.nvidia.com/blog/gpudirect-storage/

NVIDIA technical blog explaining GPUDirect Storage architecture and benchmarks for AI/ML training pipelines.