Architecture

A unified architecture combining data lake storage (files on S3) with warehouse capabilities (ACID, schema enforcement, SQL access) by using a table f...

A layered data quality pattern — Bronze (raw), Silver (cleansed), Gold (business-ready) — with each layer stored on object storage.

The design pattern of keeping data in S3 while running independent, elastically scaled compute engines against it.

A pattern that stores raw data on S3 and maintains a vector index over embeddings that points back to S3 objects.

A batch pattern where embeddings are generated from S3-stored data on a schedule, with resulting vectors written back to object storage or a vector in...

A pattern of running ML/LLM models on local hardware against data stored in or pulled from S3, avoiding cloud-based inference APIs.

A data quality pattern where data lands in a raw S3 zone, undergoes validation, and is promoted to a curated zone only after passing audits.

Moving data between hot, warm, and cold storage tiers based on access frequency. S3 itself offers tiering (Standard, Infrequent Access, Glacier).