Retrieval Engineering
The discipline of building production retrieval systems that go beyond basic Retrieval-Augmented Generation (RAG) — orchestrating hybrid retrieval (vector + BM25 + graph), maintaining retrieval freshness against changing object stores, synchronizing embeddings, and operating directly against lakehouse formats rather than copying data into proprietary vector databases.
Definition
The discipline of building production retrieval systems that go beyond basic Retrieval-Augmented Generation (RAG) — orchestrating hybrid retrieval (vector + BM25 + graph), maintaining retrieval freshness against changing object stores, synchronizing embeddings, and operating directly against lakehouse formats rather than copying data into proprietary vector databases.
Monolithic vector search fails on precise keyword queries (SKUs, error codes, proper nouns) and on retrieval freshness. Production pipelines now fuse semantic similarity, lexical search (BM25), and deterministic graph queries — and they execute these against multimodal lakehouse formats (Lance, Iceberg) on S3 rather than migrating data into separate engines. Retrieval Engineering names this maturation from "vector DB" to "retrieval pipeline."
Hybrid retrieval over S3-backed embeddings, multimodal retrieval across text/image/video on Lance format, retrieval freshness via embedding-sync pipelines (FreshnessProbe pattern), retrieval observability with LangSmith / Arize Phoenix / Ragas / DeepEval, graph traversal on temporal knowledge graphs.
Connections 4
Outbound 3
scoped_to2Inbound 1
scoped_to1