This index reports what the industry did — the solves / alternative_to / competes_with edges are observable facts. But we also make editorial calls about where things are heading, and those deserve a different standard: dated, falsifiable, and publicly scored. This page is that ledger. Every call states what would change our mind. When we're wrong, it stays here with the correction.
Writing registers used across this site:
- Observed — supported by relationship edges and primary sources; stated plainly.
- Reported — a claim from a named source we haven't independently verified; attributed inline.
- Our read — our synthesis; always dated, always with revision conditions. Everything below is this register.
Open calls
2026-07-15 — The general-purpose open Iceberg catalog consolidates on Apache Polaris
The call: Polaris becomes the default open catalog choice; Project Nessie's Git-for-data semantics get absorbed (into Polaris or the Iceberg REST spec) rather than winning as a standalone catalog.
Signals: ASF Top-Level graduation (Feb 2026), monthly release train through 1.6.0, Cloudera adoption + contributed Ranger plugin, CVE cluster handled with fast disclosure/patching; Nessie lacks credential vending and RBAC, and two independent practitioner assessments advise against greenfield use.
We revise if: Nessie ships native credential vending/RBAC; significant new greenfield Nessie deployments surface; a third catalog (Lakekeeper, Gravitino) takes the general-purpose slot instead.
2026-07-15 — Kafka becomes optional in single-consumer lakehouse CDC
The call: For pipelines whose only consumer is the lakehouse, direct CDC→Iceberg (Flink CDC, Supermetal-class single-process tools, streaming SQL databases) displaces Debezium→Kafka→processor as the default architecture for new builds by end of 2027. Where the change stream genuinely fans out to many consumers, Kafka stays.
Signals: Normalized benchmarks showing 7× snapshot gaps untuned; Flink CDC's own no-Kafka positioning; the "Kafka-less" tooling wave (RisingWave, BladePipe, Estuary) documented across independent sources.
We revise if: new-deployment surveys/case studies through 2027 show Kafka-based CDC holding share for lakehouse-only pipelines; the single-process tools stall pre-GA or accumulate correctness incidents.
2026-07-15 — The two-engine (Spark + Flink) pattern erodes where feature parity matters
The call: Spark Real-Time Mode erodes Flink's share of workloads whose pain is training/serving logic drift — not Flink's latency crown. Consolidation is a 2027 question; the burden of proof has flipped, adoption hasn't yet.
Signals: RTM GA (March 2026) with sub-100ms production references (Coinbase, DraftKings); the architectural mechanism (long epochs, concurrent stages, non-blocking operators) is published, not just benchmarked.
We revise if: RTM production regressions surface at scale; Flink's response closes the single-engine gap; the launch references stay the only references by mid-2027.
2026-07-15 — Vector search dissolves into the lakehouse; the standalone RAM-first vector DB niche narrows
The call: The default home for vectors becomes lake-resident formats and object-storage-native engines (External Collections, S3 Vectors, Turbopuffer-class, Lance/Vortex); standalone RAM-first vector databases narrow to latency-critical serving tiers.
Signals: Milvus 3.0's zero-copy lake queries and Zilliz's Lakebase pivot; every 2026 format release (Paimon 1.4, Lance, Vortex) adding vector primitives; S3 Vectors' price cuts; Pinecone responding with lakehouse integration (OneLake) and a price-floor tier.
We revise if: lake-resident query latency stalls above serving requirements; the RAM-first vendors' growth re-accelerates on workloads that were supposed to dissolve.
Explicitly NOT called
The post-MinIO storage race
We have no winner call. RustFS leads mindshare and our own traffic, but it is pre-GA (1.0.0-beta series), independent benchmarks show MinIO still ~3× faster on reads, and clustering is the stated GA blocker. The "RustFS won" narrative circulating in 2026 runs ahead of the software. We index the race; we don't score it yet.
Corrections
Corrections are part of the record. Silent fixes don't build authority; visible ones do.
- 2026-07-15 — RustFS benchmark framing. We carried a vendor-side "~2.3× faster than MinIO" claim for two content waves. Independent benchmarks (rustfs#2154; a Milvus community evaluation) show the advantage is write-path only — MinIO leads pure reads by ~3× with ~10× better TTFB. The node now carries the full picture and an operator verdict.
- 2026-07-15 — Apache Paimon vector index. We briefly paired Paimon 1.4's Lumina vector index with "DiskANN" based on a secondary summary. Primary documentation doesn't support the pairing; the claim was removed the same day.
- 2026-07-15 — Duplicate Vortex node. The July wave briefly created a second Vortex entry alongside an existing richer one; merged same-day, count corrected 417→416.