When the Paper Trail Stopped Matching the Software

Every claim on this index traces to a source, and every source is a bet that whoever wrote it checked their own work. Most of the time that bet pays off. This past week it didn't, three separate times, in three different ways — and the fourth thing we found wasn't a bad claim at all. It was two real bugs sitting in the issue tracker of a table format that 58% of organizations now use for business-critical analytics.

That combination — a foundational spec accumulating genuine production scars, alongside a handful of secondary claims that had simply gone stale — is a more honest snapshot of ecosystem maturity than either story told alone. Bugs mean people are running the software hard enough to find them. Stale claims mean nobody circled back to check. Both are normal. Neither should sit uncorrected on a reference page.

The bugs hiding in the spec everyone already trusts

Apache Iceberg is not a fringe project. An independent 2026 survey of 252 senior data and IT leaders found 58% already run business-critical analytics on it, 95% are using or planning to use it for AI/ML workloads, and 79% plan to migrate remaining data onto it within a year. That's the "the format won" read, and the adoption numbers support it.

It's also exactly the size a real bug needs to matter. Two showed up in Iceberg's own GitHub issue tracker:

Issue #17338 is a race condition in CachingCatalog. When OpenLineage — the lineage standard that's supposed to make data provenance auditable — emits a lineage event, it triggers a background loadTable() call. If that fires while a writer thread is mid-transaction, the writer can read a stale cached table reference instead of the current one. The layer built to make writes more observable can, under the wrong timing, corrupt them instead. Anyone running an OpenLineage emitter against a live Iceberg write path should treat this as a documented trap, not a hypothetical.

Issue #11664 is quieter and arguably worse, because nothing errors. Iceberg transactions initialize with a default LoggingMetricsReporter instead of whatever custom MetricsReporter you configured — so CommitReport events, which carry commit duration and file-change counts, silently never reach your reporter. ScanReport events flow through fine, which is exactly what makes this easy to miss: half your telemetry pipeline works, so nothing looks broken. Anyone building commit-level audit or cost tooling on a custom reporter should verify CommitReport events actually arrive rather than assume parity with scans.

Neither bug undermines the case for Iceberg. They undermine the case for citing "Iceberg is production-ready" as a settled fact instead of a claim with edges — the same distinction this index tries to hold everywhere else.

Claims that outlived their expiration date

The second pattern was smaller-stakes but more common: descriptions that were accurate when written and never revisited.

Apache Gravitino's summary still described it as having "graduated from Apache Incubator" without stating when — while separately, deeper in the same page, newer coverage already assumed full Top-Level Project status. It graduated in June 2025. The correction was one sentence; the claim had been sitting unreconciled for over a year.

IDrive e2's summary was worse, because the contradiction wasn't waiting on outside research — it was already on the page. The top-line description said "no object lock, no versioning, no event-notification surface" as part of the pricing trade-off. Two paragraphs later, the same node's own changelog recorded Bucket Event Notifications shipping in March 2025 and a Veeam Ready Repository certification listing Object Lock support in Governance mode. The summary and the changelog disagreed with each other, on the same page, and neither side had been updated to reconcile it. That's now fixed — both features shipped, the trade-off framing has been corrected accordingly, and it's logged as a public correction rather than a silent edit.

Chroma got a cleaner miss. Vector-database circles spent part of the summer discussing an apparent "2.4.2" version jump that would have marked a major breaking change to the wire protocol. It traces to 1.5.10.dev242 — a pre-release development build tagged August 4, 2026. The 242 is a dev-build suffix, not a version number. Chroma's core has held stable API compatibility on the 1.x line since July 2025. Nobody lied; somebody misread a tag, and it propagated.

None of these three are dramatic. That's the point — small, unglamorous, and exactly the kind of thing that calcifies into "common knowledge" if nobody checks the source a second time.

What's actually maturing underneath

Strip out the corrections and there's a real, coherent shift happening at the catalog layer, and it's the kind of shift that makes bugs like #17338 and #11664 inevitable rather than embarrassing: infrastructure gets more failure modes exactly when it gets used for more things.

The Iceberg REST Catalog Spec just grew a dedicated telemetry endpoint — POST /v1/{prefix}/namespaces/{namespace}/tables/{table}/metrics — through which Spark, Trino, Flink, and any other spec-compliant engine can push scan and commit reports back to the catalog. Before this, the catalog knew who was authorized to touch a table; it had no record of what they actually did with that access. The new endpoint closes that gap without instrumenting the compute layer at all — which is precisely the kind of catalog-as-control-plane pattern this index has been tracking since Apache Polaris, Unity Catalog, and Lakekeeper all started converging on the REST Catalog API as the common interop point rather than competing on it.

Apache XTable sits one layer up from that: instead of picking a winning table format, it translates metadata across Iceberg, Delta, and Hudi so the same physical data files stay readable from whichever engine a given team already runs. The bet embedded in both moves is the same — the war over which table format wins is quietly being replaced by infrastructure that assumes you'll need more than one.

Vector search keeps dissolving into the storage layer

Back in July this index made a call: the default home for vectors would stop being a standalone RAM-first database and start being the lake itself. Two independent product bets from opposite directions just reinforced it.

Amazon S3 Vectors puts native vector buckets and indexes directly inside S3 — up to 20 trillion vectors per bucket, no separate database to run. At current published pricing, storing a million 1,536-dimension vectors costs roughly $0.30/month; the comparable managed tier on Pinecone, Weaviate, or Qdrant starts above $30/month for the same volume, largely because those products require an ETL step out of S3 in the first place. S3 Vectors skips that step by construction.

VectorChord attacks the same problem from inside PostgreSQL. Where pgvector's HNSW index is memory-bound — billion-scale workloads either spill to disk and lose performance or buy enormous RAM machines — VectorChord's disk-based IVF+RaBitQ design cuts RAM requirements by roughly 70% while indexing a billion vectors in under two hours on a 16-vCPU box. Earth Genome's production deployment (3.2 billion vectors) reports 5x faster queries and 16x faster inserts against the same workload on pgvector. Vectors stay inside PostgreSQL's transaction boundary the whole time — no separate system to keep consistent.

Underneath both sits RDMA — the transport that makes remote object access fast enough to compete with local memory in the first place, now shipping as a standard CUDA Toolkit component rather than a technical preview. Worth naming plainly rather than skipping past: RDMA's transport layer has no TLS-equivalent. Academic work has demonstrated packet injection against production RDMA implementations, and CISA's January 2026 vulnerability bulletin alone listed five separate kernel RDMA CVEs patched in a single month. None of that is a reason to avoid RDMA for internal storage fabrics — it's a reason not to treat "it's fast" and "it's secure" as the same claim, which is the same discipline this whole post has been about.

What changed on the index

Four new nodes: Godwit Sync, a migration tool built specifically for MinIO-to-RustFS transfers with Object Lock and version-history preservation — flagged with a source note, since every reference to it currently traces back to the vendor's own site; S3 Annotations, AWS's June 2026 addition letting agents attach structured metadata to individual S3 objects without a side-table, also vendor-sourced pending independent coverage; Lance Data Storage Version, the manifest-level compatibility contract that's deliberately decoupled from the Lance SDK's much faster-moving package version number — the same category of confusion that produced the Chroma dev-tag scare above, except this one got a dedicated node instead of a viral misread; and the Iceberg REST Metrics API covered earlier. Two corrections are logged on the calls ledger: the Gravitino graduation date and the IDrive e2 feature framing.

The index runs on the idea that the strongest signal isn't a conclusion, it's a relationship — this technology solves that pain point, this spec depends on that standard — because relationships have to stay consistent with each other in a way a standalone verdict doesn't. None of the corrections above added a dramatic new relationship. They just made sure the ones already here were still true.