Supermetal

A Rust-based CDC/data-integration platform that moves data from a database's write-ahead log to a committed Apache Iceberg table on S3 in a **single process** — no Kafka, no ZooKeeper, no intermediate JVM stream processor. WAL events decode directly into Apache Arrow record batches and write out as Parquet: columnar end-to-end, eliminating the serialize-to-JSON/deserialize-in-Flink/re-serialize-to-Parquet tax of the traditional Debezium→Kafka→Flink pipeline. Per [Supermetal — Announcing the Apache Iceberg target](https://supermetal.io/blog/iceberg-target).

For a large class of workloads, the multi-hop CDC architecture exists for historical reasons, not technical ones. Supermetal's bet is that a phase-aware single process beats a distributed pipeline on everything but extreme scale: during snapshot it runs a highly parallelized append-only mode (no dedup, perfectly sized Parquet straight to S3); when it transitions to live CDC it switches to writing **Iceberg V3 deletion vectors** (or V2 position deletes) for update handling — the writer changes behavior by pipeline phase, a sideways-information-passing trick borrowed from database engines.

Postgres/MySQL→Iceberg replication without operating Kafka or Flink, fast initial snapshots of large operational databases into the lakehouse, small-team CDC where the pipeline must be one deployable unit, migrations off multi-hop Debezium stacks.