Open-source replication of databases and Kafka into Apache Iceberg.
Built by Datazip.
Website · Docs · Blog · Slack · X
| Project | What it does |
|---|---|
| olake | CDC, full-refresh and incremental replication from PostgreSQL, MySQL, MongoDB, Oracle, SQL Server, DB2, Kafka and S3 into Apache Iceberg (or Parquet). Single Go binary, Apache-2.0. |
| olake-fusion | Iceberg table maintenance: compaction, snapshot expiry, orphan-file cleanup. |
| olake-ui | Web UI and backend for configuring sources, destinations and sync jobs. |
| olake-helm | Helm chart for running OLake on Kubernetes. |
| olake-docs | Source for olake.io: docs, blog, customer stories. |
| workshops | Hands-on workshops on Apache Iceberg and related tooling. |
- No Kafka Connect, Flink or Spark in the path. OLake reads WAL / binlog / change streams and writes Iceberg directly.
- Catalogs: AWS Glue, REST, Hive, JDBC, Polaris, Nessie, Lakekeeper, Unity, Snowflake Open Catalog.
- Operational: parallel chunked backfills, schema evolution, checkpointed resumable state, exactly-once-style writes.
- Runs anywhere: CLI, Docker, Kubernetes (Helm). Optional web UI.
docker run -it olakego/source-postgres:latest --helpQuickstart: https://olake.io/docs/getting-started
- Slack: https://olake.io/slack
- Issues and feature requests: https://github.com/datazip-inc/olake/issues
- Contributing guide: https://github.com/datazip-inc/olake/blob/master/CONTRIBUTING.md
- Benchmarks: https://olake.io/docs/benchmarks
Used in production by PhysicsWallah, AstroTalk, LendingKart, Cordial, Bitespeed and Xeno. Read the customer stories.