Skip to content

Commit 78da023

Browse files
committed
docs(clickhouse): strengthen re-materialization warning for multi-gateway migration
Expand the "Adding a second gateway" section to make the cost explicit: - Callout box flagging full re-materialization is required - Distinguish FULL (recreate once) vs INCREMENTAL_BY_TIME_RANGE (full historical backfill) impact - Note that old 2-level names appear as Removed and --forward-only does not help (models are treated as new, not modified) Signed-off-by: Michael Day <michael.day@cloudkitchens.com> Signed-off-by: mday-io <mdaytn@gmail.com>
1 parent a1f6226 commit 78da023

1 file changed

Lines changed: 10 additions & 1 deletion

File tree

docs/integrations/engines/clickhouse.md

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -435,7 +435,16 @@ When a SQLMesh project uses ClickHouse alongside a catalog-aware gateway such as
435435

436436
### Adding a second gateway to an existing ClickHouse-only project
437437

438-
If your project previously used ClickHouse as the only gateway, your models were fingerprinted with 2-level FQNs (`db.table`). Adding a catalog-aware gateway for the first time causes all ClickHouse models to be treated as new versions (their FQNs change to `__{gateway_name}__.db.table`), triggering a full re-materialization on the next `sqlmesh apply`. This is a one-time cost.
438+
!!! warning "Re-materialization required"
439+
Adding a catalog-aware gateway (such as Trino or BigQuery) to a project that previously used ClickHouse as the only gateway triggers a **full re-materialization of every ClickHouse model** on the next `sqlmesh apply`. Plan for this before making the change.
440+
441+
If your project previously used ClickHouse as the only gateway, your models were fingerprinted with 2-level FQNs (`db.table`). Adding a catalog-aware gateway causes all ClickHouse models to be treated as new versions (their FQNs change to `__{gateway_name}__.db.table`):
442+
443+
- `FULL` models are recreated once — cost is proportional to the size of each table.
444+
- `INCREMENTAL_BY_TIME_RANGE` models require a **full historical backfill** from the model's configured start date.
445+
- The old 2-level model names appear as **Removed** in the plan and will be cleaned up after the environment TTL expires.
446+
447+
This is a one-time cost at the transition point and does not recur. There is no way to skip it — `--forward-only` does not apply because SQLMesh treats the 3-level names as new models, not modified ones.
439448

440449
### Virtual catalog naming
441450

0 commit comments

Comments
 (0)