Skip to content

[server][lake] TableInfo exposes inherited Iceberg warehouse as a table property #3812

Description

@litiliu

Search before asking

  • I searched in the issues and found nothing similar.

Fluss version

main (development)

Please describe the bug 🐞

Description

For an Iceberg-enabled Fluss table, Admin#getTableInfo (and consumers such as catalogs or UIs) exposes cluster-level Iceberg catalog options as table properties by adding a table. prefix.

In the reproduced setup, the Fluss cluster is configured with:

datalake.format: iceberg
datalake.iceberg.type: glue
datalake.iceberg.warehouse: s3://<bucket>/fluss/iceberg

an Iceberg-enabled table is reported with:

table.datalake.iceberg.warehouse=s3://<bucket>/fluss/iceberg

However, an existing Iceberg table may be stored at an explicit location such as s3://<bucket>/fluss/native_implicit_day_pk_demo_v2. The reported warehouse still points to s3://<bucket>/fluss/iceberg, even though the table is stored somewhere else. Because Fluss exposes this value as a table property, users can easily mistake it for the table's actual location.

How to reproduce

  1. Use a Fluss cluster configured with datalake.iceberg.warehouse: s3://<bucket>/fluss/iceberg.
  2. Create native_implicit_day_pk_demo_v2 in the same Glue catalog with the explicit table location s3://<bucket>/fluss/native_implicit_day_pk_demo_v2.
  3. Create or bind the matching Fluss table with table.datalake.enabled=true.
  4. Call Admin#getTableInfo or inspect the table through a catalog/UI.
  5. Observe that table.datalake.iceberg.warehouse contains the cluster-level default warehouse even though the Iceberg table uses a different location.

Expected behavior

TableInfo.properties should not make inherited cluster-level catalog configuration look like persisted table properties. Inherited/resolved catalog options should be distinguishable from stored table properties, and a warehouse value should not be presented as the actual location of an existing Iceberg table.

Current behavior and impact

LakeStorageUtils#generateDefaultTableLakeOptions prefixes cluster options with table., and TableRegistration#toTableInfo overlays them when returning TableInfo.

This is misleading for users and metadata consumers. It also means the reported table properties can change when cluster configuration changes, without any ALTER TABLE, and the problem is visible when binding existing Iceberg tables with custom locations.

Related history

Solution

Separate persisted table properties from inherited/resolved lake catalog options.

Possible approaches:

  • expose resolved lake catalog options through a dedicated metadata structure instead of merging them into TableInfo.properties; or
  • retain the resolved options but mark their source explicitly and keep persisted table properties separate.

Add a regression test with a cluster warehouse and an existing Iceberg table whose explicit location differs from that warehouse.

Are you willing to submit a PR?

  • I'm willing to submit a PR!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions