Skip to content

[server] Support configurable time partition format for auto-partitioned tables#3200

Open
wattt3 wants to merge 3 commits into
apache:mainfrom
wattt3:auto-partition-time-format
Open

[server] Support configurable time partition format for auto-partitioned tables#3200
wattt3 wants to merge 3 commits into
apache:mainfrom
wattt3:auto-partition-time-format

Conversation

@wattt3
Copy link
Copy Markdown

@wattt3 wattt3 commented Apr 25, 2026

Purpose

Linked issue: close #3191

Brief change log

  • Add table.auto-partition.time-format to override the unit partition value format
  • Validate pattern syntax at table creation
    • the zero-padded / lex-orderable contract is the users's responsibility. (retention compares partition names via TreeMap)

Tests

API and Format

Documentation

@wattt3
Copy link
Copy Markdown
Author

wattt3 commented Apr 25, 2026

@luoyuxia PTAL, thanks.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new table option to customize the string format used for time-based auto-partition values, propagating that option through partition generation/retention logic and documenting the new behavior.

Changes:

  • Introduce table.auto-partition.time-format (no default; derived from time-unit when unset) and wire it into auto-partition creation and retention.
  • Validate custom time format pattern syntax during table descriptor validation, with new unit tests.
  • Update docs and existing tests/call sites for updated partition utility APIs.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
website/docs/table-design/data-distribution/partitioning.md Documents the new table.auto-partition.time-format option for auto-partitioned tables.
website/docs/engine-flink/options.md Exposes the new option in Flink engine table options documentation.
fluss-server/src/test/java/org/apache/fluss/server/utils/TableDescriptorValidationTest.java Adds coverage for accepting/rejecting custom time-format values at table creation validation time.
fluss-server/src/test/java/org/apache/fluss/server/coordinator/TableManagerITCase.java Updates test helper call site for new generateAutoPartition(...) signature.
fluss-server/src/main/java/org/apache/fluss/server/utils/TableDescriptorValidation.java Adds table-create-time validation for time-format pattern syntax.
fluss-server/src/main/java/org/apache/fluss/server/coordinator/AutoPartitionManager.java Passes time-format through to partition pre-creation and retention cutoff calculation.
fluss-common/src/test/java/org/apache/fluss/utils/PartitionUtilsTest.java Adds tests for partition name generation with a custom time format.
fluss-common/src/main/java/org/apache/fluss/utils/PartitionUtils.java Extends partition time generation/validation to accept an optional custom time-format and uses Locale.ROOT.
fluss-common/src/main/java/org/apache/fluss/utils/AutoPartitionStrategy.java Adds timeFormat to the resolved auto-partition strategy from table options.
fluss-common/src/main/java/org/apache/fluss/config/ConfigOptions.java Defines the new table.auto-partition.time-format config option and its description.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread website/docs/table-design/data-distribution/partitioning.md Outdated
Comment thread website/docs/engine-flink/options.md Outdated
@wattt3
Copy link
Copy Markdown
Author

wattt3 commented May 19, 2026

@luoyuxia
Addresed Copilot's review.
PTAL, thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support configurable time partition format for auto-partitioned tables

2 participants