chore: merge upstream v1.5-variegata for DuckDB v1.5.1#3
Merged
Conversation
The file size check (>= target_file_size) was intended for MERGE_ADJACENT_TABLES to avoid merging already-large files, but it incorrectly also gated REWRITE_DELETES. This caused data files larger than target_file_size (default 512 MB) with associated delete files to be silently skipped by ducklake_rewrite_data_files, leaving delete files that can never be eliminated. Fixes duckdb#821
…iles Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…-files Fix: rewrite_data_files skips large files with delete files
All access paths to table_data_changes go through the guard lock
ducklake_delete_orphaned_files now only deletes parquet files
Fix issue when parsing table macros
This reverts commit 644f808.
… multiple macros in same transaction
Cleanup separator in DATA_PATH
Fix issue with macro and schema being created in same transaction and multiple macros in same transaction
…d-output fix: compaction should not inherit per_thread_output from lake config
…asted to handle the sink, since merge gotta do that
Fix for s3 and SET disabled_filesystems = 'LocalFileSystem'
… in the insertion)
…elete_inling Cache existence of GetInlinedDeletionTableName
Inline Insertions in Updates
Ducklake serialize
Copy the delete data to avoid corruption of snapshot_ids borking delete merges
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Merge upstream
duckdb/ducklakev1.5-variegatabranch into relytcloud's main for DuckDB v1.5.1 ABI compatibility.Conflict resolutions
4 files had merge conflicts between relytcloud customizations and upstream changes:
postgres_metadata_manager.hpp-- KeptInlinedDeletionTableExistsoverride (relytcloud), dropped duplicateTransformInlinedDatadeclarationpostgres_metadata_manager.cpp-- Combined bothTransformInlinedDataimplementations (upstream's BLOB->VARCHAR Reinterpret + relytcloud's VectorOperations::Cast for general type mismatches)ducklake_metadata_manager.cpp-- Kept relytcloud's virtualQuery()calls + upstream's error-throwing patterns; kept relytcloud'sInlinedDeletionTableExists()andGetActiveFiles()helpersducklake_transaction.cpp-- Kept relytcloud's null guard onconnection->BeginTransaction()Known issue
DuckDB v1.5.1 changed catalog transaction isolation: a catalog ATTACHed via
ExecuteDuckDBQuery(SPI/raw_query path) becomes invisible in subsequent PG transactions. This causes"Catalog __ducklake_metadata_pgducklake does not exist"errors inpg_regresstests (which use file mode where each statement is a separate transaction). Manualpsql -c(single transaction) works fine. The fix likely requires ATTACHing via the DuckDB connection directly instead of through SPI.Related
mainbranch (CI green)