to 4.0: perf(data-branch): tighten table CTS scan range and undo blockio read filtering (cherry-pick #24371 to 4.0-dev)#24405
Open
gouhongshen wants to merge 2 commits into
Conversation
Qodo reviews are paused for this user.Troubleshooting steps vary by plan Learn more → On a Teams plan? Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center? |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Backport of two follow-up commits from #24371 to
4.0-dev.What type of PR is this?
Which issue(s) this PR fixes:
issue #21979 (cherry-picked from #24371)
What this PR does / why we need it:
This PR ports the two commits that landed on top of the original #24371 backport:
ad6fb086fe0perf: tighten data branch table cts scan range3fbc5fbf9arevert: undo blockio read commit-ts filteringChanges
mo_tables.CollectChangeslower bound used by data branch table creation-ts resolution. Instead of scanning fromBuildTS(0,1), the code reads the owning database row frommo_databasewith its catalog primary key and derives a tighter lower bound fromcreated_time.mo_tables.__mo_cpkey_colPK filter while resolving the table creation commit timestamp, so CollectChanges can prune unrelated catalog rows.4.0-devReader fallback path for table creation-ts resolution.LoadColumnsWithMetahelper and related tests.4.0-dev conflict notes
4.0-devdoes not have the main-branchCollectChangesDebugLabel/PKReplaySpeccontext used byad6fb086fe0, so this backport keeps the4.0-devPK filter shape and does not introduce those unrelated APIs.pkg/fileservice/concurrent_read_oom_test.gois already deleted on4.0-dev; the source commit only changed one comment in that file, so this PR keeps the file deleted.Verification on 4.0-dev
source $HOME/.zshrc && moenv && go test ./pkg/frontend -run 'Test(PackMoDatabaseCPKey|DatabaseCreatedTimeToCollectLowerBound|TryBuildCurrentMoTablesCPKeyFilter)'source $HOME/.zshrc && moenv && go test ./pkg/vm/engine/tae/blockio ./pkg/objectio/ioutil -run '^$'source $HOME/.zshrc && moenv && go test ./pkg/vm/engine/tae/blockioSpecial notes for your reviewer:
diff_12.sqlCI runtime improved from roughly 145s to roughly 4s after the data branch lower-bound optimization.