Skip to content

to 4.0: perf(data-branch): tighten table CTS scan range and undo blockio read filtering (cherry-pick #24371 to 4.0-dev)#24405

Open
gouhongshen wants to merge 2 commits into
matrixorigin:4.0-devfrom
gouhongshen:codex/port-data-branch-read-undo-4.0-dev
Open

to 4.0: perf(data-branch): tighten table CTS scan range and undo blockio read filtering (cherry-pick #24371 to 4.0-dev)#24405
gouhongshen wants to merge 2 commits into
matrixorigin:4.0-devfrom
gouhongshen:codex/port-data-branch-read-undo-4.0-dev

Conversation

@gouhongshen
Copy link
Copy Markdown
Contributor

@gouhongshen gouhongshen commented May 15, 2026

Backport of two follow-up commits from #24371 to 4.0-dev.

What type of PR is this?

  • API-change
  • BUG
  • Improvement
  • Documentation
  • Feature
  • Test and CI
  • Code Refactoring

Which issue(s) this PR fixes:

issue #21979 (cherry-picked from #24371)

What this PR does / why we need it:

This PR ports the two commits that landed on top of the original #24371 backport:

commit summary
ad6fb086fe0 perf: tighten data branch table cts scan range
3fbc5fbf9a revert: undo blockio read commit-ts filtering

Changes

  • Tighten the mo_tables.CollectChanges lower bound used by data branch table creation-ts resolution. Instead of scanning from BuildTS(0,1), the code reads the owning database row from mo_database with its catalog primary key and derives a tighter lower bound from created_time.
  • Add a mo_tables.__mo_cpkey_col PK filter while resolving the table creation commit timestamp, so CollectChanges can prune unrelated catalog rows.
  • Keep the existing 4.0-dev Reader fallback path for table creation-ts resolution.
  • Undo the blockio read commit-ts filtering changes from the three earlier read-path commits, including the now-dead LoadColumnsWithMeta helper and related tests.

4.0-dev conflict notes

  • 4.0-dev does not have the main-branch CollectChangesDebugLabel / PKReplaySpec context used by ad6fb086fe0, so this backport keeps the 4.0-dev PK filter shape and does not introduce those unrelated APIs.
  • pkg/fileservice/concurrent_read_oom_test.go is already deleted on 4.0-dev; the source commit only changed one comment in that file, so this PR keeps the file deleted.

Verification on 4.0-dev

  • source $HOME/.zshrc && moenv && go test ./pkg/frontend -run 'Test(PackMoDatabaseCPKey|DatabaseCreatedTimeToCollectLowerBound|TryBuildCurrentMoTablesCPKeyFilter)'
  • source $HOME/.zshrc && moenv && go test ./pkg/vm/engine/tae/blockio ./pkg/objectio/ioutil -run '^$'
  • source $HOME/.zshrc && moenv && go test ./pkg/vm/engine/tae/blockio

Special notes for your reviewer:

@qodo-code-review
Copy link
Copy Markdown

Qodo reviews are paused for this user.

Troubleshooting steps vary by plan Learn more →

On a Teams plan?
Reviews resume once this user has a paid seat and their Git account is linked in Qodo.
Link Git account →

Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center?
These require an Enterprise plan - Contact us
Contact us →

@gouhongshen gouhongshen changed the title to 4.0: port data branch lower-bound and blockio read undo to 4.0: perf(data-branch): tighten table CTS scan range and undo blockio read filtering (cherry-pick #24371 to 4.0-dev) May 15, 2026
@gouhongshen gouhongshen added kind/bug Something isn't working kind/refactor Code refactor kind/test-ci labels May 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kind/bug Something isn't working kind/enhancement kind/refactor Code refactor kind/test-ci size/L Denotes a PR that changes [500,999] lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants