columnar: support bucket parallel read in region (#10871)

[ti-chi-bot]

This is an automated cherry-pick of #10871

What problem does this PR solve?

Issue Number: close #10844

Problem Summary:
The read concurrency is limited in region level.

What is changed and how it works?

1. Support buckets level concurrency read in one region.
2. Optimize the streams dispatching for pipelines. Change read task pre-alloc to on-demand to avoid thread imbalance.
3. Lazy create columnar reader & prefetch create reader in async thread.
4. Reuse SnapAccess for buckets concurrent read in one request.

Check List

Tests

• Unit test
• Integration test
• Manual test (add detailed scripts or steps below)
• No code

Side effects

• Performance regression: Consumes more CPU
• Performance regression: Consumes more Memory
• Breaking backward compatibility

Documentation

• Affects user behaviors
• Contains syntax changes
• Contains variable changes
• Contains experimental features
• Changes MySQL compatibility

Release note

None

Summary by CodeRabbit

• New Features

  • PD-backed caching for region bucket keys and shared-snapshot de-duplication to reduce duplicate leader snapshot requests and improve read latency.
  • New engine interface hooks to fetch region bucket keys and clear shared-snapshot state.

• Refactor

  • Disaggregated columnar read pipeline rebuilt to a shared reader-task/work model with lazy, on-demand reader materialization for better parallelism and resource efficiency.
  • Server skips legacy read-thread initialization when columnar mode is enabled.
  • Source operator logging adds rows_per_sec and bytes_per_sec.

• Tests

  • Unit test added for cache eviction behavior during in-flight loads.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

🗂️ Base branches to auto review (3)

• release-8.5
• release-7.5
• release-8.1

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: f69fd2bf-088c-4b1c-a30b-79a62f9eec40

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[JaySon-Huang]

/retest

[JaySon-Huang]

/retest

[ti-chi-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: JaySon-Huang, yongman

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [JaySon-Huang,yongman]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[ti-chi-bot]

[LGTM Timeline notifier]

Timeline:

• 2026-06-11 14:09:54.879549705 +0000 UTC m=+1055495.949867125: ☑️ agreed by yongman.
• 2026-06-11 15:28:09.811801144 +0000 UTC m=+1060190.882118534: ☑️ agreed by JaySon-Huang.