Skip to content

Enforce co-partitioning for sort merge and symmetric hash joins#23480

Open
gene-bordegaray wants to merge 1 commit into
apache:mainfrom
gene-bordegaray:gene.bordegaray/2026/07/co-partitioned-join-requirements
Open

Enforce co-partitioning for sort merge and symmetric hash joins#23480
gene-bordegaray wants to merge 1 commit into
apache:mainfrom
gene-bordegaray:gene.bordegaray/2026/07/co-partitioned-join-requirements

Conversation

@gene-bordegaray

@gene-bordegaray gene-bordegaray commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Rationale for this change

Partitioned sort merge and symmetric hash joins process matching child partition indexes together. Independent KeyPartitioned requirements do not express that cross-child invariant.

What changes are included in this PR?

  • Declare co-partitioned input requirements for SortMergeJoinExec and partitioned SymmetricHashJoinExec
  • Add sanity-check coverage for compatible and incompatible hash-partitioned children.
  • Add range-partitioning SLT coverage showing both joins repartitioned until their dedicated Range opt-in work lands.

Are these changes tested?

Yes.

Are there any user-facing changes?

None.

@github-actions github-actions Bot added core Core DataFusion crate sqllogictest SQL Logic Tests (.slt) physical-plan Changes to the physical-plan crate labels Jul 11, 2026
.map(|exec| Arc::new(exec) as Arc<dyn ExecutionPlan>)
}

fn hash_partitioned_exec(

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

using hash to prove that co-partitioning dows work fro an accepted partitioning type

@gene-bordegaray gene-bordegaray marked this pull request as ready for review July 11, 2026 13:54
@gene-bordegaray

Copy link
Copy Markdown
Contributor Author

@stuhood @mithuncy @gabotechs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core Core DataFusion crate physical-plan Changes to the physical-plan crate sqllogictest SQL Logic Tests (.slt)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Audit multi-child Distribution::KeyPartitioned requirements before general Range satisfaction

1 participant