Skip to content

fix: Projection stats Absent for columns referenced >1 time#22679

Merged
Dandandan merged 2 commits into
apache:mainfrom
neilconway:neilc/fix-projection-stats
Jun 1, 2026
Merged

fix: Projection stats Absent for columns referenced >1 time#22679
Dandandan merged 2 commits into
apache:mainfrom
neilconway:neilc/fix-projection-stats

Conversation

@neilconway
Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Rationale for this change

ProjectionExprs::project_statistics uses std::mem::take to move an input column's ColumnStatistics into the output when given a direct column reference. This means if the column is referenced again (either directly or in a CAST expression), the statistics are Absent. The simple fix is to just clone instead of take.

This pattern crops up in TPC-DS q54, which includes a CTE that projects both d_date_sk and CAST(d_date_sk AS Float64), but it's a more general bug.

What changes are included in this PR?

  • Fix bug
  • Add unit tests

Are these changes tested?

Yes; new test added.

Are there any user-facing changes?

No.

@github-actions github-actions Bot added the physical-expr Changes to the physical-expr crates label May 31, 2026
Copy link
Copy Markdown
Contributor

@Dandandan Dandandan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice find!

Copy link
Copy Markdown
Member

@asolimando asolimando left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Comment thread datafusion/physical-expr/src/projection.rs Outdated
@neilconway neilconway changed the title fix: Projection stats were Absent for columns referenced >1 time fix: Projection stats Absent for columns referenced >1 time Jun 1, 2026
@Dandandan Dandandan enabled auto-merge June 1, 2026 11:14
@Dandandan Dandandan added this pull request to the merge queue Jun 1, 2026
Merged via the queue into apache:main with commit 85bc5ef Jun 1, 2026
38 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

physical-expr Changes to the physical-expr crates

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Projection stats Absent for columns referenced >1 time

3 participants