Bump parquet from 1.16.0 to 1.17.0#17504
Conversation
There was a problem hiding this comment.
Pull request overview
This PR upgrades the Parquet library from version 1.16.0 to 1.17.0 and addresses a breaking change in how INT96 fields are identified. The upgrade was necessary due to a bug fix in parquet-java (PR #3311) that changed the field name representation for INT96 types.
Changes:
- Bumped parquet version from 1.16.0 to 1.17.0 in the root POM
- Updated INT96 field identification logic to check fixed-size byte arrays with documentation instead of relying on field names
- Improved code consistency by removing unnecessary fully qualified class names and adding defensive assertions
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| pom.xml | Updates parquet dependency version to 1.17.0 |
| ParquetAvroRecordExtractor.java | Refactors INT96 detection logic to use fixed-size and documentation checks instead of field name comparison, removes unused import, and simplifies Schema type references |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #17504 +/- ##
=========================================
Coverage 63.25% 63.26%
+ Complexity 1477 1476 -1
=========================================
Files 3167 3167
Lines 189175 189177 +2
Branches 28948 28950 +2
=========================================
+ Hits 119661 119678 +17
+ Misses 60235 60202 -33
- Partials 9279 9297 +18
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Fix the way to identify the
INT96fieldThe old way breaks because of this bug fix: apache/parquet-java#3311
The name of the field is no longer
"INT96"