Support non-sequence meta-features in PyGrain packing transformations.#1334
Open
copybara-service[bot] wants to merge 1 commit into
Open
Support non-sequence meta-features in PyGrain packing transformations.#1334copybara-service[bot] wants to merge 1 commit into
copybara-service[bot] wants to merge 1 commit into
Conversation
Previously, only sequence meta-features (defined in `length_struct`) were supported. Non-sequence meta-features (in `meta_features` but not in `length_struct`) were stripped or caused errors during packing. This change adds support for non-sequence meta-features in both FirstFit and BestFit packing methods: - Python implementation (`PackedBatch`): Non-sequence meta-features are identified, accumulated as lists during packing, and yielded as 1D numpy object arrays of lists. - Python Iterator (`PackingDatasetIterator`): The `_combined_struct` is updated to include non-sequence meta-features so they are not stripped from input elements. - Refactored common key-extraction logic into shared helper functions in `packing_packed_batch.py`. - Added unit tests in `testing_util.py` to verify FirstFit and BestFit with non-sequence meta-features (both fixed and variable shapes). - Updated docstrings in `packing.py` to document the behavior of sequence vs non-sequence meta-features. PiperOrigin-RevId: 926448116
copybara-service
Bot
force-pushed
the
test_926448116
branch
from
f1634b9 to
47d5b08
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Support non-sequence meta-features in PyGrain packing transformations.
Previously, only sequence meta-features (defined in
length_struct) were supported. Non-sequence meta-features (inmeta_featuresbut not inlength_struct) were stripped or caused errors during packing.This change adds support for non-sequence meta-features in both FirstFit and BestFit packing methods:
PackedBatch): Non-sequence meta-features are identified, accumulated as lists during packing, and yielded as 1D numpy object arrays of lists.PackingDatasetIterator): The_combined_structis updated to include non-sequence meta-features so they are not stripped from input elements.packing_packed_batch.py.testing_util.pyto verify FirstFit and BestFit with non-sequence meta-features (both fixed and variable shapes).packing.pyto document the behavior of sequence vs non-sequence meta-features.