Conversation
fix: use type() to build _BaseClass when MONAI is installed
Feature/minimal deps
Add Hugging Face model export functionality and update README with usage examples Closes #45
There was a problem hiding this comment.
Pull request overview
This PR adds a HuggingFace transformers-compatible export path for the pretrained SPECTRE model, reorganizes packaging dependencies into install extras, and hardens the codebase against missing optional dependencies (e.g., MONAI, Accelerate, OmegaConf, SciPy, Pandas) so base/inference installs are lighter.
Changes:
- Add HuggingFace export artifacts (
hf_export/*) and an export/upload script (scripts/export_hf.py) to publish atransformers-loadable model. - Split
pyproject.tomldependencies into optional extras (training,eval,gds-cuda12,all) and add runtime deps needed for inference/HF usage. - Add optional-import guards and clearer ImportErrors across data/transforms/utils, plus meta-tensor-safe initialization in ViT modules.
Reviewed changes
Copilot reviewed 33 out of 34 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| src/spectre/utils/distributed.py | Make accelerate optional and fail fast with a clearer error when distributed init is used without it. |
| src/spectre/utils/dataloader.py | Make MONAI optional; raise a targeted error when dataloader creation is attempted without MONAI. |
| src/spectre/utils/config.py | Make OmegaConf optional; raise targeted errors for config load/save paths when missing. |
| src/spectre/utils/collate.py | Make MONAI optional; raise targeted errors for MONAI-based collators when missing. |
| src/spectre/utils/_utils.py | Make MONAI optional; gate determinism helper on MONAI availability. |
| src/spectre/transforms/scale_intensity_range.py | Make MONAI optional; prevent transform use without MONAI while keeping import-time safe. |
| src/spectre/transforms/largest_multiple_crop.py | Make MONAI optional; prevent transform use without MONAI while keeping import-time safe. |
| src/spectre/transforms/generate_report.py | Make MONAI optional; prevent transform use without MONAI while keeping import-time safe. |
| src/spectre/transforms/combine_labels.py | Make MONAI optional; prevent transform use without MONAI while keeping import-time safe. |
| src/spectre/ssl/transforms/siglip_transform.py | Make MONAI optional; prevent transform use without MONAI while keeping import-time safe. |
| src/spectre/ssl/transforms/mae_transform.py | Make MONAI optional; prevent transform use without MONAI while keeping import-time safe. |
| src/spectre/ssl/transforms/dino_transform.py | Make MONAI optional; prevent transform use without MONAI while keeping import-time safe. |
| src/spectre/models/vision_transformer.py | Avoid initializing meta tensors; adjust stochastic depth schedule computation. |
| src/spectre/models/vision_transformer_features.py | Avoid initializing meta tensors; adjust stochastic depth schedule computation. |
| src/spectre/losses/mask_classification_loss.py | Make SciPy optional; raise targeted error when the loss is instantiated without SciPy. |
| src/spectre/data/total_segmentator.py | Route datasets through _base_datasets wrappers and gate Pandas usage. |
| src/spectre/data/sinoct.py | Route datasets through _base_datasets wrappers and gate Pandas usage. |
| src/spectre/data/panorama.py | Route datasets through _base_datasets wrappers for MONAI optionality. |
| src/spectre/data/nlst.py | Route datasets through _base_datasets wrappers for MONAI optionality. |
| src/spectre/data/merlin.py | Route datasets through _base_datasets wrappers and gate Pandas usage. |
| src/spectre/data/inspect.py | Route datasets through _base_datasets wrappers and gate Pandas usage. |
| src/spectre/data/ct_rate.py | Route datasets through _base_datasets wrappers and gate Pandas usage. |
| src/spectre/data/amos.py | Route datasets through _base_datasets wrappers for MONAI optionality. |
| src/spectre/data/abdomenct_1k.py | Route datasets through _base_datasets wrappers for MONAI optionality. |
| src/spectre/data/abdomen_atlas.py | Route datasets through _base_datasets wrappers for MONAI optionality. |
| src/spectre/data/_base_datasets.py | Introduce MONAI/GDS dependency gates and wrapper dataset classes; update MONAI utility calls. |
| src/spectre/configs/init.py | Make OmegaConf optional and avoid import-time crashes by warning and setting defaults to None. |
| scripts/export_hf.py | Add an export/test/upload script that builds a minimal spectre package snapshot for the Hub. |
| README.md | Document new transformers loading path and new installation extras. |
| pyproject.toml | Split dependencies into extras and add runtime dependencies for HF usage. |
| hf_export/modeling_spectre.py | Add a transformers.PreTrainedModel wrapper for SPECTRE inference. |
| hf_export/configuration_spectre.py | Add a transformers.PretrainedConfig for SPECTRE. |
| hf_export/metadata.yaml | Add HF model card metadata frontmatter. |
| .gitignore | Ignore generated HF export artifacts. |
Comments suppressed due to low confidence (1)
README.md:56
- The comment says the dummy input is shaped
(batch, crops, channels, height, width, depth), but the example tensorx = torch.randn(1, 1, 384, 384, 256)is 5D(batch, channels, height, width, depth)and only later reshaped into patch crops. Updating the comment would avoid confusion about the expected model input shape.
# Dummy input: (batch, crops, channels, height, width, depth)
# For a (3 x 3 x 4) grid of (128 x 128 x 64) CT patches -> Total scan size (384 x 384 x 256)
x = torch.randn(1, 1, 384, 384, 256)
B, C, H, W, D = x.shape
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| "kvikio", | ||
| ] | ||
|
|
||
| all = ["spectre-fm[training,eval,gds-cuda12]"] |
Comment on lines
+7
to
+16
| def __init__( | ||
| self, | ||
| backbone_name="vit_large_patch16_128", | ||
| backbone_kwargs={ | ||
| "num_classes": 0, | ||
| "global_pool": '', | ||
| "pos_embed": "rope", | ||
| "rope_kwargs": {"base": 1000.0}, | ||
| "init_values": 1.0, | ||
| }, |
Comment on lines
+10
to
+32
| backbone_kwargs={ | ||
| "num_classes": 0, | ||
| "global_pool": '', | ||
| "pos_embed": "rope", | ||
| "rope_kwargs": {"base": 1000.0}, | ||
| "init_values": 1.0, | ||
| }, | ||
| feature_combiner_name="feat_vit_large", | ||
| feature_combiner_kwargs={ | ||
| "num_classes": 0, | ||
| "global_pool": "", | ||
| "pos_embed": "rope", | ||
| "rope_kwargs": {"base": 100.0}, | ||
| "init_values": 1.0, | ||
| }, | ||
| **kwargs, | ||
| ): | ||
| super().__init__(**kwargs) | ||
|
|
||
| self.backbone_name = backbone_name | ||
| self.backbone_kwargs = backbone_kwargs or {} | ||
| self.feature_combiner_name = feature_combiner_name | ||
| self.feature_combiner_kwargs = feature_combiner_kwargs or {} |
Comment on lines
+33
to
+36
| return_dict=False, | ||
| **kwargs, | ||
| ): | ||
| outputs = self.model(pixel_values, grid_size=grid_size) |
Comment on lines
+291
to
+295
| repo_id=args.repo_id, | ||
| repo_type="model", | ||
| commit_message=args.commit_message, | ||
| delete_patterns=["*"], | ||
| ) |
| f"{hashfile}-{k}-{i}", dtype=meta_i_k["dtype"], like=cp.empty(()) | ||
| ) | ||
| item_k = convert_to_tensor(item[i].reshape(meta_i_k["shape"]), device=f"cuda:{self.device}") | ||
| item_k = monai.utils.convert_to_tensor(item[i].reshape(meta_i_k["shape"]), device=f"cuda:{self.device}") |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request introduces support for exporting and using the pretrained SPECTRE model directly with the HuggingFace
transformerslibrary, reorganizes dependencies for more flexible installation, and improves optional dependency handling. The most significant changes include a new export script for HuggingFace, the addition of model and config classes compatible withtransformers, and updates to the documentation and packaging to reflect these improvements.HuggingFace Integration:
hf_export/modeling_spectre.pyandhf_export/configuration_spectre.pyimplementingSpectreModelandSpectreConfigfor compatibility with HuggingFacetransformers. [1] [2]hf_export/metadata.yamlfor model card metadata on HuggingFace Hub.scripts/export_hf.py, a script to export, test, and optionally upload the SPECTRE model to the HuggingFace Hub, including selective packaging of source files.Dependency Management and Installation:
pyproject.tomlto split dependencies into optional groups (training,eval,gds-cuda12,all), making it easier to install only what is needed for inference, training, or evaluation.src/spectre/data/_base_datasets.pyandsrc/spectre/configs/__init__.pyto handle missing optional dependencies gracefully, raising clear errors or warnings if modules likemonai,cupy, oromegaconfare not installed. [1] [2]Documentation Updates:
README.mdto announce HuggingFace support, provide new import examples usingtransformers, and clarify installation options for different use cases (inference, training, evaluation, GDS/CUDA). [1] [2] [3] [4]These changes make it much easier to use SPECTRE for inference via HuggingFace, streamline installation for different user needs, and improve the robustness of the codebase with respect to optional dependencies.