GARfVDB Training Visualization Fixes#59
Merged
Merged
Conversation
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
…te the image_view on first add_image Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
…eckpoint - Adjusted overlay width to 1440 and height to 720 to match the nanovdb-editor viewport for correct alignment. - Removed unused original training image size assertions and added handling for field of view (FOV) changes in camera state checks. Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR fixes GARfVDB’s integration with nanovdb-editor overlay rendering by aligning overlay projection with the viewer camera (via FOV), adjusting defaults to match the editor viewport, and addressing a known fvdb JaggedTensor shape edge case that could break feature aggregation.
Changes:
- Update overlay rendering in
view_checkpoint.pyandtrain_segmentation.pyto derive intrinsics from the viewer FOV and adjust default overlay dimensions for correct viewport alignment. - Reorder viewer initialization to avoid Vulkan/CUDA initialization-order crashes; make overlay creation safer (lazy creation in checkpoint viewer).
- Fix
GARfVDBModel.get_encoded_features()aggregation when fvdb returns a 1-level jagged tensor.
Reviewed changes
Copilot reviewed 4 out of 5 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
instance_segmentation/garfvdb/view_checkpoint.py |
Viewer init moved earlier; overlay now lazily created; projection derived from viewer FOV; camera-change tracking updated to include FOV. |
instance_segmentation/garfvdb/train_segmentation.py |
Viewer init reordered (before CUDA); overlay defaults updated; overlay projection derived from viewer FOV; visualization callback wiring refactored. |
instance_segmentation/garfvdb/garfvdb/model.py |
Guard jsum to avoid incorrect reduction when fvdb returns a 1-level JaggedTensor. |
instance_segmentation/garfvdb/garfvdb_environment.yml |
Update environment dependencies (compiler/tooling + additional packages). |
.gitignore |
Ignore .ply data files. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
… handling for inverse calculation in train_segmentation.py Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
- Moved the visualization callback assignment to the runner creation in train_segmentation.py. - Added getter and setter for the visualization callback in GaussianSplatScaleConditionedSegmentation class to improve encapsulation. Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
…I to prevent further attempts. Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
…nerate camera states, preventing potential errors during rendering. - Added checks for zero-length vectors in eye direction, right, and up vectors to ensure valid transformations. Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
…ting the segmentation overlay image fails, providing clearer feedback on the issue and disabling overlay visualization for the session. Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
Signed-off-by: Jonathan Swartz <jonathan@jswartz.info>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR fixes the integration of GARfVDB with the
nanovdb-editor's overlay rendering feature. This has several correctness fixes as well as fixes for crashes which are dependent on merging openvdb/nanovdb-editor#166 first to be able to utilize.