Release-4

✨ What's Changed

• Vision Pipeline v1 — Initial release of the full perception pipeline with camera integration, object detection, and sub-door normal estimation @Mohsin-Ali-Mirza

• Table height estimation via RANSAC — Dedicated ROS node for computing table height using RANSAC plane fitting @rinaalo

• Motor Pose rotation support — Added rotation handling for motor, speaker, actuation; poses and class names now returned in correct order @Mohsin-Ali-Mirza

• Low GPU mode — Configuration profile for running the pipeline on resource-constrained hardware @Mohsin-Ali-Mirza

• Pickable / non-pickable class handling — Detected objects are now classified by graspability, enabling downstream manipulation planning @Mohsin-Ali-Mirza

• Table height estimation via RANSAC and PCD Voxel — Table height using RANSAC and PCD Voxel @AsjadAfnan

Extra

• Voice command dispatcher — Added voice_dispatcher.py for processing and routing voice commands through the pipeline @Mohsin-Ali-Mirza

Full Changelog: v3.0.0...release-4

Images Labeled

Contributor	Screwdriver	Motor Grip	Screw	Subdoor	Total
Mohsin	200	1,950	258	330	2,738
Rina	196	1,723	260	330	2,509
Asjad	196	0	260	50	506

Roboflow Project IDs

Dataset Name	Roboflow Project ID
Screw Dataset	screw-zu99g
Car_Objects Dataset	fix_motor + backup_dataset
Screwdriver Dataset	screwdriver-gftmm
Subdoor Dataset	subdoor_segmentation

Release 3

What's Changed

• Table Height Predictor by @Mohsin-Ali-Mirza in #42
• Added Action Server for Perception by @Mohsin-Ali-Mirza in #43
• Added Table Segmentation by @rinaalo in #38
• Tool Detection and Pose Estimation by @AsjadAfnan at here

Full Changelog: v2.0.1...v3.0.0

Milestone Release 2

New Features & Core Functionality

• Object Classifier for Manipulation: A new ROS 2 node has been created to classify detected objects as either 'Pickable' or 'Non-Pickable'. This provides essential downstream information for robotic grasping and interaction tasks.
  • Assignee: Mohsin-Ali-Mirza

Bug Fixes & System Improvements

• Resolved Absolute Path Integration Bug: Fixed a critical issue where hardcoded model paths prevented the system from being portable. The nodes now use dynamic, package-relative paths, allowing the project to run on any machine without code changes.

  • Assignee: Mohsin-Ali-Mirza

• Unified Robot & RealSense Launch File: A single, flexible launch file has been developed to start the perception stack with either the Kinova robot's camera or a RealSense camera. This streamlines the startup process and simplifies configuration.

  • Assignee: Mohsin-Ali-Mirza

• Main Branch Cleanup: Performed repository maintenance by cleaning the main branch to ensure code stability and a clean version history for future development.

  • Assignee: Mohsin-Ali-Mirza

• Parameters for Yolo Detection: A much more flexible script, which allows the user to now set cpu/gpu and confidence interval without manually changing the main script for the Yolo Object Detection Model.

  • Assignee: Asjad-Afnan

Data & Model Foundation

• Extensive Image Annotation Completed: A total of 1310 images by Asjad Afnan and 3364 images by Mohsin Ali Mirza were meticulously annotated. This includes 6 objects (Assembly, Wire-Plug, Control-Unit, Speaker, Motor, Enclosure). The dataset was enriched with realistic noise from both human and tool-based labeling processes to improve model generalization and robustness.

• Annotated_Dataset.zip gives a visualization of the annotation of the bounding boxes for the dataset.

• Final Dataset.zip is the Yolov8 supported dataset used to train the model.

Documentation & Planning

• Updated Documentation for Zenoh & Image Pipeline: The project's documentation has been significantly improved with new, detailed instructions for setting up the ROS 2 environment with rmw_zenoh_cpp and the image-pipeline package.

  • Assignee: Mohsin-Ali-Mirza

• Strategic Plan for Perception Team: A comprehensive proposal outlining a future roadmap and strategic plan for the Perception Team was developed and presented.

  • Assignee: Mohsin-Ali-Mirza

Milestone Release 1

Highlights

• Fine-tuning pipeline using Vision Transformer for annotation
• YOLOv8 training and inference notebooks
• ROS 2 nodes for object detection and pose estimation
• Support for RealSense and Kinova cameras
• Visualization in RViz2 with preconfigured layout
• Dataset for 4 car parts [Control Unit, Window Assembly, Loud Speaker, Control Unit] in 640x480 and 1280x720 resolutions

Setup

• Python 3.12 and ROS 2 Humble supported
• Two workspaces: master_project (robot integration) and ros2_ws (perception logic)
• Pretrained model saved to runs/detect/.../best.pt
• Topics: /annotated_images, /cropped_pointcloud

Refer to the README for installation and usage instructions.