Aruco grid filter by sophia-mina · Pull Request #18 · vortexntnu/vortex-image-filtering

sophia-mina · 2026-02-11T20:31:47Z

ArUco grid filter

During the last competition, one of the ArUco markers on the structure was obscured by a grid, causing the detector to fail. This is because successful scanning requires that the detector can first identify the marker’s encoded pattern.

The filter removes the grid from the image and uses the inpaint function from OpenCV which is constrained by a bounding box to reconstruct the ArUco marker.

colcon build --symlink-install --packages-up-to image_filtering aruco_detector
source install/setup.bash
ros2 launch image_filtering image_filtering.launch.py

and in another terminal write

ros2 launch aruco_detector aruco_detector.launch.py

Use the simulator with the scenario:=tacc.

… necessary variables

codecov · 2026-02-11T20:40:36Z

Codecov Report

❌ Patch coverage is 0% with 72 lines in your changes missing coverage. Please review.
✅ Project coverage is 0.00%. Comparing base (d1b80c4) to head (93a3bdb).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
...nclude/image_filtering/lib/filters/remove_grid.hpp	0.00%	60 Missing ⚠️
image-filtering/src/ros/image_filtering_ros.cpp	0.00%	12 Missing ⚠️

Additional details and impacted files

@@          Coverage Diff          @@
##            main     #18   +/-   ##
=====================================
  Coverage   0.00%   0.00%           
=====================================
  Files         20      21    +1     
  Lines        418     490   +72     
  Branches      34      39    +5     
=====================================
- Misses       418     490   +72

Flag	Coverage Δ
unittests	`0.00% <0.00%> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
...-filtering/include/image_filtering/lib/typedef.hpp	`0.00% <ø> (ø)`
image-filtering/src/ros/image_filtering_ros.cpp	`0.00% <0.00%> (ø)`
...nclude/image_filtering/lib/filters/remove_grid.hpp	`0.00% <0.00%> (ø)`

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

kluge7

What you need to do:

Fix pre-commit errors
Make it so when you rotate the box, you only rotate that and not the rest of the image
Optimize the code?
Use binary_threshold (apply_fixed_threshold?) instead (avoids duplicating code)

Copilot

Pull request overview

This PR introduces a new image filter called "RemoveGrid" to address a competition issue where ArUco markers on a structure were obscured by a green grid, preventing successful detection. The filter removes the grid using color-based masking and inpaints the affected regions using OpenCV's inpainting algorithms, followed by binary thresholding to reconstruct the ArUco marker for detection.

Changes:

Added RemoveGrid filter implementation with rotation, cropping, green channel masking, inpainting, and binary thresholding
Integrated the new filter into the ROS node with configurable parameters
Updated configuration defaults to use the new filter with specific camera topics

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
image-filtering/include/image_filtering/lib/filters/remove_grid.hpp	New filter implementation that rotates, crops, masks green grid pixels, inpaints, and applies binary threshold
image-filtering/include/image_filtering/lib/typedef.hpp	Added RemoveGrid enum value and string mapping for filter type selection
image-filtering/include/image_filtering/lib/filters/all_filters.hpp	Added include for the new remove_grid.hpp header
image-filtering/src/ros/image_filtering_ros.cpp	Added parameter handling and instantiation logic for RemoveGrid filter
image-filtering/config/image_filtering_params.yaml	Changed default filter to remove_grid and updated topic subscriptions and encodings for testing

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-02-16T20:06:59Z

+    if (original.empty()) {
+        spdlog::error("RemoveGrid: input image is empty");
+        return;
+    }
+
+    if (original.type() != CV_8UC3) {
+        spdlog::error("RemoveGrid: expected CV_8UC3 image");
+        original.copyTo(filtered);
+        return;


The early return on line 41 when the input image is empty leaves the 'filtered' output parameter uninitialized. This could cause undefined behavior if the caller attempts to use the filtered image. Consider either copying the original to filtered (as done in the type check case on line 46), or ensuring the filtered image is initialized to an empty Mat before returning.

Copilot · 2026-02-16T20:06:59Z

+    cv::invertAffineTransform(rotation_matrix, inv_rot);
+
+    cv::warpAffine(rotated_out, filtered, inv_rot, original.size(),
+                   cv::INTER_LINEAR, cv::BORDER_CONSTANT, cv::Scalar(0, 0, 0));


The forward rotation uses INTER_NEAREST interpolation, while the inverse rotation on line 126 uses INTER_LINEAR. This inconsistency could introduce artifacts or alignment issues. For geometric transformations that should be reversible, it's generally better to use the same interpolation method for both forward and inverse operations.

Suggested change

cv::INTER_LINEAR, cv::BORDER_CONSTANT, cv::Scalar(0, 0, 0));

cv::INTER_NEAREST, cv::BORDER_CONSTANT, cv::Scalar(0, 0, 0));

Copilot · 2026-02-16T20:06:59Z

+    sub_topic: "/cam/image_color"
    pub_topic: "/filtered_image"
-    input_encoding: "mono8"
-    output_encoding: "mono8"
+    input_encoding: "rgb8"
+    output_encoding: "rgb8"

    filter_params:
-      filter_type: "otsu"
+      filter_type: "remove_grid"


The default configuration has been changed to use the new 'remove_grid' filter and different topics ('/cam/image_color' instead of '/fls_publisher/display_mono', 'rgb8' encoding instead of 'mono8'). This changes the default behavior of the image filtering node. Consider whether these changes should be committed as defaults, or if they should be reverted to maintain the original default behavior, with the new filter being opt-in via configuration override.

Copilot · 2026-02-16T20:06:59Z

+                      cv::Mat& filtered) const override;
+
+   private:
+    RemoveGridParams params_;


The member variable is named 'params_' instead of following the codebase convention of 'filter_params_' used by other filters (e.g., Flip, Overlap, MedianBinary, BinaryThreshold, OtsuSegmentation). This inconsistency should be corrected to maintain naming convention uniformity across filter implementations.

Copilot · 2026-02-16T20:06:59Z

+    // Center crop
+    // ----------------------------
+    int crop_w = std::min(params_.width, rotated.cols);
+    int crop_h = std::min(params_.height, rotated.rows);


If the configured width or height exceeds the rotated image dimensions, the crop parameters are safely clamped using std::min. However, there's no warning or logging to inform the user that their configured dimensions were not achievable. Consider adding a debug or warning log when the crop dimensions need to be clamped, so users understand their configuration may not be having the intended effect.

Suggested change

int crop_h = std::min(params_.height, rotated.rows);

int crop_h = std::min(params_.height, rotated.rows);

if (crop_w != params_.width || crop_h != params_.height) {

spdlog::warn(

"RemoveGrid: requested crop size (width={}, height={}) exceeds "

"rotated image size (width={}, height={}); clamping to "

"(width={}, height={})",

params_.width,

params_.height,

rotated.cols,

rotated.rows,

crop_w,

crop_h);

}

replaced the binary thresholding with apply_fixed_threshold to avoid duplicating code

kluge7 · 2026-03-12T18:25:14Z

+    if (original.empty()) {
+        spdlog::error("RemoveGrid: input image is empty");
+        return;
+    }


Add unit test for this

kluge7 · 2026-03-12T18:25:27Z

+    if (original.type() != CV_8UC3) {
+        spdlog::error("RemoveGrid: expected CV_8UC3 image");
+        original.copyTo(filtered);
+        return;


Add unit test for this

kluge7 · 2026-03-12T18:27:22Z

+    dilated.col(dilated.cols - 1).setTo(0);
+
+    if (cv::countNonZero(dilated) == 0) {
+        original.copyTo(filtered);   // or original.copyTo(filtered);


I dont understand the comment

AntTra · 2026-03-18T16:50:36Z

+        inpaint_radius: 1.0
+        rotation: 0
+        height: 400
+        width: 400


AntTra

please dont h8 me

AntTra · 2026-03-18T16:51:53Z

+
+/////////////////////////////
+// Remove grid filter
+/////////////////////////////


gpt? gemini? claude?

AntTra · 2026-03-18T16:53:09Z

+
+struct RemoveGridParams {
+    double threshold_green;
+    int threshold_binary;


threshold_binary was double in config. why dis int

AntTra · 2026-03-18T16:54:13Z

+
+    // ----------------------------
+    // Rotate directly into cropped output
+    // ----------------------------


AntTra · 2026-03-18T16:55:17Z

+    int crop_w = std::min(params_.width, original.cols);
+    int crop_h = std::min(params_.height, original.rows);
+
+    if (crop_w != params_.width || crop_h != params_.height) {


if (crop_w > params_.width || crop_h > params_.height) { ?

AntTra · 2026-03-18T16:59:53Z

Not a problem but you use a lot of redundant variables in this file, a lot of them are not necessary and you would shorten the code

AntTra · 2026-03-18T17:07:51Z

+    dilated.col(dilated.cols - 1).setTo(0);
+
+    if (cv::countNonZero(dilated) == 0) {
+        original.copyTo(filtered);  // or original.copyTo(filtered);


AntTra · 2026-03-18T17:08:19Z

+    // ----------------------------
+    // Inpaint grid
+    // ----------------------------


useless gpt comment

AntTra · 2026-03-18T17:08:47Z

+    // ----------------------------
+    // Binary threshold (on cropped ROI)


useless gpt

AntTra · 2026-03-18T17:11:00Z

+
+    cv::Mat thresh_bgr;
+    cv::cvtColor(thresh_gray, thresh_bgr, cv::COLOR_GRAY2BGR);


Is this necessary for the ArUco detector to work? This should be optional (converting is a slow operation, and this is mainly for debugging)

AntTra · 2026-03-18T17:11:11Z

+    // ----------------------------
+    // Undo rotation & merge (using M)


sophia-mina and others added 2 commits January 19, 2026 20:50

filter to remove grid (without box)

5af770a

feat: implemented bounded filtering box in image_processing and added…

dc8e520

… necessary variables

sophia-mina linked an issue Feb 11, 2026 that may be closed by this pull request

[TASK] Detecting an ArUco marker obscured by a grid #14

Closed

7 tasks

renamed a variable

9cc2cf3

kluge7 added 2 commits February 16, 2026 19:23

Merge remote-tracking branch 'origin' into aruco-grid-filter

04ce549

chore: add configs values for remove_grid

bb11065

kluge7 requested review from kluge7 February 16, 2026 18:42

kluge7 reviewed Feb 16, 2026

View reviewed changes

added pre-commit changes

abb087f

kluge7 requested a review from Copilot February 16, 2026 20:01

Copilot started reviewing on behalf of kluge7 February 16, 2026 20:01 View session

Copilot AI reviewed Feb 16, 2026

View reviewed changes

fixed the rotation of the image

ed1e516

replaced the binary thresholding with apply_fixed_threshold to avoid duplicating code

sophia-mina requested a review from AntTra March 2, 2026 16:37

sophia-mina and others added 2 commits March 2, 2026 19:17

Merge branch 'main' into aruco-grid-filter

5f6fa43

optimizing the filter (still not finished)

c76c93d

kluge7 requested changes Mar 12, 2026

View reviewed changes

AntTra reviewed Mar 18, 2026

View reviewed changes

AntTra requested changes Mar 18, 2026

View reviewed changes

sophia-mina added 2 commits March 26, 2026 11:54

fixed comments

556b003

chore: apply pre-commit changes

93a3bdb

sophia-mina force-pushed the aruco-grid-filter branch from 4779026 to 93a3bdb Compare March 26, 2026 10:56

kluge7 merged commit 896ae87 into main Mar 26, 2026
5 checks passed

kluge7 deleted the aruco-grid-filter branch March 26, 2026 11:01

	cv::INTER_LINEAR, cv::BORDER_CONSTANT, cv::Scalar(0, 0, 0));
	cv::INTER_NEAREST, cv::BORDER_CONSTANT, cv::Scalar(0, 0, 0));

-    int crop_h = std::min(params_.height, rotated.rows);
+    int crop_h = std::min(params_.height, rotated.rows);
+    if (crop_w != params_.width || crop_h != params_.height) {
+        spdlog::warn(
+            "RemoveGrid: requested crop size (width={}, height={}) exceeds "
+            "rotated image size (width={}, height={}); clamping to "
+            "(width={}, height={})",
+            params_.width,
+            params_.height,
+            rotated.cols,
+            rotated.rows,
+            crop_w,
+            crop_h);
+    }

		// ----------------------------
		// Binary threshold (on cropped ROI)


		cv::Mat thresh_bgr;
		cv::cvtColor(thresh_gray, thresh_bgr, cv::COLOR_GRAY2BGR);

		// ----------------------------
		// Undo rotation & merge (using M)

Conversation

sophia-mina commented Feb 11, 2026

ArUco grid filter

Uh oh!

codecov Bot commented Feb 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

kluge7 left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Feb 16, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 16, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 16, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 16, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 16, 2026

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AntTra left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

codecov Bot commented Feb 11, 2026 •

edited

Loading