Rotated bounding box NMS implementation for GPU

[zy1git]

Summary:
Enabled rotated box NMS on CUDA, building on the CPU implementation from #9450. The CUDA kernel is adapted from Detectron2's nms_rotated_cuda_kernel, using single_box_iou_rotated instead of devIoU for IoU computation, and storing 5 values per box in shared memory instead of 4. The result gathering reuses TorchVision's existing gather_keep_from_mask GPU kernel, avoiding the GPU→CPU→GPU transfer that Detectron2's implementation requires.

Test Plan:

Added device parametrization (cpu_and_cuda()) to all rotated NMS tests.

Run pytest test/test_ops.py::TestNMS -v -k "rotated" All 110 tests pass locally (CPU and CUDA).

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/vision/9478

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

• [CI[B200] Smoke test encounters CUDA Unknown error for dgxb200-03 and dgxb200-04

❌ 2 New Failures

As of commit 2d22de0 with merge base d7400a3 ():

NEW FAILURES - The following jobs have failed:

• Tests / unittests-macos (3.10, macos-m1-stable) / macos-job (gh)
  test/test_transforms_v2.py::TestErase::test_transform_image_correctness[4-cpu-dtype1-value-random]
• Tests / unittests-macos (3.14, macos-m1-stable) / macos-job (gh)
  test/test_transforms_v2.py::TestErase::test_transform_image_correctness[4-cpu-dtype1-value-random]

This comment was automatically generated by Dr. CI and updates every 15 minutes.