Reproducible cuDSS allocation failure during global_positioning on 4262-image fisheye dataset
I am seeing a reproducible cuDSS allocation failure during
global_positioning on a 4262-image fisheye dataset.
This looks similar to #215 /
#191, but I’m posting full reproducible
details from my setup.
Environment
- OS: Ubuntu 24.04.3 LTS
- CPU: AMD Ryzen 9 5900XT (32 logical CPUs)
- RAM: 64 GB
- GPU: NVIDIA GeForce RTX 3090 (24 GB VRAM)
- NVIDIA driver: 570.211.01
Build / Versions
- GLOMAP binary:
/home/ubu/dev/src/glomap_cuda120/install/bin/glomap
glomap --help shows: This version was compiled with CUDA!-
Local glomap repo revision used for this build:
14e9104c2e60bc3f68fbc8b645b0d79a62f97f7a
-
COLMAP in the same pipeline:
COLMAP 3.13.0 (Commit d69fddb2 on 2026-02-15 with CUDA)
Dataset
- 4262 images (2-camera rig), total image size approximately 2.4–2.6 GB
- Image dimensions:
3840x3840
Feature extraction settings
SiftExtraction.max_image_size=3200SiftExtraction.max_num_features=16384SiftExtraction.peak_threshold=0.0033- GPU feature extraction enabled
Extracted features (from logs)
- min: 10426
- max: 21180
- mean: 15638.68 per image
Command used
/home/ubu/dev/src/glomap_cuda120/run_glomap_120.sh mapper \
--database_path /.../database/database.db \
--image_path /.../distorted/images \
--output_path /.../distorted/sparse \
--GlobalPositioning.use_gpu 1 \
--GlobalPositioning.gpu_index -1 \
--BundleAdjustment.use_gpu 1 \
--BundleAdjustment.gpu_index -1 \
--BundleAdjustment.optimize_intrinsics 1 \
--BundleAdjustment.optimize_principal_point 1 \
--BundleAdjustment.optimize_rig_poses 1 \
--BundleAdjustment.max_num_iterations 300 \
--skip_pruning 0
I also reproduced the same failure with --skip_pruning 1.
Error output
During Running global positioning ...:
Linear solver fatal error: cudssExecute with CUDSS_PHASE_FACTORIZATION failed Got error: CUDSS_STATUS_ALLOC_FAILED
Terminating: Linear solver failed due to unrecoverable non-numeric causes.
Ceres Solver Report ... Termination: FAILURE
Important behavior
Even after this solver failure, mapper continues and exports a COLMAP model.
In that case, the model quality can be invalid or corrupted
(for example, extremely large reprojection error in analyzer output).
If I force CPU solvers
(GlobalPositioning.use_gpu=0, BundleAdjustment.use_gpu=0),
the reconstruction completes with normal stats.
Questions
-
Is this expected behavior when cuDSS runs out of memory
(i.e. continue execution and still export a model),
or should
mapper fail hard in this case?
-
Are there any recommended flags or settings to reduce cuDSS memory pressure
in
global_positioning for datasets of this scale?
Reproducible cuDSS allocation failure during
global_positioningon 4262-image fisheye datasetI am seeing a reproducible cuDSS allocation failure during
global_positioningon a 4262-image fisheye dataset.This looks similar to #215 / #191, but I’m posting full reproducible details from my setup.
Environment
Build / Versions
/home/ubu/dev/src/glomap_cuda120/install/bin/glomapglomap --helpshows:This version was compiled with CUDA!14e9104c2e60bc3f68fbc8b645b0d79a62f97f7aCOLMAP 3.13.0 (Commit d69fddb2 on 2026-02-15 with CUDA)Dataset
3840x3840Feature extraction settings
SiftExtraction.max_image_size=3200SiftExtraction.max_num_features=16384SiftExtraction.peak_threshold=0.0033Extracted features (from logs)
Command used
I also reproduced the same failure with
--skip_pruning 1.Error output
During Running global positioning ...:
Important behavior
Even after this solver failure,
mappercontinues and exports a COLMAP model.In that case, the model quality can be invalid or corrupted (for example, extremely large reprojection error in analyzer output).
If I force CPU solvers (
GlobalPositioning.use_gpu=0,BundleAdjustment.use_gpu=0), the reconstruction completes with normal stats.Questions
mapperfail hard in this case?global_positioningfor datasets of this scale?