LAMMPS compilation failed: error "undefined reference to cudaGetDriverEntryPointByVersion" in libc10_cuda.so

[Demosunner]

why I can not compile lam file after using below Instruction
I copy the error information, I need help
How can I defuse this question ?

Problem Description

When compiling LAMMPS (with the ML-METATOMIC package enabled), the linking stage failed and the error message undefined reference to cudaGetDriverEntryPointByVersion was reported. It seems that there is a conflict between the CUDA library in the conda environment and the system's CUDA library.

cmake -DPKG_ML-METATOMIC=ON \
      -DLAMMPS_INSTALL_RPATH=ON \
      -DCMAKE_PREFIX_PATH="$CMAKE_PREFIX_PATH" \
      ../cmake


CMake Deprecation Warning at CMakeLists.txt:22 (cmake_policy):
  The OLD behavior for policy CMP0109 will be removed from a future version
  of CMake.

-- Running check for auto-generated files from make-based build system
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE  
-- Caffe2: Found protobuf with new-style protobuf targets.
-- Caffe2: Protobuf version 31.1.0
-- Found CUDAToolkit: /usr/include (found version "12.0.140") 
-- The CUDA compiler identification is NVIDIA 12.0.140
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - done
-- Check for working CUDA compiler: /usr/bin/nvcc - skipped
-- Detecting CUDA compile features
-- Detecting CUDA compile features - done
-- PyTorch: CUDA detected: 12.0.140
-- PyTorch: CUDA nvcc is: /usr/bin/nvcc
-- PyTorch: CUDA toolkit directory: 
-- PyTorch: Header version is: 12.0
-- Found Python: /usr/bin/python3 (found version "3.12.3") found components: Interpreter 
CMake Warning at /home/mnmt/anaconda3/envs/lammps-metatomic/share/cmake/Caffe2/public/cuda.cmake:116 (message):
  Failed to compute shorthash for libnvrtc.so
Call Stack (most recent call first):
  /home/mnmt/anaconda3/envs/lammps-metatomic/share/cmake/Caffe2/Caffe2Config.cmake:86 (include)
  /home/mnmt/anaconda3/envs/lammps-metatomic/share/cmake/Torch/TorchConfig.cmake:68 (find_package)
  Modules/Packages/ML-METATOMIC.cmake:12 (find_package)
  CMakeLists.txt:594 (include)



-- Found metatensor_torch: /home/mnmt/anaconda3/envs/lammps-metatomic/lib/libmetatensor_torch.so (Required is at least version "0.8.0") 
-- Caffe2: Found protobuf with new-style protobuf targets.
-- Caffe2: Protobuf version 31.1.0
CMake Warning at /home/mnmt/anaconda3/envs/lammps-metatomic/share/cmake/Torch/TorchConfig.cmake:22 (message):
  static library kineto_LIBRARY-NOTFOUND not found.
Call Stack (most recent call first):
  /home/mnmt/anaconda3/envs/lammps-metatomic/share/cmake/Torch/TorchConfig.cmake:125 (append_torchlib_if_found)
  /home/mnmt/anaconda3/envs/lammps-metatomic/lib/cmake/metatomic_torch/metatomic_torch-config.cmake:13 (find_package)
  Modules/Packages/ML-METATOMIC.cmake:54 (find_package)
  CMakeLists.txt:594 (include)

-- Generating style headers...
-- Generating package headers...
-- Generating lmpinstalledpkgs.h...
-- Found Python3: /usr/bin/python3 (found version "3.12.3") found components: Interpreter 
-- The following tools and libraries have been found and configured:
 * Git
 * MPI
 * OpenMP
 * absl
 * Threads
 * CUDAToolkit
 * Python
 * metatensor (required version >= 0.1.18)
 * metatensor_torch (required version >= 0.8.0)
 * utf8_range
 * Caffe2
 * Torch (required version >= 2.9)
 * metatomic_torch (required version >= 0.1.7)
 * Python3

-- <<< Build configuration >>>
   LAMMPS Version:   2025.9.10.99 
   Operating System: Linux Ubuntu" 24.04
   CMake Version:    3.28.3
   Build type:       RelWithDebInfo
   Install path:     /home/mnmt/.local
   Generator:        Unix Makefiles using /usr/bin/gmake
-- Enabled packages: ML-METATOMIC
-- <<< Compilers and Flags: >>>
-- C++ Compiler:     /usr/bin/c++
      Type:          GNU
      Version:       13.3.0
      C++ Standard:  17
      C++ Flags:     -O2 -g -DNDEBUG
      Defines:       LAMMPS_SMALLBIG;LAMMPS_MEMALIGN=64;LAMMPS_OMP_COMPAT=4;LAMMPS_GZIP;LAMMPS_FFMPEG
-- C compiler:       /usr/bin/cc
      Type:          GNU
      Version:       13.3.0
      C Flags:       -O2 -g -DNDEBUG
-- <<< Linker flags: >>>
-- Executable name:  lmp
-- Static library flags:    
-- <<< MPI flags >>>
-- MPI_defines:      MPICH_SKIP_MPICXX;OMPI_SKIP_MPICXX;_MPICC_H
-- MPI includes:     /home/mnmt/anaconda3/envs/lammps-metatomic/include
-- MPI libraries:    /home/mnmt/anaconda3/envs/lammps-metatomic/lib/libmpi.so;
-- Configuring done (6.6s)
CMake Warning at CMakeLists.txt:250 (add_executable):
  Cannot generate a safe runtime search path for target lmp because files in
  some directories may conflict with libraries in implicit directories:

    runtime library [libgomp.so.1] in /usr/lib/gcc/x86_64-linux-gnu/13 may be hidden by files in:
      /home/mnmt/anaconda3/envs/lammps-metatomic/lib
    runtime library [libcudart.so.12] in /usr/lib/x86_64-linux-gnu may be hidden by files in:
      /home/mnmt/anaconda3/envs/lammps-metatomic/lib

  Some of these libraries may not be found correctly.

-- Generating done (0.1s)
-- Build files have been written to: /home/mnmt/software/lammps/lammps-metatomic/build

error position

mnmt@HP-Workstation:~/software/lammps/lammps-metatomic/build$ cmake --build . --parallel 4 
[ 98%] Built target lammps
[ 98%] Building CXX object CMakeFiles/lmp.dir/home/mnmt/software/lammps/lammps-metatomic/src/main.cpp.o
[100%] Linking CXX executable lmp
/usr/bin/ld: /home/mnmt/anaconda3/envs/lammps-metatomic/lib/libc10_cuda.so: undefined reference to `cudaGetDriverEntryPointByVersion@libcudart.so.12'
collect2: error: ld returned 1 exit status
gmake[2]: *** [CMakeFiles/lmp.dir/build.make:188：lmp] 错误 1
gmake[1]: *** [CMakeFiles/Makefile2:205：CMakeFiles/lmp.dir/all] 错误 2
gmake: *** [Makefile:136：all] 错误 2

other information

(lammps-metatomic) mnmt@HP-Workstation:~/software/lammps/lammps-metatomic/build$ ls -la /home/mnmt/anaconda3/envs/lammps-metatomic/lib/libcudart*
lrwxrwxrwx 1 mnmt mnmt 48  1月 26 15:52 /home/mnmt/anaconda3/envs/lammps-metatomic/lib/libcudart.so.12 -> ../targets/x86_64-linux/lib/libcudart.so.12.9.79
lrwxrwxrwx 1 mnmt mnmt 48  1月 26 15:52 /home/mnmt/anaconda3/envs/lammps-metatomic/lib/libcudart.so.12.9.79 -> ../targets/x86_64-linux/lib/libcudart.so.12.9.79

[Luthaf]

Hey! This looks like an issue of trying to use a version of libtorch (here libc10_cuda.so, which is part of the C++ API of torch) that was compiled with a different version of cuda than the one available on this machine.

It looks like you are using torch from conda, is this correct? In this case, I would advise also using the cuda libraries from conda. Can you share the output of conda list and conda info?

[Demosunner]

I copy the conda list and conda info output. And I supply conda env error information. Would you give me some advises?

[Demosunner]

Thank you for your reply.

conda list

(lammps-metatomic) mnmt@HP-Workstation:~/software/lammps/lammps-metatomic/build$ conda list
# packages in environment at /home/mnmt/anaconda3/envs/lammps-metatomic:
#
# Name                     Version          Build                                       Channel
_openmp_mutex              4.5              7_kmp_llvm                                  conda-forge
attr                       2.5.2            h39aace5_0                                  conda-forge
cuda-cudart                12.9.79          h5888daf_0                                  conda-forge
cuda-cudart_linux-64       12.9.79          h3f2d84a_0                                  conda-forge
cuda-cupti                 12.9.79          h676940d_1                                  conda-forge
cuda-nvrtc                 12.9.86          hecca717_1                                  conda-forge
cuda-nvtx                  12.9.79          hecca717_1                                  conda-forge
cuda-version               12.9             h4f385c5_3                                  conda-forge
cudnn                      9.10.2.21        hbcb9cd8_0                                  conda-forge
fftw                       3.3.10           mpi_openmpi_h4a81ba8_9                      conda-forge
fmt                        12.1.0           hff5e90c_0                                  conda-forge
gsl                        2.7              he838d99_0                                  conda-forge
icu                        78.2             h33c6efd_0                                  conda-forge
lammps-metatomic           2025.09.10.mta2  cpu_h84a34fa_mpi_openmpi_git.dd4d142_12000  metatensor
libabseil                  20250512.1       cxx17_hba17884_0                            conda-forge
libblas                    3.11.0           5_h5875eb1_mkl                              conda-forge
libcap                     2.77             h3ff7636_0                                  conda-forge
libcblas                   3.11.0           5_hfef963f_mkl                              conda-forge
libcublas                  12.9.1.4         h676940d_1                                  conda-forge
libcudnn                   9.10.2.21        hf7e9902_0                                  conda-forge
libcudnn-dev               9.10.2.21        h58dd1b1_0                                  conda-forge
libcudss                   0.7.1.4          h58dd1b1_0                                  conda-forge
libcufft                   11.4.1.4         hecca717_1                                  conda-forge
libcufile                  1.14.1.1         hbc026e6_1                                  conda-forge
libcurand                  10.3.10.19       h676940d_1                                  conda-forge
libcusolver                11.7.5.82        h676940d_2                                  conda-forge
libcusparse                12.5.10.65       hecca717_2                                  conda-forge
libgcc                     15.2.0           he0feb66_16                                 conda-forge
libgcc-ng                  15.2.0           h69a702a_16                                 conda-forge
libgfortran                15.2.0           h69a702a_16                                 conda-forge
libgfortran-ng             15.2.0           h69a702a_16                                 conda-forge
libgfortran5               15.2.0           h68bc16d_16                                 conda-forge
libhwloc                   2.12.2           default_hafda6a7_1000                       conda-forge
libiconv                   1.18             h3b78370_2                                  conda-forge
liblapack                  3.11.0           5_h5e43f62_mkl                              conda-forge
liblzma                    5.8.2            hb03c661_0                                  conda-forge
libmagma                   2.9.0            ha7672b3_6                                  conda-forge
libmetatensor              0.1.19           h10efecc_0                                  conda-forge
libmetatensor-torch        0.8.3            h36e24b1_0                                  conda-forge
libmetatomic-torch         0.1.7            h36e24b1_0                                  conda-forge
libnl                      3.11.0           hb9d3cd8_0                                  conda-forge
libnvjitlink               12.9.86          hecca717_2                                  conda-forge
libprotobuf                6.31.1           h49aed37_4                                  conda-forge
libstdcxx                  15.2.0           h934c35e_16                                 conda-forge
libstdcxx-ng               15.2.0           hdf11a46_16                                 conda-forge
libsystemd0                258.3            h6569c3e_0                                  conda-forge
libtorch                   2.9.1            cuda129_mkl_hf5f578d_303                    conda-forge
libudev1                   258.3            h6569c3e_0                                  conda-forge
libuv                      1.51.0           hb03c661_1                                  conda-forge
libxml2                    2.15.1           he237659_1                                  conda-forge
libxml2-16                 2.15.1           hca6bf5a_1                                  conda-forge
libzlib                    1.3.1            hb9d3cd8_2                                  conda-forge
llvm-openmp                21.1.8           h4922eb0_0                                  conda-forge
mkl                        2025.3.0         h0e700b2_463                                conda-forge
mpi                        1.0              openmpi                                     conda-forge
nccl                       2.29.2.1         h4d09622_1                                  conda-forge
openmpi                    4.1.6            hc5af2df_101                                conda-forge
plumed-metatomic           2.10.0           mpi_openmpi_hda07afa_1                      metatensor
pybind11-abi               11               hc364b38_1                                  conda-forge
rdma-core                  61.0             h192683f_0                                  conda-forge
sleef                      3.9.0            ha0421bc_0                                  conda-forge
tbb                        2022.3.0         hb700be7_2                                  conda-forge
zlib                       1.3.1            hb9d3cd8_2                                  conda-forge

conda info

(lammps-metatomic) mnmt@HP-Workstation:~/software/lammps/lammps-metatomic/build$ conda info

     active environment : lammps-metatomic
    active env location : /home/mnmt/anaconda3/envs/lammps-metatomic
            shell level : 1
       user config file : /home/mnmt/.condarc
 populated config files : /home/mnmt/anaconda3/.condarc
                          /home/mnmt/anaconda3/condarc.d/anaconda-auth.yml
          conda version : 25.11.1
    conda-build version : 25.11.1
         python version : 3.13.9.final.0
                 solver : libmamba (default)
       virtual packages : __archspec=1=broadwell
                          __conda=25.11.1=0
                          __cuda=13.0=0
                          __glibc=2.39=0
                          __linux=6.14.0=0
                          __unix=0=0
       base environment : /home/mnmt/anaconda3  (writable)
      conda av data dir : /home/mnmt/anaconda3/etc/conda
  conda av metadata url : None
           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /home/mnmt/anaconda3/pkgs
                          /home/mnmt/.conda/pkgs
       envs directories : /home/mnmt/anaconda3/envs
                          /home/mnmt/.conda/envs
               platform : linux-64
             user-agent : conda/25.11.1 requests/2.32.5 CPython/3.13.9 Linux/6.14.0-37-generic ubuntu/24.04.2 glibc/2.39 solver/libmamba conda-libmamba-solver/25.11.0 libmambapy/2.3.2 aau/0.7.5 c/. s/. e/.
                UID:GID : 1000:1000
             netrc file : None
           offline mode : False

nvcc -version

(lammps-metatomic) mnmt@HP-Workstation:~/software/lammps/lammps-metatomic/build$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2023 NVIDIA Corporation
Built on Fri_Jan__6_16:45:21_PST_2023
Cuda compilation tools, release 12.0, V12.0.140
Build cuda_12.0.r12.0/compiler.32267302_0

And I also used torch from conda, but it return me the same error.

when I run

cmake -DPKG_ML-METATOMIC=ON       -DLAMMPS_INSTALL_RPATH=ON       -DCMAKE_PREFIX_PATH="$CMAKE_PREFIX_PATH"       ../cmake

the warning is

CMake Warning at /home/mnmt/anaconda3/envs/lammps-metatomic/share/cmake/Torch/TorchConfig.cmake:22 (message):
  static library kineto_LIBRARY-NOTFOUND not found.
Call Stack (most recent call first):
  /home/mnmt/anaconda3/envs/lammps-metatomic/share/cmake/Torch/TorchConfig.cmake:125 (append_torchlib_if_found)
  /home/mnmt/anaconda3/envs/lammps-metatomic/lib/cmake/metatensor_torch/metatensor_torch-config.cmake:14 (find_package)
  /home/mnmt/anaconda3/envs/lammps-metatomic/lib/cmake/metatomic_torch/metatomic_torch-config.cmake:5 (find_package)
  Modules/Packages/ML-METATOMIC.cmake:100 (find_package)
  CMakeLists.txt:594 (include)

and 
CMake Warning at CMakeLists.txt:250 (add_executable):
  Cannot generate a safe runtime search path for target lmp because files in
  some directories may conflict with libraries in implicit directories:

    runtime library [libgomp.so.1] in /usr/lib/gcc/x86_64-linux-gnu/13 may be hidden by files in:
      /home/mnmt/anaconda3/envs/lammps-metatomic/lib
    runtime library [libcudart.so.12] in /usr/lib/x86_64-linux-gnu may be hidden by files in:
      /home/mnmt/anaconda3/envs/lammps-metatomic/lib

  Some of these libraries may not be found correctly.

after that using

cmake --build . --parallel 4

finally output is

[ 98%] Linking CXX static library liblammps.a
[ 98%] Built target lammps
[ 98%] Building CXX object CMakeFiles/lmp.dir/home/mnmt/software/lammps/lammps-metatomic/src/main.cpp.o
[100%] Linking CXX executable lmp
/usr/bin/ld: /home/mnmt/anaconda3/envs/lammps-metatomic/lib/libc10_cuda.so: undefined reference to `cudaGetDriverEntryPointByVersion@libcudart.so.12'
collect2: error: ld returned 1 exit status
gmake[2]: *** [CMakeFiles/lmp.dir/build.make:188：lmp] 错误 1
gmake[1]: *** [CMakeFiles/Makefile2:205：CMakeFiles/lmp.dir/all] 错误 2
gmake: *** [Makefile:136：all] 错误 2

and I have got a file which name is 'liblammps.a', when the error happened.

[Luthaf]

As a separate point, you have lammps-metatomic already installed from conda, why do you want to also compile the code from source? It looks like you got the CPU build, but we also have CUDE enabled builds, which I'm happy to help you install.

[Luthaf]

Otherwise, you have a libtorch version cuda129_mkl_hf5f578d_303, meaning libtorch wants cuda 12.9. But cmake finds your global nvcc/cuda installation (Found CUDAToolkit: /usr/include (found version "12.0.140") ), which is a lot older.

I can see two different things you could try:

1. updating your global cuda installation to 12.9
2. using the cuda installation from conda to build lammps

For 2, you'll likely need to do something like

conda activate lammps-metatomic
conda install cuda-toolkit

Here you should check that you get version 12.9 of the cuda toolkit.

conda activate lammps-metatomic
export CUDA_HOME=/home/mnmt/anaconda3/envs/lammps-metatomic

cd path/to/lammps
rm -rf build/
mkdir build && cd build

cmake -DPKG_ML-METATOMIC=ON       -DLAMMPS_INSTALL_RPATH=ON       -DCMAKE_PREFIX_PATH="$CMAKE_PREFIX_PATH"       ../cmake

Here you need to check that cmake finds and uses cuda 12.9, before starting a new build.

[Demosunner]

Thanks for your reply. It helps me lot.