Compile Yambo 5.3 with HPC SDK 25.1 Trouble

[huy]

Hi everyone,
Thanks so much in advance for your attention.
I am a newbie in DFT, especially in Yambo so I have no experience to express my trouble formally, please let me know if something unclear to you.
Now, I attempt to compile Yambo 5.3 using HPC-SDK 25.1, Cuda 12.6 in my WSL Ubuntu 24.04, runs in Windows 11 with my core i7-14700 equipped with VGA card Nvidia GTX 4070 Ti Super, information below:

Code: Select all

 nvidia-smi
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 572.16                 Driver Version: 572.16         CUDA Version: 12.8     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                  Driver-Model | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 4070 ...  WDDM  |   00000000:01:00.0  On |                  N/A |
|  0%   42C    P8             13W /  285W |     762MiB /  16376MiB |      4%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

I use this instruction https://wiki.yambo-code.eu/wiki/index.p ... n_compiler to compile my yambo, from hpc sdk 25.1/cuda 12.6; openmpi-4.1.7 and added these lines to my .bash file

Code: Select all

NVARCH=`uname -s`_`uname -m`; export NVARCH
NVCOMPILERS=/opt/nvidia/hpc_sdk; export NVCOMPILERS
MANPATH=$MANPATH:$NVCOMPILERS/$NVARCH/25.1/compilers/man; export MANPATH
PATH=$NVCOMPILERS/$NVARCH/25.1/compilers/bin:/usr/openmpi-4.1.7/bin:$PATH; export PATH
export LD_LIBRARY_PATH=$NVCOMPILERS/$NVARCH/25.1/compilers/lib:/usr/openmpi-4.1.7/lib:$NVCOMPILERS/$NVARCH/25.1/cuda/lib64:$LD_LIBRARY_PATH

Double check the installing of Hpc skd and openmpi-4.1.7 show

Code: Select all

nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2024 NVIDIA Corporation
Built on Tue_Oct_29_23:50:19_PDT_2024
Cuda compilation tools, release 12.6, V12.6.85
Build cuda_12.6.r12.6/compiler.35059454_0

and

Code: Select all

mpirun --version
mpirun (Open MPI) 4.1.7

Report bugs to http://www.open-mpi.org/community/help/

I compiled yambo-5.3 with the configuration

Code: Select all

sudo ./configure MPIFC=/usr/openmpi-4.1.7/bin/mpif90 FC=/opt/nvidia/hpc_sdk/Linux_x86_64/25.1/compilers/bin/nvfortran F77=/opt/nvidia/hpc_sdk/Linux_x86_64/25.1/compilers/bin/nvfortran CPP="cpp -E"  FPP="/opt/nvidia/hpc_sdk/Linux_x86_64/25.1/compilers/bin/nvfortran -Mpreprocess -E"  PFC=/usr/openmpi-4.1.7/bin/mpif90 F90SUFFIX=".f90" --enable-memory-profile  --enable-open-mp --enable-par-linalg  --enable-hdf5-par-io  --enable-slepc-linal --enable-cuda-fortran --with-cuda-libs=/opt/nvidia/hpc_sdk/Linux_x86_64/25.1/cuda/lib64 --with-cuda-libdir=/opt/nvidia/hpc_sdk/Linux_x86_64/25.1/cuda/lib64 --with-cuda-incs=/opt/nvidia/hpc_sdk/Linux_x86_64/25.1/cuda/include --with-cuda-includedir=/opt/nvidia/hpc_sdk/Linux_x86_64/25.1/cuda --with-cuda-path=/opt/nvidia/hpc_sdk/Linux_x86_64/25.1/cuda/bin --with-cuda-cc=89 --with-cuda-runtime=12.6

but it seem not work to me

Code: Select all

...
[src/tools] lib_Y_tools.a (lib)
make[2]: *** [/home/ygpu/yambo-5.3/config/mk/local/rules.mk:15: mod_pars.o] Error 2
        [driver] yambo (setup)
yambo linking failed. Check log/compile_yambo.log
make[1]: *** [config/mk/global/actions/compile_yambo.mk:43: yambo] Error 1
yambo build failed
make: *** [Makefile:54: all] Error 1

I attached the configure result, config log and compile_yambo log below to convenient.
I tried with configure without sudo, but it doesn't work as well. Could someone give me advice to solve this trouble!

[Nicola Spallanzani]

Dear Huy,
It looks like you're having issues with your MPI configuration. Specifically, from the config.log, it appears that the configure script is unable to find the mpicc wrapper, which is essential for compiling certain libraries, including NetCDF. The final error in the Yambo compilation occurs because it fails to locate the NetCDF module.

A possible quick fix is to explicitly set the path to the mpicc wrapper using the appropriate environment variable:

Code: Select all

MPICC=/usr/openmpi-4.1.7/bin/mpicc

However, I recommend using the OpenMPI provided by the Nvidia HPC SDK you just installed. It works well and is properly configured.

Additionally, I suggest avoiding manually setting environment variables for compilers and MPI wrappers. The safest approach is to use the modulefiles provided by Nvidia, located at: /opt/nvidia/hpc_sdk/modulefiles

Before using modulefiles, you need to install the module management tool:

Code: Select all

sudo apt install environment-modules
echo "source /etc/profile.d/modules.sh" >> ~/.bashrc
echo "module use /opt/nvidia/hpc_sdk/modulefiles" >> ~/.bashrc
source ~/.bashrc

The commands above only need to be used the first time.

Now, you can load the Nvidia SDK module, which will correctly set up all environment variables for both compilers and MPI:

Code: Select all

module load nvhpc/25.1

You can verify the setup with:

Code: Select all

$ mpirun --version
mpirun (Open MPI) 4.1.7rc1

Report bugs to http://www.open-mpi.org/community/help/

$ mpif90 --version

nvfortran 25.1-0 64-bit target on x86-64 Linux -tp haswell 
NVIDIA Compilers and Tools
Copyright (c) 2025, NVIDIA CORPORATION & AFFILIATES.  All rights reserved.

Finally, you can compile Yambo by running the configure script as follows:

Code: Select all

make distclean

./configure MPIFC=mpif90 MPICC=mpicc \
    FC=nvfortran F77=nvfortran CC=nvc CPP="cpp -E"  FPP="nvfortran -Mpreprocess -E"  F90SUFFIX=".f90" \
    --enable-memory-profile --enable-time-profile  --enable-open-mp --enable-par-linalg --enable-hdf5-par-io  --enable-slepc-linalg \
    --enable-cuda-fortran --with-cuda-cc=89 --with-cuda-runtime=12.6

make -j core

You don’t really need to specify any additional CUDA flags.

I will update the wiki page with these instructions soon.
Best,
Nicola

[Nicola Spallanzani]

I've noticed another potential issue that could cause problems when running the code.

Your system has the CUDA 572.16 driver installed, which I assume came with CUDA 12.8. According to the official documentation (https://docs.nvidia.com/deploy/cuda-compatibility/), CUDA only supports forward compatibility. This means that with the CUDA 12.8 driver installed, you can only use CUDA 12.8 or newer at runtime.

This could be problematic since NVHPC 25.1 includes CUDA 12.6. However, Yambo might still work because it mainly relies on common CUDA features.

If you experience crashes while running Yambo, you'll need to recompile it using CUDA 12.8. Unfortunately, Yambo doesn't currently provide an option to do this directly. To address this, I've prepared a patch for Yambo 5.3.0's configure script, which will be included in future releases.

Code: Select all

wget https://raw.githubusercontent.com/nicspalla/my-repo/refs/heads/develop/packages/yambo/cuda_runtime.patch
patch -u configure -i cuda_runtime.patch

Once you've applied the patch, you can run the same configure command as before without the flag --cuda-runtime, but this time, make sure to set the environment variable NVHPC_CUDA_HOME to the correct installation path for CUDA 12.8.

[huy]

Dear Nicola Spallanzani
Thank you so much for your advice.
Unfortunately, I have to say the error remains. I follow 100% your instruction in this post and also the newest update in yambo-wiki after re-install Ubuntu-24-04 LTS on WSL 2.0. After re-install Ubuntu I've just run

Code: Select all

sudo apt-get -yqq update
sudo apt-get upgrade -y
sudo apt-get install -y build-essential

After this, all steps following instruction with configure

Code: Select all

./configure MPIFC=mpif90 MPICC=mpicc FC=nvfortran F77=nvfortran CPP="cpp -E"  FPP="nvfortran -Mpreprocess -E" F90SUFFIX=".f90" --enable-memory-profile  --enable-open-mp --enable-par-linalg  --enable-hdf5-par-io  --enable-slepc-linalg  --enable-cuda-fortran --with-cuda-cc=89 --with-cuda-runtime=12.6

And the error

Code: Select all

....
        [src/tools] lib_Y_tools.a (lib)
make[2]: *** [/home/ygpu/yambo-5.3.0/config/mk/local/rules.mk:15: mod_pars.o] Error 2
        [driver] yambo (setup)
yambo linking failed. Check log/compile_yambo.log
make[1]: *** [config/mk/global/actions/compile_yambo.mk:43: yambo] Error 1
yambo build failed
make: *** [Makefile:54: all] Error 1

and as log files below.
Noted that, I done this set-up without seeing your newest reply, since I will try again with CUDA 12.8 as your new instruction.
Thanks again for your help.
Sincerely,
Huy

[Nicola Spallanzani]

I see that there is still an issue with the compilation of the NetCDF library. I will now try the procedure myself from scratch in a Docker container with the specifications you described. I'll let you know as soon as possible.

[Nicola Spallanzani]

I think I have reproduced the issue. I noticed that my installation of NetCDF and LibXC was failing because some Autotools and zlib packages were missing.

So, try installing these packages:

Code: Select all

sudo apt install automake autoconf libtool zlib1g-dev

Then start the installation from scratch by running "make distclean". To be safe, also remove the existing installation of the HDF5 libraries by deleting the entire "lib/external/nvfortran" folder.

You can also add this flag to Yambo's configure line to redirect the installation path of the libraries:

Code: Select all

--with-extlibs-path=$HOME/extlibs

Best,
Nicola

[huy]

Dear Nicola Spallanzani,
Thank so much for your instruction.
Following your previous advice of install Autotools and zlib packages, I success compile yambo-5.3.0 by NVFortran.
Note that, before carry out installing, I downgrade my Nvidia Driver to 560.35 with CUDA 12.6, then reinstall Ubuntu-24.04 on WSL 2.0.

Code: Select all

nvidia-smi
Tue Mar 11 21:15:34 2025
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 560.35.02              Driver Version: 560.94         CUDA Version: 12.6     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA GeForce RTX 4070 ...    On  |   00000000:01:00.0  On |                  N/A |
|  0%   44C    P8             13W /  285W |     588MiB /  16376MiB |     19%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

Code: Select all

yambo --version

This is yambo - MPI+GPU+CUDA+CUDAF+OpenMP+SLK+SLEPC+HDF5_MPI_IO - Ver. 5.3.0 Revision 23927 Hash 1730222ea

Thank you again for your prompt help.
Sincerely,
Huy