Compiling Yambo with gpu

[ezekiel]

Hi,
Could you guide me on how to compile YAMBO on HPC successfully? I have the following modules:

chpc/parallel_studio_xe/2019u5
chpc/parallel_studio_xe/2020u1
chpc/cuda/10.0
chpc/cuda/10.1
chpc/nvhpc/22.2
chpc/pgi/19.10

Thank you

[ezekiel]

The architecture we have for our GPUs
cuda_arch=70

We have NVIDIA V100's
https://wiki.chpc.ac.za/guide:gpu

[sanoj]

The architecture we have for our GPUs
cuda_arch=70

We have NVIDIA V100's
https://wiki.chpc.ac.za/guide:gpu

Hi Ezekiel,

I used the following to successfully compile. I am using Nvidia_nvhpc_24.7 (installed in /opt/nvidia/hpc_sdk/ ). I have Nvidia RTX 5000 with cc=75. Add your cuda path and cuda runtime.

./configure FC=nvfortran F77=nvfortran MPIFC=mpif90 MPIF77=mpif90 'CPP=cpp -E' 'FPP=nvfortran -Mpreprocess -E' CC=nvc --with-cuda-path=/opt/nvidia/hpc_sdk/Linux_x86_64/2024/cuda --with-cuda-runtime=12.5 --with-cuda-cc=75 --enable-open-mp --enable-cuda-fortran --enable-par-linalg --enable-dp --enable-time-profile --enable-msgs-comps

[Nicola Spallanzani]

Dear Ezekiel,
looking at the wiki page you shared, I noticed that there should be many more CUDA modules available than the ones you listed. It might be the same for the NVHPC modules as well, which is the only one you need (though version 22.2 is a bit outdated).

My advice is to try compiling by loading the module "chpc/nvhpc/22.2" and running this configuration command (like suggested by Sanoj):

Code: Select all

module load chpc/nvhpc/22.2
wget https://github.com/yambo-code/yambo/archive/refs/tags/5.3.0.tar.gz -O yambo-5.3.0.tar.gz
tar zxvf yambo-5.3.0.tar.gz
cd yambo-5.3.0
./configure FC=nvfortran F77=nvfortran MPIFC=mpif90 MPIF77=mpif90 'CPP=cpp -E' 'FPP=nvfortran -Mpreprocess -E' CC=nvc --with-cuda-runtime=11.6 --with-cuda-cc=70 --enable-open-mp --enable-cuda-fortran --enable-par-linalg --enable-time-profile --enable-memory-profile
make -j core

If this installation doesn't work, I recommend downloading a more recent version of NVHPC and installing it yourself. It's quite easy, just follow the instructions.
https://developer.nvidia.com/hpc-sdk-releases
After that, load the "nvhpc" module from the modulefiles directory of your NVHPC installation and try running the same configure command again. The only thing you might need to adjust is specifying a different CUDA runtime version. You can find the available versions in the cuda directory of your NVHPC installation.

Best regards,
Nicola

[ezekiel]

Hi Nicola,
When doing make -j core, I got this error in the log file: NVFORTRAN-F-0000-Internal compiler error. interf:new_symbol, symbol not found 636 (mod_wrapper_omp.f90: 16)
NVFORTRAN/x86-64 Linux 22.2-0: compilation aborted
How do I resolve this?

Ezekiel

[Charles_Crosby]

I've added nvhpc-24.7. It is the last one that will work on Lengau, due to the CentOS 7 operating system. I will also this morning add a custom-compiled openmpi-4.1.7 with cuda support. Once done I will share the module names here.

Charles

[Charles_Crosby]

OK, to add to that:

module load chpc/compmech/openmpi/4.1.7-nvhpc-24.7

will give you the no-mpi version of nvhpc-24.7 as well as openmpi-4.1.7.

Charles

[Charles_Crosby]

Thank you Nicola, your advice has worked. Ezekiel, you can test:

module load chpc/compmech/yambo/5.3.0_with_cuda

Charles

[ezekiel]

Okay. I will do the test. Thank you.