Yambo 4.0.1 rev89 GW calculation with SOC

[caowd]

Dear Developers

I am doing GW calculations by yambo 4.0.1 with SOC included.

I have got the following error message after some ndb.dipoles_fragment_i files are generated.
--------------------------------------------------------------------------------------------------------------------------
[ganbo110:30734] *** Process received signal ***
[ganbo110:30734] Signal: Segmentation fault (11)
[ganbo110:30734] Signal code: Invalid permissions (2)
[ganbo110:30734] Failing at address: 0x7f186e642440
[ganbo110:30734] [ 0] /lib64/libpthread.so.0(+0xf710) [0x7f1871812710]
[ganbo110:30734] [ 1] /home_b/caowd/yambo-4.0.1-rev.89/bin/yambo(matrix_operate_mp_mat_c2r_+0x18d) [0x5dc5ed]
[ganbo110:30734] [ 2] /home_b/caowd/yambo-4.0.1-rev.89/bin/yambo(io_dipoles_+0x5ef) [0x520a6f]
[ganbo110:30734] [ 3] /home_b/caowd/yambo-4.0.1-rev.89/bin/yambo(dipole_driver_+0xb7b) [0x49e65b]
[ganbo110:30734] [ 4] /home_b/caowd/yambo-4.0.1-rev.89/bin/yambo(x_os_+0x4924) [0x4a6b04]
[ganbo110:30734] [ 5] /home_b/caowd/yambo-4.0.1-rev.89/bin/yambo(x_em1_+0x9ff) [0x4a1fbf]
[ganbo110:30734] [ 6] /home_b/caowd/yambo-4.0.1-rev.89/bin/yambo(yambo_driver_+0x873) [0x43e843]
[ganbo110:30734] [ 7] /home_b/caowd/yambo-4.0.1-rev.89/bin/yambo(main+0xd33) [0x43ba33]
[ganbo110:30734] [ 8] /lib64/libc.so.6(__libc_start_main+0xfd) [0x7f187148dd5d]
[ganbo110:30734] [ 9] /home_b/caowd/yambo-4.0.1-rev.89/bin/yambo() [0x43ac39]
[ganbo110:30734] *** End of error message ***
------------------------------------------------------------------------------------------------------------------
But if I do a serial calculation, there is no error but it takes too much time.
So I guess it may be related to the allocation of the CPUs in the parallel calculation. Could you please help me on this?

my configure file is
----------------------
MKL_DIR="/opt/intel/composer_xe_2013_sp1.2.144/mkl/lib/intel64/"

./configure \
FC=ifort \
CC=icc \
--with-blas-libs="-L$MKL_DIR -lmkl_intel_lp64 -lmkl_sequential -lmkl_core" \
--with-lapack-libs="-L$MKL_DIR -lmkl_intel_lp64 -lmkl_sequential -lmkl_core" \
--with-fft-libs="-mkl" \
--with-netcdf-path="/opt/netcdf/4.3.2/intel_14.0" \
--with-hdf5-path="/opt/netcdf/4.3.2/intel_14.0" \
--enable-open-mp \
--enable-openmpi \
--enable-time-profile
-----------------------

my GW input file is

----------------------------------
em1d # [R Xd] Dynamical Inverse Dielectric Matrix
gw0 # [R GW] GoWo Quasiparticle energy levels
ppa # [R Xp] Plasmon Pole Approximation
HF_and_locXC # [R XX] Hartree-Fock Self-energy and Vxc
ElecTemp = 0.000001
X_all_q_CPU= "1,4,1,1" # [PARALLEL] CPUs for each role
X_all_q_ROLEs= "q,k,c,v" # [PARALLEL] CPUs roles (q,k,c,v)
X_all_q_nCPU_invert=1 # [PARALLEL] CPUs for matrix inversion
SE_CPU= "4,1,1" # [PARALLEL] CPUs for each role
SE_ROLEs= "q,qp,b" # [PARALLEL] CPUs roles (q,qp,b)
EXXRLvcs= 57223 RL # [XX] Exchange RL components
Chimod= "hartree" # [X] IP/Hartree/ALDA/LRC/BSfxc
% QpntsRXp
1 | 168 | # [Xp] Transferred momenta
%
% BndsRnXp
1 | 24 | # [Xp] Polarization function bands
%
NGsBlkXp= 1 RL # [Xp] Response block size
% LongDrXp
0.000000 | 0.000000 | 1.000000 | # [Xp] [cc] Electric Field
%
PPAPntXp= 27.21138 eV # [Xp] PPA imaginary energy
% GbndRnge
1 | 24 | # [GW] G[W] bands range
%
GDamping= 0.10000 eV # [GW] G[W] damping
dScStep= 0.10000 eV # [GW] Energy step to evalute Z factors
GTermKind= "none" # [GW] GW terminator ("none","BG" Bruneval-Gonze,"BRS" Berger-Reining-Sottile)
DysSolver= "n" # [GW] Dyson Equation solver (`n`,`s`,`g`)
%QPkrange # [GW] QP generalized Kpoint/Band indices
1|168| 13 | 20|
%
%QPerange # [GW] QP generalized Kpoint/Energy indices
1|168| 0.0|-1.0|
%

----------------------------------

my job submitted script file is

--------------------------------------
#!/bin/bash
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=4
#SBATCH --time=2000
#SBATCH --job-name=NaBi_scf_nc
#SBATCH --output=out.log_
#SBATCH --error=err.out.log_
#SBATCH --exclusive
#SBATCH --partition=12_cores
source ~/.bashrc

srun /home_b/caowd/yambo-4.0.1-rev.89/bin/yambo -F gw.in -J gw -C gw_out
----------------------------------------------------------------

[caowd]

ADDITION:

When the total number of CPUs is 4 and the setting in the input file is
---------------------------------------------------------------------------------------
X_all_q_CPU= "1,4,1,1" # [PARALLEL] CPUs for each role
X_all_q_ROLEs= "q,k,c,v" # [PARALLEL] CPUs roles (q,k,c,v)
X_all_q_nCPU_invert=0 # [PARALLEL] CPUs for matrix inversion
SE_CPU= "4,1,1" # [PARALLEL] CPUs for each role
SE_ROLEs= "q,qp,b" # [PARALLEL] CPUs roles (q,qp,b)
----------------------------------------------------------------------------------------

The error message in the l- file is
-----------------------------------------------------------------------------------------------------
P0003: [ERROR] STOP signal received while in :[05] Dynamic Dielectric Matrix (PPA)
P0003: [ERROR][NetCDF] NetCDF: Unknown file format
-----------------------------------------------------------------------------------------------------

Could you please help me on this? Thank you very much.

[andrea.ferretti]

Dear Wen-Dong Cao,

besides the Netcdf error, the seg-fault you got is a rather serious error which should not happen under normal conditions.
Some questions to try to understand the origin of the problem:

* did you check that your compiled version of the code is working properly ?
* does is work in serial and in parallel on some test cases ?

so far, being your problem showing up in parallel, I would investigate whether you really need the
--enable-openmpi flag.

here the story is the following:
if you are using a openmpi implementation of MPI (or any other implementation based on that), you *must* set
--enable-openmpi, otherwise you should not.
This may break the parallel implementation of yambo (in future or stable releases this condition will hopefully be removed, but at the moment
is there and must be carefully checked).

The above not being the problem (though I hope it is), I would then look into the netcdf linking
(e.g. you could leave the netcdf flags during configuration empty in order to use the internal version of the library)

Andrea

[caowd]

Dear andrea.ferretti

Thank you very much.

Those problems are indeed related to the Netcdf library and solved by using the internal one.

[andrea.ferretti]

Hi,

glad the problem has been solved and thanks for letting us know.
As a curiosity, were you running on top of databases produced by p2y/yambo using netcdf 4.3.2 ?
or were the DBs produced by a different version of the code/netcdf, etc ?

take care
Andrea

[caowd]

Hi

The way I used the previous netcdf library is by loading the module "netcdf/4.3.2_intel_14.0" and setting the corresponding path in the configuration.
So the version may be 4.3.2

Best Wishes.