Band gap convergence

Concerns issues with computing quasiparticle corrections to the DFT eigenvalues - i.e., the self-energy within the GW approximation (-g n), or considering the Hartree-Fock exchange only (-x)

Moderators: Davide Sangalli, andrea.ferretti, myrta gruning, andrea marini, Daniele Varsano

Laura Caputo
Posts: 88
Joined: Sun Apr 11, 2021 3:02 pm

Band gap convergence

Post by Laura Caputo » Mon Jan 17, 2022 11:30 am

Hello,

I am converging the empty bands for calculating the GW correction to the band gap. I have done this convergence study for different models obtaining convergence almost always when having >140/200 empty bands (using the BG terminator). However, a particular model is giving me problems when converging the bands (it seem to converge quite slowly). I have converged the response block size and bands at the same time, and at fixed NGsBlkXp the energy at different bands goes like this

100 4,22
150 3,7018
200 3,7112
250 3,7096
300 3,7128
350 3,7172
450 3,7172

The system is particularly big and I've also seen that it converges very slowly with k-point sampling so I need to make a compromise between the feasibility of the calculation and accuracy. In fact, considering 350 bands would request a lot of memory and I'm not sure I could finish the calculation (I've finished the calculation with 200 bands). What value could be safe to take to calculate the GW corrections?
Laura Caputo
Ph.D. Student
Université Catholique de Louvain
https://uclouvain.be/fr/repertoires/laura.caputo

User avatar
Daniele Varsano
Posts: 3773
Joined: Tue Mar 17, 2009 2:23 pm
Contact:

Re: Band gap convergence

Post by Daniele Varsano » Mon Jan 17, 2022 2:42 pm

Dear Laura,
from the number you posted, it seems to me that 200 bands are enough to have a converged gap with more than reasonable accuracy.
If you have memory problems (I presume in the calculation of the screening) these can be mitigated by choosing an optimal parallelization strategy for memory distribution, ue assigning CPU on the bands only balancing the number of CPU in valence and conduction bands according to their proportion (Nvalnce/cpu_v \sim Nconduct/cpu_c)

Best,
Daniele
Dr. Daniele Varsano
S3-CNR Institute of Nanoscience and MaX Center, Italy
MaX - Materials design at the Exascale
http://www.nano.cnr.it
http://www.max-centre.eu/

Laura Caputo
Posts: 88
Joined: Sun Apr 11, 2021 3:02 pm

Re: Band gap convergence

Post by Laura Caputo » Mon Jan 17, 2022 2:58 pm

Dear Daniele,

Thanks. I had the doubt about the convergence since when I plotted the lines, 200 bands didn't seem converged but looking at the numbers directly could be easier.
As said, I've calculated the gap with 200 bands, without having too many memory problems. However, as you said, I have these kind of problems when calculating the screening with higher parameters. Just to fully understand your suggestion, in my input files I usually select something like:

PAR_def_mode= "balanced" # [PARALLEL] Default distribution mode ("balanced"/"memory"/"workload")
X_and_IO_CPU= "1 10 12" # [PARALLEL] CPUs for each role
X_and_IO_ROLEs= "k c v" # [PARALLEL] CPUs roles (q,g,k,c,v)
X_and_IO_nCPU_LinAlg_INV=-1 # [PARALLEL] CPUs for Linear Algebra (if -1 it is automatically set)
X_Threads=1 # [OPENMP/X] Number of threads for response functions
DIP_CPU= "1 10 12" # [PARALLEL] CPUs for each role
DIP_ROLEs= "k c v" # [PARALLEL] CPUs roles (k,c,v)
DIP_Threads=1 # [OPENMP/X] Number of threads for dipoles
SE_CPU= "1 1 97" # [PARALLEL] CPUs for each role
SE_ROLEs= "q qp b" # [PARALLEL] CPUs roles (q,qp,b)

for OpenMP/MPI calculation, depending on how demanding the calculation is. If I understood correctly, I should modify the PARALLEL options depending on how many filled/empty bands I have in proportion, right?
Laura Caputo
Ph.D. Student
Université Catholique de Louvain
https://uclouvain.be/fr/repertoires/laura.caputo

User avatar
Daniele Varsano
Posts: 3773
Joined: Tue Mar 17, 2009 2:23 pm
Contact:

Re: Band gap convergence

Post by Daniele Varsano » Mon Jan 17, 2022 4:35 pm

Dear Laura,
If I understood correctly, I should modify the PARALLEL options depending on how many filled/empty bands I have in proportion, right?
Yes,
these are the parameter you should change depending on the filled/empty bands proportion.

Code: Select all

X_and_IO_CPU= "1 10 12"
X_and_IO_ROLEs= "k c v" 
and it is desirable to assign more CPUs to the conduction as they are much numerous.

Best,

Daniele
Dr. Daniele Varsano
S3-CNR Institute of Nanoscience and MaX Center, Italy
MaX - Materials design at the Exascale
http://www.nano.cnr.it
http://www.max-centre.eu/

Laura Caputo
Posts: 88
Joined: Sun Apr 11, 2021 3:02 pm

Re: Band gap convergence

Post by Laura Caputo » Tue Jan 18, 2022 11:56 am

Dear Daniele,
After your suggestion I have been trying to run Yambo in parallel with different options, for example:
X_and_IO_CPU= "1 6 4" # [PARALLEL] CPUs for each role
X_and_IO_ROLEs= "k c v" # [PARALLEL] CPUs roles (q,g,k,c,v)
X_and_IO_nCPU_LinAlg_INV=-1 # [PARALLEL] CPUs for Linear Algebra (if -1 it is automatically set)
X_Threads=12 # [OPENMP/X] Number of threads for response functions
DIP_CPU= "1 6 4" # [PARALLEL] CPUs for each role
DIP_ROLEs= "k c v" # [PARALLEL] CPUs roles (k,c,v)
DIP_Threads=12 # [OPENMP/X] Number of threads for dipoles
SE_CPU= "1 1 24" # [PARALLEL] CPUs for each role
SE_ROLEs= "q qp b" # [PARALLEL] CPUs roles (q,qp,b)
SE_Threads=12 # [OPENMP/GW] Number of threads for self-energy

and in the file generated in the LOG, there isn't 'switching to default' so I guess the parallelization was working. However, to do each step, it takes more than 10h... am I doing something incorrectly or is just the calculation very demanding? On the cluster that I'm using, I can't use more than 24CPU but it seems that using more threads doesn't speed up the calculation a lot, is it true?
Laura Caputo
Ph.D. Student
Université Catholique de Louvain
https://uclouvain.be/fr/repertoires/laura.caputo

User avatar
Daniele Varsano
Posts: 3773
Joined: Tue Mar 17, 2009 2:23 pm
Contact:

Re: Band gap convergence

Post by Daniele Varsano » Tue Jan 18, 2022 12:05 pm

Dear Laura,
You can use threads, but I would not use more than 8/16, as you are using already 24 MPI I do not know how much would be performant the hyperthreading.
In order to improve memory distribution, you can probably put more CPU on conduction and reduce the valence ones. Note this will distribute the memory in an efficient wat but it is not
guaranteed that it is the most performant strategy.

Best,
Daniele
Dr. Daniele Varsano
S3-CNR Institute of Nanoscience and MaX Center, Italy
MaX - Materials design at the Exascale
http://www.nano.cnr.it
http://www.max-centre.eu/

Laura Caputo
Posts: 88
Joined: Sun Apr 11, 2021 3:02 pm

Re: Band gap convergence

Post by Laura Caputo » Tue Jan 18, 2022 12:08 pm

Dear Daniele,

Thanks for the suggestion. So, if also in this case the calculation is slow, the only way for speeding it up is to use more MPI, right?
Laura Caputo
Ph.D. Student
Université Catholique de Louvain
https://uclouvain.be/fr/repertoires/laura.caputo

User avatar
Daniele Varsano
Posts: 3773
Joined: Tue Mar 17, 2009 2:23 pm
Contact:

Re: Band gap convergence

Post by Daniele Varsano » Tue Jan 18, 2022 12:12 pm

Dear Laura,
Using more MPI of course it will improve the speed of the calculation. Another way is to see if there are parameters you can reduce without losing accuracy.
Finally, it also depends on which part of the code is the more time consuming (computation of X0, X, dipoles, GW summation etc...).

Best,
Daniele
Dr. Daniele Varsano
S3-CNR Institute of Nanoscience and MaX Center, Italy
MaX - Materials design at the Exascale
http://www.nano.cnr.it
http://www.max-centre.eu/

Laura Caputo
Posts: 88
Joined: Sun Apr 11, 2021 3:02 pm

Re: Band gap convergence

Post by Laura Caputo » Tue Jan 18, 2022 12:13 pm

Dear Daniele,

Thank you very much for the clarification.

Have a nice day,
Laura
Laura Caputo
Ph.D. Student
Université Catholique de Louvain
https://uclouvain.be/fr/repertoires/laura.caputo

Laura Caputo
Posts: 88
Joined: Sun Apr 11, 2021 3:02 pm

Re: Band gap convergence

Post by Laura Caputo » Thu Jan 20, 2022 6:04 pm

Dear Daniele,

Reviewing my calculations, I've noticed that I have converged the band gap using Gamma point instead of the point where the direct band gap is. Could the convergence be strongly affected by this?
Laura Caputo
Ph.D. Student
Université Catholique de Louvain
https://uclouvain.be/fr/repertoires/laura.caputo

Post Reply