RIM step hangs when calculating HF bandgap

Concerns issues with computing quasiparticle corrections to the DFT eigenvalues - i.e., the self-energy within the GW approximation (-g n), or considering the Hartree-Fock exchange only (-x)

Moderators: Davide Sangalli, andrea.ferretti, myrta gruning, andrea marini, Daniele Varsano

arb83@cam.ac.uk
Posts: 98
Joined: Thu Jul 02, 2020 3:56 pm

Re: RIM step hangs when calculating HF bandgap

Post by arb83@cam.ac.uk » Thu Jul 30, 2020 6:28 pm

Hi Daniele

So I've tried increasing to 12million G vectors and the BZ Q(8) ratio remains roughly constant (0.7028 compared to 0.7022 for 1million G vectors). I've also tried increasing 'RandGvec', I get very little change again:

RandGvec = 1RL; Q(8) ratio = 0.7022
RandGvec = 9RL; Q(8) ratio = 0.7022
RandGvec = 1000RL; Q(8) ratio = 0.7021
RandGvec = 12000RL; Q(8) ratio = 0.7025.

Do you think I should be increasing these variables much further or is there another issue?

A couple of other things I have noticed:

- as in the r-RandGvec files, about 70-80% of the points are falling outside the sphere radius (this continues when I increase the number of sampling points);
- when I increase RandGvec, the value of 'Gamma point sphere radius' in the RIM integral doesn't change (I'm not sure if it should?).

All the best,

Alan
Alan Bowman
University of Cambridge

User avatar
Daniele Varsano
Posts: 3816
Joined: Tue Mar 17, 2009 2:23 pm
Contact:

Re: RIM step hangs when calculating HF bandgap

Post by Daniele Varsano » Fri Jul 31, 2020 9:10 am

Dear Alan,
probably I was not clear enough:
When you add RandGvec you apply the Monte Carlo integration outside the BZ, it is normal that Q(8) ratio does not change because you looking always at the same integral and it tells you need to go outside the BZ: What you are looking at is statistical fluctuation of the MC integration on the same integral and it is correct it does not change. It just tells you that you are using enough RandQpoints.
- as in the r-RandGvec files, about 70-80% of the points are falling outside the sphere radius (this continues when I increase the number of sampling points);
Ok this is normal, it is how work the MC integration.
- when I increase RandGvec, the value of 'Gamma point sphere radius' in the RIM integral doesn't change (I'm not sure if it should?).
It does not have to change.

What I have suggested is to increase the RandGvec and look at the convergence of the <HF> value.

Best,
Daniele
Dr. Daniele Varsano
S3-CNR Institute of Nanoscience and MaX Center, Italy
MaX - Materials design at the Exascale
http://www.nano.cnr.it
http://www.max-centre.eu/

arb83@cam.ac.uk
Posts: 98
Joined: Thu Jul 02, 2020 3:56 pm

Re: RIM step hangs when calculating HF bandgap

Post by arb83@cam.ac.uk » Fri Jul 31, 2020 10:26 am

Dear Daniele

Thanks for the clarification. This brings me back to my original question - when I increase RandGvec past 12000RL the RIM integrals no longer run due to memory issues. If I run the calculation on 160 cores then I think the integrals part are only running on 8 cores. Is a different parallelisation strategy the best way to solve this or is there something else I should be doing?

Best wishes,

Alan
Alan Bowman
University of Cambridge

User avatar
Daniele Varsano
Posts: 3816
Joined: Tue Mar 17, 2009 2:23 pm
Contact:

Re: RIM step hangs when calculating HF bandgap

Post by Daniele Varsano » Fri Jul 31, 2020 10:31 am

Dear Alan,
I can only repeat my answer.
The algorithm it is not meant for such large number of Gvector outside the BZ.
If it does not converge after few hundreds of G vector I suspect that something odd is happening, as the code was never used for such a large number of G vector, this is because in line of principle the MC integral and the one done my simple summation should be the same.
What I suggested you is to look at the convergence increasing the number of G vectors from few tenth to some hundreds (which is the usual way the technique is used) and see the behaviour of your <HF> matrix elements in order if you have a monotonic convergence or strange and suspicious behaviour.
If I run the calculation on 160 cores then I think the integrals part are only running on 8 cores. Is a different parallelisation strategy the best way to solve this or is there something else I should be doing?
This is because the algorithm it is parallelized over q points and you have 8 q points. The parallelisation has been not extended to G vectors as it never happened to use such large number of Gvectors.

Best,
Daniele
Dr. Daniele Varsano
S3-CNR Institute of Nanoscience and MaX Center, Italy
MaX - Materials design at the Exascale
http://www.nano.cnr.it
http://www.max-centre.eu/

arb83@cam.ac.uk
Posts: 98
Joined: Thu Jul 02, 2020 3:56 pm

Re: RIM step hangs when calculating HF bandgap

Post by arb83@cam.ac.uk » Fri Jul 31, 2020 11:17 am

Dear Daniele

That's really helpful, thank you. I guess I will just have to stop at 12000RL in that case. The HF convergence is monotonic and is close to converged at 12000RL, but it sounds like I won't be able to increase beyond this value (I'm using high memory nodes to even do this calculation!).

With best wishes,

Alan
Alan Bowman
University of Cambridge

arb83@cam.ac.uk
Posts: 98
Joined: Thu Jul 02, 2020 3:56 pm

Re: RIM step hangs when calculating HF bandgap

Post by arb83@cam.ac.uk » Thu Mar 11, 2021 11:15 am

Dear Daniele

I hope you are well. I have a quick question - when using a truncated coulomb potential in the z direction (i.e. 'box Z'), do I need to converge for the size of the vacuum in the z direction (i.e. many DFT calculations with different vacuum sizes, setting the coulomb cutoff to just less than the vacuum in each case and look at the HF energy)? And if so will the vacuum size at HF convergence also be the correct size for GW calculations?

To give a little detail, I am carrying out a GW convergence on a 2D system. I find everything converges relatively easily except NGsBlkXp, which is still not converged at a value of 11Ry (when using BG termination for both Xterm and Gterm). I am wondering whether this could be related to a sharp cut in the coulomb potential at the box edge. I should also note my simulation cell (in the z direction) goes from z=0 to z=76. I have set the coulomb cutoff to z=70, and there are atoms present from z=0 to z=59.

Apologies, I couldn't find this on the yambo wiki.

All the best,

Alan
Alan Bowman
University of Cambridge

User avatar
Daniele Varsano
Posts: 3816
Joined: Tue Mar 17, 2009 2:23 pm
Contact:

Re: RIM step hangs when calculating HF bandgap

Post by Daniele Varsano » Thu Mar 11, 2021 12:05 pm

Dear Alan,
as a rule of thumb you need enough vacuum such that your interaction extends all over your system but does not allow interaction among replica. In order to fulfil this condition you need an amount of vacuum a bit larger then your system extension (note the condition depends on the dimensionality of your system, for 2D should be L < 2D , where is your supercell size and D the extension of the system, see e.g Jarvis et a. Phys. Rev. B 56, 14972 1997 and Rozzi, Varsano et al. PHYSICAL REVIEW B 73, 205119 2006).
Using this setting you can avoid extensive convergence tests wrt the supercell size.
Note that the coulomb cutoff size should be set just a bit smaller of the cell size.
If you have an orthorhombic cell I strongly suggest you to use:

Code: Select all

CUTGeo= "ws z"
CUTwsGvec= 0.700000

in this case you do not have to take care of the cutoff size.
will the vacuum size at HF convergence also be the correct size for GW calculations?
In my experience, the correlation part is more sensible to the volume than the HF part in absolute value, but the condition above should be ok for both.
I am wondering whether this could be related to a sharp cut in the coulomb potential at the box edge.
I do not think so, when using a cutoff epsm1 is forced to be 1 for q=0 and becomes sharp with respect the bulk case, and this reflects in a harder convergence wrt the k point sampling and not the matrix dimension.
My suggestion anyway is to use BG termination for the GTerm and avoid it for the Xterm.

Best,

Daniele
Dr. Daniele Varsano
S3-CNR Institute of Nanoscience and MaX Center, Italy
MaX - Materials design at the Exascale
http://www.nano.cnr.it
http://www.max-centre.eu/

arb83@cam.ac.uk
Posts: 98
Joined: Thu Jul 02, 2020 3:56 pm

Re: RIM step hangs when calculating HF bandgap

Post by arb83@cam.ac.uk » Thu Mar 11, 2021 7:32 pm

Dear Daniele

Many thanks for the comprehensive answer. I have one follow up question: assume I have a cubic system with vacuum in the z direction, which extends from z=0 to z=50. Let us say (following your advice) there are atoms for a length of 25 in this direction. Should these atoms be positioned to span from z=0 to z=25, or from z=12.5 to z=37.5 within the unit cell? In the former case, if I put the coulomb cutoff at z=45, I don't understand how this does not result in a very sharp cutoff for the atoms positioned at z=0 (when considering in the negative z direction).

Also, it would be good to know why not to use BGterm for the Xterm?

All the best,

Alan
Alan Bowman
University of Cambridge

User avatar
Daniele Varsano
Posts: 3816
Joined: Tue Mar 17, 2009 2:23 pm
Contact:

Re: RIM step hangs when calculating HF bandgap

Post by Daniele Varsano » Thu Mar 11, 2021 10:11 pm

Dear Alan,
Should these atoms be positioned to span from z=0 to z=25, or from z=12.5 to z=37.5 within the unit cell?
It doesn't matter, you have periodic boundary condition and the potential is defined in function of |r-r'| and not (r-r'), what matters is the distance and not the relative position. In other words, for each point r, it goes to zero after a defined distance independently from where r is placed in the unit cell.
I don't understand how this does not result in a very sharp cutoff
It is sharp in real space, and you need to sum up a large amount of Fourier components to represent it, here instead you use the Fourier components you need to converge the screening.
Also, it would be good to know why not to use BGterm for the Xterm?
I found that even if it accelerate the convergence wrt number of bands, at a fixed number of bands it is computationally heavy, so in may cases it is more convenient to use more bands and avoid it.

Best,
Daniele
Dr. Daniele Varsano
S3-CNR Institute of Nanoscience and MaX Center, Italy
MaX - Materials design at the Exascale
http://www.nano.cnr.it
http://www.max-centre.eu/

arb83@cam.ac.uk
Posts: 98
Joined: Thu Jul 02, 2020 3:56 pm

Re: RIM step hangs when calculating HF bandgap

Post by arb83@cam.ac.uk » Fri Mar 12, 2021 11:17 am

Dear Daniele

That's great, thank you very much!

All the best,

Alan
Alan Bowman
University of Cambridge

Post Reply