RIM step hangs when calculating HF bandgap

[arb83@cam.ac.uk]

Dear Yambo community forum

I am calculating HF bandgaps of a 2D structure. I am using the RIM method. I have been converging the HF bandgap with respect to 'RandQpts' and 'RandGvec'. For 'RandGvec', when I increase this number above 10000RL the calculation hangs at the 'RIM integrals' section. I believe that this is something to do with overuse of memory. My question is whether different parallelisation strategies are likely to resolve this problem, or whether there are any other methods to reduce the memory use of individual cores?

In my calculation I am using 160 cores (spread over either 5 or 10 nodes) and, with 'RandGvec' a little less than 10000RL the RIM calculation takes around 5 minutes.

Any suggestions are much appreciated.

With best wishes,

Alan

[Daniele Varsano]

Dear Alan,
please sign your post with your name and affiliation, this is a rule of the forum. You can do once for all by filling the signature in your user profile.
Honestly I never found the need to use such a large value of RandGvec, and the procedure is meant to be used for few tenth or few hundreds of G vectors, as the integral of the Coulomb potential calculated with a Monte Carlo procedure hardly differs from the \sum_qi 1/|qi+G|^2 Delta q_i for large G, that is what yambo does for all the G larger than RandGvec. Of course be sure that the stochastic integrals are accurate and this is obtained by using few millions of q points (1-3 millions is generally enough).

Anyway, if you are dealing with a 2D structure, the suggestion is to use a truncated coulomb potential. If your unit cell is orthorhombic the ws cutoff is preferred:

Code: Select all

CUTGeo= "ws z"
CUTwsGvec= 0.700000    

In case it is not orthorhombic you need to use the

Code: Select all

CUTGeo= "box z"                   # [CUT] Coulomb Cutoff geometry: box/cylinder/sphere/ws X/Y/Z/XY..
% CUTBox
 0.000000 | 0.000000 | zcut |        # [CUT] [au] Box sides
%

with zcut slightly shorter than your cell size in that direction, and in this case you need the RIM to build up the modified coulomb potential, but as said before using at most few hundreds of G vector should be enough.

Best,
Daniele

[arb83@cam.ac.uk]

Dear Daniele

Apologies, I have posted my name and affiliation now.

I was unclear on what I meant by a 2D material ('quasi-2D is probably more correct). In my example I have no vacuum, but have two relatively thick layers (with two interfaces between these layers) and k-point grid of 2 6 1 (no k-point shift so gamma point is included). As I have no vacuum a truncated coulomb potential will not work (potentially I shouldn't be using RIM either in this case?).

I fully agree that I shouldn't require RandGvec to be so large (total FFTGvecs is ~300000). However, I am observing changes on the order of 0.1eV in the hartree-fock bandgap even if I increase RandGvec from 8000 to 10000.

One point I have noted is that RIM seems to hang because it is out of memory. This occurs in the 'Momentum loop' step. When I run the calculation on 160 nodes the 'Momentum loop' step only seems to run on 8 nodes, hence my original question about parallelisation options.

Any suggestions would be much appreciated.

With best wishes,

Alan

[Daniele Varsano]

Dear Alan,

(potentially I shouldn't be using RIM either in this case?).

Yes, you can avoid to use the RIM, but the RIM should accelerate convergence wrt k point sampling.

However, I am observing changes on the order of 0.1eV in the hartree-fock bandgap even if I increase RandGvec from 8000 to 10000.

That's suspicious, as I wrote before the RIM it is not meant to be used for large Gvectors as it should not have any effect.

When I run the calculation on 160 nodes the 'Momentum loop' step only seems to run on 8 nodes

Possibly you should check your parallelisation strategy for the "q" role.

Anyway, let me tell you again that RIM is not meant for such large number of G vector,s in the report you can check the ratio between integrals calculated via MC and by simple summation for each q at G=0 (first BZ), if at the edge of the BZ the ratio is nearly 1, then the RIM is not needed for more than 1 G vector, for inhomogeneous cells sometimes it is needed to take some more G into account. I'm afraid what you are observing it is not physical.

Best,
Daniele

[arb83@cam.ac.uk]

Dear Daniele

Thanks for your reply. I'm slightly confused by the first part of your answer - does RIM just give faster convergence wrt k-points, or is it a necessity to use due to instability in the non-periodic direction? You suggest the former in your answer but my understanding was the latter (based on e.g. http://www.yambo-code.org/wiki/index.ph ... al_systems).

Best wishes,

Alan

[Daniele Varsano]

Dear Alan,

does RIM just give faster convergence wrt k-points, or is it a necessity to use due to instability in the non-periodic direction?

Both of them. Anyway, even in case you do not use the RIM, the divergence is regularised by considering the integral around q=0 analytically in a spherical domain around gamma, while integrals for all the other q are treated as sum_i v(qi)delta_omega(qi). This could be not accurate for small grids while the denser is the gird the more accurate is the value. Anyway the divergence is removed. When using the RIM, the integral are performed via Monte Carlo technique for all the q points of the Bz and also outside the BZ when including more than 1 Gvec.

Best,
Daniele

[arb83@cam.ac.uk]

Dear Daniele

That makes more sense, thank you.

I'm still having the same problem that when I increase RandGvec above 1Ry the Hartree-Fock bandgap changes dramatically. I'm attaching an example here (RandGvec=1Ry and RandGvec=8Ry) alongside the r_setup file in case you're able to see something. I have shortened the files as they were too large to upload fully. I fully agree what I'm seeing is unphysical but I don't understand what is going wrong in my calculations. Any suggestions would be much appreciated.

All the best,

Alan

[arb83@cam.ac.uk]

Apologies, can't seem to upload all relevant files in one post...

[arb83@cam.ac.uk]

Final file.

[Daniele Varsano]

Dear Alan,

for what I can see you have a very small BZ and indeed you need to add more G vecs in the random integration.
This can bee seen by the ratio between integrals calculated with and wo RIM:

Code: Select all

 Summary of Coulomb integrals for non-metallic bands |Q|[au] RIM/Bare:

  Q [1]:0.1000E-40.8322 * Q [5]:  0.05569  0.64606
  Q [2]:  0.09007  0.82336 * Q [6]: 0.105354 0.759931
  Q [3]: 0.180145 0.719085 * Q [7]: 0.187945 0.713837
  Q [4]: 0.270218 0.700439 * Q [8]: 0.273396 0.702232

At the edge of the BZ Q(8) the ration is about 0.7 which is rather small.

My suggestion here is:
1) Consider more than 1M q-points, at least 3 millions, this comes nearly for free and provide you better accuracy.
2) Note you are not considering 1 or 8 Ry, but 1 or 8 RL, so it seems 8RL are not enough, I suggest you to look at the convergence adding RL e.g.
1-10-50-100-200-400

and see if you are approaching convergence regularly or some strange behaviour appears, I say so because the algorithm have been never used for thousand of G vectors and I cannot exclude that some instabilities can appears.

Best,
Daniele