How to calculate RAM for BSE

[haseebphysics1]

Dear Yamboers,

For the following parameters, what's the formula to calculate the ram/cpu for the BSE? I remember, this was answered somewhere in the forum but I could not find one. The dimension comes from Nk x NV x NC = dim, right?

# BSK|Identifier :5843
# |Dimension :155520
# |Bands :153 - 170

Now should I square dim, and then multiply by 16 and then further divide by 1024 thrice to get ans back in GBs? was that in this way?
If yes, then why should I square the "dim" matrix?

Regards,

[claudio]

Dear Haseeb

if I'm correct the amount of memory you need should be:

Memory (Gb) = 155520 x 155520 x 4 byte / 1024^3 = 90 Gb

I considered that you compiled yambo in single precision, therefore
a single-precision floating number is large 4 byte (32 bits) and a Gigabiyte is 1024^3 bytes.

https://en.wikipedia.org/wiki/Single-pr ... int_format

https://en.wikipedia.org/wiki/Gigabyte

In general, from personal experience, you can diagonalize a BSE matrix of maximum 35.000 size,
while with iterative diagonalization you can arrive up to 130.000 or something more.

best
Claudio

[Daniele Varsano]

Dear Haseeb,

then why should I square the "dim" matrix?

These are the number of elements of a squared matrix of dimension "dim".

a single-precision floating number is large 4 byte

This is a complex matrix so I think that there is a factor 2.

Daniele

[haseebphysics1]

Dear claudio and Daniele, thanks for the replies!

a single-precision floating number is large 4 byte (32 bits)

1: So, can you please suggest, how I can check my installation type? 32 or 64 bits? And for the same size of matrix 64-bit installation takes more memory?

2: In BSE, is it better to parallelize over k instead of eh or t to save memory, right?

3: And, in one of the calculations, when I changed the number of bands in BSE, the BSEkernel did not seem to be calculated again (I saw the log file), although in the output file dimension was updated! But I was worried that has it computed the right results or not? I removed all the ndb.BS files in the SAVE before starting the new calculation.



Regards,

[Daniele Varsano]

Dear Haseeb,
1. the yambo compilation by default is in single-precision unless you do not include the "--enable-dp " option in the configure.
2. You have many k points so parallelizing over k should be the best strategy
3. Your calculation seems successful, the kernel has been written to the disk:

Code: Select all

[WR./BSEBnds_153-170_low_range_0-3//ndb.BS_Q1_CPU_0]

Actually I do not know why there is not the estimation time of the BSE kernel construction in the log files, anyway, the code spent more than 9h in the kernel building:

Code: Select all

X T_space Kernel :     9h23m

Note that you are using very low parameters in the kernel:

Code: Select all

  |RL vectors    [exchange]: 183
|RL vectors [correlation]:  75

In general, note you are dealing with a massive calculation (160 occupied bands and 976 k points), such calculation needs many resources. I do not know exactly what system you are dealing with (some complex bismuth tungsten oxide) , but maybe you can inspect if it is not possible to reduce the unit cell and if there are symmetries (in the present setup you have no symmetry) that for some reason are not taken into account by QE, this will help a lot.

Best,
Daniele

[haseebphysics1]

Dear Daniele, thanks for this useful insight.

In general, note you are dealing with a massive calculation (160 occupied bands and 976 k points), such calculation needs many resources. I do not know exactly what system you are dealing with (some complex bismuth tungsten oxide) , but maybe you can inspect if it is not possible to reduce the unit cell and if there are symmetries (in the present setup you have no symmetry) that for some reason are not taken into account by QE, this will help a lot.

However, I want to ask that those 976 k-points are for the IBZ, while full BZ comprises 1944 (18 x 6 x 18 k-mesh). I calculated the dimension of the BSEmatrix with k-points of the full BZ k-points and which is also coming out to be consistent with the dimension mentioned in the report file!

Now, the only thing to reduce the size of the memory is to lower the bands or k-mesh size. Right? How looking for some symmetry will help me here?

Thanks,

[claudio]

Dear Haseeb,

unfortunately, symmetries do not help with the size.

The size of the BSE matrix depends by the number of k-points in the BZ,
while symmetries help in the construction of the BSE matrix elements.

best
Claudio

[jasonhan0710]

Hi Daniele,

Does the memory usage means the memory size allocated for each core? For example, if yambo reports that

Code: Select all

    BSK|Identifier              : 2358
       |Dimension               :  53248
       |Bands                   :  75   95

which means I need 21.125GB (53248^2*8/1024^3) memory. If my cluster has 192GB RAM and 64 cores, should I reduce the number of cores in BSE calculation?

Thank you!

Best,
Jason

[Daniele Varsano]

Dear Jason,

yes, you should use fewer cores per node.

Best,
Daniele

[Fadil]

Dear all,

My bse calculation terminates without any error in the 'diago solver' section.
According to the Ram calculation below,;
28224^2*8/1024^3 = 6GB
Approximately 6 GB of RAM is required per core.

There are 256 GB RAM on my system, I am using 16 cores.
You can find the report file in the attachment.
Do you have any idea what is the problem?