printing time and % in LOG, BSE diagonalization

[malwi]

Dear Yambo TEAM,

All the best for 2024, further success in Yambo development and your private life!

I went over the wall-time when making full diagonalization in BSE with almost 1000 k-points.
I did not find any estimate in LOG directory how much it would take.
If such info was available it is always possible to ask administrators for an extension for the job.

The last info in LOG is

Code: Select all

 <56s> P1-ac0137: Loading full BSE kernel |########################################| [100%] 52s(E) 52s(X)
 <01m-00s> P1-ac0137: [06.02] BSE solver(s) @q1
 <01m-00s> P1-ac0137: [LA] SERIAL linear algebra
 <01m-00s> P1-ac0137: [06.02.01] Diago Solver @q1
 <01m-00s> P1-ac0137: Folding BSE Kernel |                                        | [000%] --(E) --(X)

and it is not updated for many hours.

Would it be possible to see in the LOG files which k-point is done, as it is in the kernel step?

Code: Select all

<02h-30m> P1-ac0183: Kernel |##################################      | [085%] 02h-29m(E) 02h-55m(X)
 <02h-33m> P1-ac0183: Kernel |###################################     | [087%] 02h-32m(E) 02h-53m(X)
 <02h-35m> P1-ac0183: Kernel |####################################    | [090%] 02h-34m(E) 02h-51m(X)
 <02h-37m> P1-ac0183: Kernel |#####################################   | [092%] 02h-36m(E) 02h-49m(X)
 <02h-41m> P1-ac0183: Kernel |######################################  | [095%] 02h-40m(E) 02h-49m(X)
 <02h-45m> P1-ac0183: Kernel |####################################### | [097%] 02h-44m(E) 02h-48m(X)
 <02h-49m> P1-ac0183: Kernel |########################################| [100%] 02h-48m(E) 02h-48m(X)
 <02h-49m> P1-ac0183: [08] Timing Overview
 <02h-49m> P1-ac0183: [TIMING]            io_ATMPROJ_pwscf :      0.0000s CPU

Thank you in advance.
All the best!
Gosia

[Daniele Varsano]

Dear Gosia,

to the best of my knowledge, this is not straightforward.
While the kernel build up it is based on multiple loop operation and the time estimation is done with a simple call to the clock, the diagonalization instead it is performed by using linear algebra subroutines. What I can tell you is that for direct diagonalization, the time-to-solution scales very badly the dimension of the matrix. How large is the dimension of your matrix? This is written in the report file, and should it be larger than few tenths of thousands I would suggest you to use iterative schemes (e.g. haydock or slepc in case you need eigenvectors).

Best,

Daniele

[malwi]

Dear Daniele,

thank you for the info. I would prefer to have vectors for many transitions and full diagonalization.
The matrix dimension is
[K] Dimension : 38890

and it seems to be a doible job, because it was not "OOM killed" but stopped "due to time limit".
I do not expect very scalable accurate estimate of the time.
But it would be good to know whether it needs time more than a week or two or two months....

Would there be any possibility to increase output in LOG files to know where we are?
The kernel was read and then it stopped exactly here

<56s> P1-ac0137: Loading full BSE kernel |########################################| [100%] 52s(E) 52s(X)
<01m-00s> P1-ac0137: [06.02] BSE solver(s) @q1
<01m-00s> P1-ac0137: [LA] SERIAL linear algebra
<01m-00s> P1-ac0137: [06.02.01] Diago Solver @q1
<01m-00s> P1-ac0137: Folding BSE Kernel | | [000%] --(E) --(X)

I am more advanced with other larger job, that takes
[K] Dimension : 248832

and I use cpu+gpu there.
This job stopped here

<03h-18m> P1-t0004: Loading full BSE kernel |########################################| [100%] 03h-17m(E) 03h-17m(X)
<04h-08m> P1-t0004: [06.02] BSE solver(s) @q1
<04h-08m> P1-t0004: [LA] SERIAL linear algebra
<04h-08m> P1-t0004: [06.03] Diago Solver @q1
<04h-08m> P1-t0004: Folding BSE Kernel | | [000%] --(E) --(X)
<04h-08m> P1-t0004: Folding BSE Kernel |########################################| [100%] --(E) --(X)

We need to analyse many excitonic states.

Best regards,
Gosia

[Daniele Varsano]

Dear Gosia,

as you can see from the log files you are using *scalar* linear algebra.
I suggest you to use parallel linear algebra, and please note that in principle you can include hundreds of eigenstates with the slepc method.

About the timing, it really depends on your CPU architecture. Maybe someone with experience on these large diagonalizations can provide you some estimates. I confirm that 38890 dimension will be time-consuming (maybe days?) while 248832 is not affordable with a direct diagonalization.

Best,

Daniele