printing time and % in LOG, BSE diagonalization

Various technical topics such as parallelism and efficiency, netCDF problems, the Yambo code structure itself, are posted here.

Moderators: Davide Sangalli, andrea.ferretti, myrta gruning, andrea marini, Daniele Varsano, Conor Hogan, Nicola Spallanzani

Post Reply
User avatar
malwi
Posts: 36
Joined: Mon Feb 29, 2016 1:00 pm

printing time and % in LOG, BSE diagonalization

Post by malwi » Tue Jan 02, 2024 4:17 pm

Dear Yambo TEAM,

All the best for 2024, further success in Yambo development and your private life!

I went over the wall-time when making full diagonalization in BSE with almost 1000 k-points.
I did not find any estimate in LOG directory how much it would take.
If such info was available it is always possible to ask administrators for an extension for the job.

The last info in LOG is

Code: Select all

 <56s> P1-ac0137: Loading full BSE kernel |########################################| [100%] 52s(E) 52s(X)
 <01m-00s> P1-ac0137: [06.02] BSE solver(s) @q1
 <01m-00s> P1-ac0137: [LA] SERIAL linear algebra
 <01m-00s> P1-ac0137: [06.02.01] Diago Solver @q1
 <01m-00s> P1-ac0137: Folding BSE Kernel |                                        | [000%] --(E) --(X)
and it is not updated for many hours.

Would it be possible to see in the LOG files which k-point is done, as it is in the kernel step?

Code: Select all

<02h-30m> P1-ac0183: Kernel |##################################      | [085%] 02h-29m(E) 02h-55m(X)
 <02h-33m> P1-ac0183: Kernel |###################################     | [087%] 02h-32m(E) 02h-53m(X)
 <02h-35m> P1-ac0183: Kernel |####################################    | [090%] 02h-34m(E) 02h-51m(X)
 <02h-37m> P1-ac0183: Kernel |#####################################   | [092%] 02h-36m(E) 02h-49m(X)
 <02h-41m> P1-ac0183: Kernel |######################################  | [095%] 02h-40m(E) 02h-49m(X)
 <02h-45m> P1-ac0183: Kernel |####################################### | [097%] 02h-44m(E) 02h-48m(X)
 <02h-49m> P1-ac0183: Kernel |########################################| [100%] 02h-48m(E) 02h-48m(X)
 <02h-49m> P1-ac0183: [08] Timing Overview
 <02h-49m> P1-ac0183: [TIMING]            io_ATMPROJ_pwscf :      0.0000s CPU
Thank you in advance.
All the best!
Gosia
dr hab. Małgorzata Wierzbowska, Prof. IHPP PAS
Institute of High Pressure Physics Polish Academy of Sciences
Warsaw, Poland

User avatar
Daniele Varsano
Posts: 3926
Joined: Tue Mar 17, 2009 2:23 pm
Contact:

Re: printing time and % in LOG, BSE diagonalization

Post by Daniele Varsano » Mon Jan 08, 2024 2:24 pm

Dear Gosia,

to the best of my knowledge, this is not straightforward.
While the kernel build up it is based on multiple loop operation and the time estimation is done with a simple call to the clock, the diagonalization instead it is performed by using linear algebra subroutines. What I can tell you is that for direct diagonalization, the time-to-solution scales very badly the dimension of the matrix. How large is the dimension of your matrix? This is written in the report file, and should it be larger than few tenths of thousands I would suggest you to use iterative schemes (e.g. haydock or slepc in case you need eigenvectors).

Best,

Daniele
Dr. Daniele Varsano
S3-CNR Institute of Nanoscience and MaX Center, Italy
MaX - Materials design at the Exascale
http://www.nano.cnr.it
http://www.max-centre.eu/

User avatar
malwi
Posts: 36
Joined: Mon Feb 29, 2016 1:00 pm

Re: printing time and % in LOG, BSE diagonalization

Post by malwi » Fri Jan 12, 2024 6:33 pm

Dear Daniele,

thank you for the info. I would prefer to have vectors for many transitions and full diagonalization.
The matrix dimension is
[K] Dimension : 38890

and it seems to be a doible job, because it was not "OOM killed" but stopped "due to time limit".
I do not expect very scalable accurate estimate of the time.
But it would be good to know whether it needs time more than a week or two or two months....

Would there be any possibility to increase output in LOG files to know where we are?
The kernel was read and then it stopped exactly here

<56s> P1-ac0137: Loading full BSE kernel |########################################| [100%] 52s(E) 52s(X)
<01m-00s> P1-ac0137: [06.02] BSE solver(s) @q1
<01m-00s> P1-ac0137: [LA] SERIAL linear algebra
<01m-00s> P1-ac0137: [06.02.01] Diago Solver @q1
<01m-00s> P1-ac0137: Folding BSE Kernel | | [000%] --(E) --(X)

I am more advanced with other larger job, that takes
[K] Dimension : 248832

and I use cpu+gpu there.
This job stopped here

<03h-18m> P1-t0004: Loading full BSE kernel |########################################| [100%] 03h-17m(E) 03h-17m(X)
<04h-08m> P1-t0004: [06.02] BSE solver(s) @q1
<04h-08m> P1-t0004: [LA] SERIAL linear algebra
<04h-08m> P1-t0004: [06.03] Diago Solver @q1
<04h-08m> P1-t0004: Folding BSE Kernel | | [000%] --(E) --(X)
<04h-08m> P1-t0004: Folding BSE Kernel |########################################| [100%] --(E) --(X)

We need to analyse many excitonic states.

Best regards,
Gosia
dr hab. Małgorzata Wierzbowska, Prof. IHPP PAS
Institute of High Pressure Physics Polish Academy of Sciences
Warsaw, Poland

User avatar
Daniele Varsano
Posts: 3926
Joined: Tue Mar 17, 2009 2:23 pm
Contact:

Re: printing time and % in LOG, BSE diagonalization

Post by Daniele Varsano » Wed Jan 17, 2024 12:33 pm

Dear Gosia,

as you can see from the log files you are using *scalar* linear algebra.
I suggest you to use parallel linear algebra, and please note that in principle you can include hundreds of eigenstates with the slepc method.

About the timing, it really depends on your CPU architecture. Maybe someone with experience on these large diagonalizations can provide you some estimates. I confirm that 38890 dimension will be time-consuming (maybe days?) while 248832 is not affordable with a direct diagonalization.

Best,

Daniele
Dr. Daniele Varsano
S3-CNR Institute of Nanoscience and MaX Center, Italy
MaX - Materials design at the Exascale
http://www.nano.cnr.it
http://www.max-centre.eu/

Post Reply