Hi.
I am trying to calculate BS spectra using Yambo. I have run my calculations for several days, but, as I have a walltime of 24 hours on my cluster, I need to restart the calculations every time. I noticed, however, that the calculation always seems to start from the beginning, and in the end of 24 hours it always seems to finish in the same point (see for example files yambo-bse-1-150.o7214861 and yambo-bse-1-150.o7223667; they were consecutive calculations). What am I doing wrong? Do I need to do anything else to restart a BSE calculation?
I really appreciate any help with this issue.
Best wishes,
Juliana
BSE - restart
Moderators: Davide Sangalli, andrea.ferretti, myrta gruning, andrea marini, Daniele Varsano, Conor Hogan
-
- Posts: 39
- Joined: Tue Feb 11, 2014 5:33 am
BSE - restart
You do not have the required permissions to view the files attached to this post.
Juliana M. Morbec, Ph.D.
Postdoctoral Researcher
Institute for Molecular Engineering, The University of Chicago
Postdoctoral Researcher
Institute for Molecular Engineering, The University of Chicago
- Daniele Varsano
- Posts: 4198
- Joined: Tue Mar 17, 2009 2:23 pm
- Contact:
Re: BSE - restart
Dear Juliana,
your job doe not stops in the excitonic matrix filling process (where a restart can be done) but in the diagonalization process, so you cannot restart it. Anyway from what I can see from your standard output:
BSK <08m-46s> BSK | | [000%] --(E) --(X)^M <22h-00m-13s> BSK |# | [005%] 21h-51m-26s(E) 18d-05h-08m-54s(X)=>> PBS: job killed: walltime 86411 exceeded limit 86400
If the timing is working it looks an impressive task to diagonalize such matrix (18 days!!!). The reason should be that the dimension of the matrix are enormous (but it is just a guess as I do not have here the report/input file, you can estimate as Nc*Nv*Nk*Nsymm).
Having matrix bigger that 5000x5000 more or less, direct diagonalization it is not efficient at all and I suggest you to resort to iterative procedure (Haydock yambo -y h). This is much more efficient, you get absorption spectra even if you pay the fact that eigenvectors are not available.
Hope it helps,
Daniele
your job doe not stops in the excitonic matrix filling process (where a restart can be done) but in the diagonalization process, so you cannot restart it. Anyway from what I can see from your standard output:
BSK <08m-46s> BSK | | [000%] --(E) --(X)^M <22h-00m-13s> BSK |# | [005%] 21h-51m-26s(E) 18d-05h-08m-54s(X)=>> PBS: job killed: walltime 86411 exceeded limit 86400
If the timing is working it looks an impressive task to diagonalize such matrix (18 days!!!). The reason should be that the dimension of the matrix are enormous (but it is just a guess as I do not have here the report/input file, you can estimate as Nc*Nv*Nk*Nsymm).
Having matrix bigger that 5000x5000 more or less, direct diagonalization it is not efficient at all and I suggest you to resort to iterative procedure (Haydock yambo -y h). This is much more efficient, you get absorption spectra even if you pay the fact that eigenvectors are not available.
Hope it helps,
Daniele
Dr. Daniele Varsano
S3-CNR Institute of Nanoscience and MaX Center, Italy
MaX - Materials design at the Exascale
http://www.nano.cnr.it
http://www.max-centre.eu/
S3-CNR Institute of Nanoscience and MaX Center, Italy
MaX - Materials design at the Exascale
http://www.nano.cnr.it
http://www.max-centre.eu/
-
- Posts: 39
- Joined: Tue Feb 11, 2014 5:33 am
Re: BSE - restart
Thanks, Daniele, for your reply.
But I think I am using Haydock method. To generate the input I used "yambo -i -V RL -b -o b -y h". I send you here the input and r_setup. I used 3x3x3 k-points in my pwscf calculation (both scf and nscf) and I have 64 occupied bands. I think I cannot reduce the k-point mesh neither the number of BSEbands. Is there any parameter in my calculation I can decrease? Maybe in BSEBands (or BndsRnXs) instead of use 1-150 (or 1-300) I can reduce the number of valence bands, something like 50-150 (or 50-300)?
Could you help me with this, please?
I will really appreciate any help you can provide.
Best wishes,
Juliana
But I think I am using Haydock method. To generate the input I used "yambo -i -V RL -b -o b -y h". I send you here the input and r_setup. I used 3x3x3 k-points in my pwscf calculation (both scf and nscf) and I have 64 occupied bands. I think I cannot reduce the k-point mesh neither the number of BSEbands. Is there any parameter in my calculation I can decrease? Maybe in BSEBands (or BndsRnXs) instead of use 1-150 (or 1-300) I can reduce the number of valence bands, something like 50-150 (or 50-300)?
Could you help me with this, please?
I will really appreciate any help you can provide.
Best wishes,
Juliana
You do not have the required permissions to view the files attached to this post.
Juliana M. Morbec, Ph.D.
Postdoctoral Researcher
Institute for Molecular Engineering, The University of Chicago
Postdoctoral Researcher
Institute for Molecular Engineering, The University of Chicago
- Daniele Varsano
- Posts: 4198
- Joined: Tue Mar 17, 2009 2:23 pm
- Contact:
Re: BSE - restart
Dear Juliana,
sorry before I was wrong, the code stops while building the BS matrix.
First of all, even not related to your problem, pay attention to the fact that having FFTGvecs > MaxGvecs it is inconsistent, maxGvecs gives you the maximum number of Gvector in all the calculation so be sure that 1000 is enough. Indeed you have a warning stating:
[WARNING]Exchange FFT size is too big. RL vectors reduced to 869
Next:
The problem here is that you are calculating a really enormous matrix:
The kernel dimension is Nc x Nv x K_bz = 86x64x27=148608!!!
are you sure you need all this occupied and unoccupied bands to get convergences?
Note that the screening here it is not a problem, as you have calculated it, so if converged you do not have to touch BndsRnXs.
My suggestion is to try look at Nc x Nv bands needed starting from few bands ie BSEBands 60-70, and enlarging the elecotron-hole windows,
55-75 etc., until you get convergences. I do not think that you need to include the deep valence bands are they are far away in energy ( ~-68 eV)
unless you are interested in the very high energy part of the spectrum. If you are interested in the onset and first peaks, electron-hole transition with very high energies should not participate in the first excitations.
Hope it helps,
Daniele
sorry before I was wrong, the code stops while building the BS matrix.
First of all, even not related to your problem, pay attention to the fact that having FFTGvecs > MaxGvecs it is inconsistent, maxGvecs gives you the maximum number of Gvector in all the calculation so be sure that 1000 is enough. Indeed you have a warning stating:
[WARNING]Exchange FFT size is too big. RL vectors reduced to 869
Next:
The problem here is that you are calculating a really enormous matrix:
The kernel dimension is Nc x Nv x K_bz = 86x64x27=148608!!!
are you sure you need all this occupied and unoccupied bands to get convergences?
Note that the screening here it is not a problem, as you have calculated it, so if converged you do not have to touch BndsRnXs.
My suggestion is to try look at Nc x Nv bands needed starting from few bands ie BSEBands 60-70, and enlarging the elecotron-hole windows,
55-75 etc., until you get convergences. I do not think that you need to include the deep valence bands are they are far away in energy ( ~-68 eV)
unless you are interested in the very high energy part of the spectrum. If you are interested in the onset and first peaks, electron-hole transition with very high energies should not participate in the first excitations.
Hope it helps,
Daniele
Dr. Daniele Varsano
S3-CNR Institute of Nanoscience and MaX Center, Italy
MaX - Materials design at the Exascale
http://www.nano.cnr.it
http://www.max-centre.eu/
S3-CNR Institute of Nanoscience and MaX Center, Italy
MaX - Materials design at the Exascale
http://www.nano.cnr.it
http://www.max-centre.eu/
-
- Posts: 39
- Joined: Tue Feb 11, 2014 5:33 am
Re: BSE - restart
Thanks Daniele for your reply.
I just did not understand when you said "Note that the screening here it is not a problem, as you have calculated it, so if converged you do not have to touch BndsRnXs.".
In this calculation I did: scf+nscf; generate the core databases (p2y); run setup (yambo); generate input (yambo -i -V RL -b -o b -y h); run yambo.
Do you think I should separate the last calculation in two parts? I mean, 1) yambo -b, checking the convergence of BndsRnXs, and after 2) yambo -o b -y h?
Thank you!
I just did not understand when you said "Note that the screening here it is not a problem, as you have calculated it, so if converged you do not have to touch BndsRnXs.".
In this calculation I did: scf+nscf; generate the core databases (p2y); run setup (yambo); generate input (yambo -i -V RL -b -o b -y h); run yambo.
Do you think I should separate the last calculation in two parts? I mean, 1) yambo -b, checking the convergence of BndsRnXs, and after 2) yambo -o b -y h?
Thank you!
Juliana M. Morbec, Ph.D.
Postdoctoral Researcher
Institute for Molecular Engineering, The University of Chicago
Postdoctoral Researcher
Institute for Molecular Engineering, The University of Chicago
- Daniele Varsano
- Posts: 4198
- Joined: Tue Mar 17, 2009 2:23 pm
- Contact:
Re: BSE - restart
Dear Juliana,
you can divide them, or do in one shot as you were doing.
What I wanted to say was that the calculation of the screened was correctly done by yambo (you should have the em1s database in the SAVE directory) and was not the source of the problem, and the problem was the size of the excitonic matrix. Anyway as you say the convergence of the bands BndsRnXs has to be checked, but as the static screening does not give you any output you need to check in the final spectrum.
Hope it is clear,
Best,
Daniele
you can divide them, or do in one shot as you were doing.
What I wanted to say was that the calculation of the screened was correctly done by yambo (you should have the em1s database in the SAVE directory) and was not the source of the problem, and the problem was the size of the excitonic matrix. Anyway as you say the convergence of the bands BndsRnXs has to be checked, but as the static screening does not give you any output you need to check in the final spectrum.
Hope it is clear,
Best,
Daniele
Dr. Daniele Varsano
S3-CNR Institute of Nanoscience and MaX Center, Italy
MaX - Materials design at the Exascale
http://www.nano.cnr.it
http://www.max-centre.eu/
S3-CNR Institute of Nanoscience and MaX Center, Italy
MaX - Materials design at the Exascale
http://www.nano.cnr.it
http://www.max-centre.eu/
-
- Posts: 39
- Joined: Tue Feb 11, 2014 5:33 am
Re: BSE - restart
Yes, it is clear.
Thank you very much, Daniele, for your help.
Thank you very much, Daniele, for your help.
Juliana M. Morbec, Ph.D.
Postdoctoral Researcher
Institute for Molecular Engineering, The University of Chicago
Postdoctoral Researcher
Institute for Molecular Engineering, The University of Chicago