How to restart 'yambo -b'

[andrea marini]

Did you create a ssh key ? Lately QE-forge asked all member to do so :

Code: Select all

Date: Sat, 18 Sep 2010 09:47:41 +0200
From: noreply@qe-forge.org
Subject: [QEForge] cvs+ssh  and svn+ssh access repository

To increase security level of qe-forge.org server, now access to cvs and svn repository through ssh tunnel is possible only if you have imported ssh key
in to the server using web interface here:

http://qe-forge.org/account/editsshkeys.php

If you need help or experience problems, please send me an email at:
albecamp@democritos.it

Regards
The hostmaster

[marco.govoni]

Thank you Andrea, I missed that key.

Anyway we are still talking with Cineca staff (Fabio Affinito) about netcdf4+hdf5+yambo.
We would like to know if yambo needs netcdf4, and hence hdf5, in the serial or parallel version.

In case some of you is running on sp6: which module do you load in order to have large files support?

Marco

[andrea marini]

A devel patched source the could solve the problem has been posted right now in the SVN repository. It corresponds to revision 12. Please follow the instructions in the Yambo download page to access the SVN repository.

Please use revisione 13. I did a mistake in revision 12 in patching the header files

Andrea

[andrea marini]

Anyway we are still talking with Cineca staff (Fabio Affinito) about netcdf4+hdf5+yambo.
We would like to know if yambo needs netcdf4, and hence hdf5, in the serial or parallel version.

I do not run on sp6 so I cannot help you. hdf5 should improve the I/O performance in teh case of BIG databases. So it should improve the I/O in both serial and parallel runs.

Andrea

[myrta gruning]

Hallo Marco

Anyway we are still talking with Cineca staff (Fabio Affinito) about netcdf4+hdf5+yambo.
We would like to know if yambo needs netcdf4, and hence hdf5, in the serial or parallel version.

The current version of Yambo does not support the hdf5 (though you can link it with netcdf4+hdf5, but you won't gain anything).
We plan to add the support to it in the next future. Also then at configure time it will be detected whether you are linking to plain netcdf or to netcdf4+hdf5, so in principle you could live with plain netcdf.

m

[marco.govoni]

Wow, this sounds bad.

To be sure that I have understood:

1) Yambo uses netcdf4 serial and NOT parallel. (please confirm)
2) netcdf4 must be compiled without hdf5 support (otherwise yambo fails to be compiled for it tries to call hdf5 libraries).

As far as I read from netcdf forums and tried by myself configuring by hand netcdf4 on Cineca sp6 (tell me if I am wrong), large files can be handled only if netcdf4 libraries are supported by hdf5.
So my question is: if netcdf4 needs hdf5 to handle large files but yambo does not like hdf5, what is the meaning of putting --enable-largedb in yambo?

Marco

[marco.govoni]

Reading more accurately the guide of netcdf4 I found this:

What is Large File Support?
Large File Support (LFS) refers to operating system and C library facilities to support files larger than 2 GiB. On many 32-bit platforms the default size of a file offset is still a 4-byte signed integer, which limits the maximum size of a file to 2 GiB. Using LFS interfaces and the 64-bit file offset type, the maximum size of a file may be as large as 263 bytes, or 8 EiB. For many current platforms, large file macros or appropriate compiler flags have to be set to build a library with support for large files. This is handled automatically in netCDF 3.6.

Thais means that I was wrong and that netcdf4 compiled WITHOUT Hdf5 are still able to handle large files. Right?
Ok, about the serial / parallel version?

Marco

[myrta gruning]

Hallo Marco

Yes, my understanding is that hdf5 is a "plus", while netcdf features, like being able to handle or not large files, stay the same (I recall this is an option you can enable when compiling netcdf)
The hdf5, as Andrea said, speeds the performance for large files. It is desirable for example when you need to convert huge wavefunction files from the DFT to the yambo format.

You can compile netcdf4 with hdf5. Maybe I was not clear on that point, sorry. For Yambo, as it is now, will just be the same since it simply won't use the hdf5. It may be convenient in the future, when hopefully the support will be enabled, so that you won't need to recompile netcdf.
About serial/parallel. I do not know how much it will matter, but again you can compile netcdf with mpi. This is just my experience. I am linking yambo to netcdf4+hdf5 compiled with mpi and it is working (but well I am on a "simple" PC).

I hope this helps
m

[marco.govoni]

Ok I understood the problem with netcdf. Thank Myrta.

In the while I tried to use rev13 (without -S) and I got a further improvement with respect to my fist posts.
After a run of 'yambo -b -V QP' with q-vectors 1|2,
the second run with q-vectors 1|4 gives:

Code: Select all

_|      _|   _|_|   _|      _|  _|_|_|     _|_|
  _|  _|   _|    _| _|_|  _|_| _|    _| _|    _|
    _|     _|_|_|_| _|  _|  _| _|_|_|   _|    _|
    _|     _|    _| _|      _| _|    _| _|    _|
    _|     _|    _| _|      _| _|_|_|     _|_|

 <---> [01] Files & I/O Directories
 <---> [02] CORE Variables Setup
 <---> [02.01] Unit cells
 <---> [02.02] Symmetries
 <---> [02.03] RL shells
 <---> [02.04] K-grid lattice
 <05s> [02.05] Energies [ev] & Occupations
 <05s> [03] Transferred momenta grid
 <05s> [M 0.334 Gb] Alloc qindx_X qindx_S (0.325)
 <10s> [04] External QP corrections (X)
 <10s> [04.01] QP Corrections extrapolation
 <10s> [05] Static Dielectric Matrix
 <10s> [RESTARTer] Section(s) completed for ./SAVE//ndb.em1s :2
 <10s> [M 2.757 Gb] Alloc WF (2.406)
 <11s> [FFT-X] Mesh size: 20  20  20
 <11s> [WF-X loader] Wfs (re)loading |                    | [000%] --(E) --(X)
 <16s> [WF-X loader] Wfs (re)loading |#############       | [066%] 05s(E) 07s(X)
 <19s> [WF-X loader] Wfs (re)loading |####################| [100%] 07s(E) 07s(X)
 <19s> [M 2.916 Gb] Alloc X_poles_tab (0.159)
 <27s> [X-CG] R(p) Tot o/o(of R)  :   569826   4968000       100

so now it correctly reads 2 q points (instead of just 1 as was before), but the file is stuck at the last line I copied, remaining running but without producing any further output.

Marco

[marco.govoni]

I tired the same also with -S.

I confirm that the problem of multiple production of files with -S flag enabled does not occur anymore (rev13).

However, the second run with q-point from 1 to 4 has a bug: it reads 2 point in the database but does not start Xo calculation for q=3 as in the case without -S.

Marco