rev 16: LFS support absent on CINECA sp6 !!
Moderators: Davide Sangalli, andrea.ferretti, myrta gruning, andrea marini, Daniele Varsano, Conor Hogan, Nicola Spallanzani
-
- Posts: 35
- Joined: Thu May 21, 2009 3:46 pm
rev 16: LFS support absent on CINECA sp6 !!
Dear all,
I changed topic because now we are working with rev 16, but the problem are the same posted in the previous topic and concerned rev 14 viewtopic.php?f=10&t=262&start=0 .
Just to remind you:
I am trying to run a simulation where the production of a database (in my case ndb.kindx) is larger that 2Gb (in my case ~6Gb). This is called, according with the netcdf standard, a Large File. In order to store this file you have to carefully set Large File Support (LFS) in the netcdf calls.
I have found that:
running on CINECA sp6 YAMBO WORKS FINE ONLY WITHOUT NETCDF. Without linking them is the only way to let Yambo correctly store the Large Files and end the run without errors on CINECA sp6.
The problems now can only be:
1) Netcdf on sp6 CINECA are bad.
2) Yambo has a bug that stops the run when a large file has to be written using netcdf.
As a lot of people, besides the YAMBO community, has been using CINECA netcdf modules for their applications and, moreover, because I tried to linked Yambo with almost any flavours of hand-installed netcdf 4 and 3.6 without removing the error: I support the second point, but I would not bet money on it.
If you cannot reproduce the error on local machines, maybe it is because the data to be contained in the output large file has to be stored in ram before being printed and maybe you need Hpc facilities for this.
In order to focus the problem, of course you shall reproduce the error on architectures other than CINECA sp6. If you have access to other clusters able to handle almost 6 or 7 Gb of ram I can give you with no problems the inputs of my test (Andrea already has). They should run just on 1 cpu (just because the portion of the code connected to my test is serial) and take almost 4 h to be accomplished.
Let me know.
Marco
I changed topic because now we are working with rev 16, but the problem are the same posted in the previous topic and concerned rev 14 viewtopic.php?f=10&t=262&start=0 .
Just to remind you:
I am trying to run a simulation where the production of a database (in my case ndb.kindx) is larger that 2Gb (in my case ~6Gb). This is called, according with the netcdf standard, a Large File. In order to store this file you have to carefully set Large File Support (LFS) in the netcdf calls.
I have found that:
running on CINECA sp6 YAMBO WORKS FINE ONLY WITHOUT NETCDF. Without linking them is the only way to let Yambo correctly store the Large Files and end the run without errors on CINECA sp6.
The problems now can only be:
1) Netcdf on sp6 CINECA are bad.
2) Yambo has a bug that stops the run when a large file has to be written using netcdf.
As a lot of people, besides the YAMBO community, has been using CINECA netcdf modules for their applications and, moreover, because I tried to linked Yambo with almost any flavours of hand-installed netcdf 4 and 3.6 without removing the error: I support the second point, but I would not bet money on it.
If you cannot reproduce the error on local machines, maybe it is because the data to be contained in the output large file has to be stored in ram before being printed and maybe you need Hpc facilities for this.
In order to focus the problem, of course you shall reproduce the error on architectures other than CINECA sp6. If you have access to other clusters able to handle almost 6 or 7 Gb of ram I can give you with no problems the inputs of my test (Andrea already has). They should run just on 1 cpu (just because the portion of the code connected to my test is serial) and take almost 4 h to be accomplished.
Let me know.
Marco
Marco Govoni
Physics Department, University of Modena and Reggio Emilia (Italy)
Physics Department, University of Modena and Reggio Emilia (Italy)
-
- Posts: 9
- Joined: Tue Aug 31, 2010 9:55 am
Re: rev 16: LFS support absent on CINECA sp6 !!
Hi Marco,
- I succesfully compiled yambo linking netcdf, so I would suppose your problem is runtime, isn't it?
- I suppose the problem is not in the NetCDF itself because, as you report, there many users that link NetCDF without reporting this kind of problems.
I just would like to add a consideration to your problem.
It would be extremely useful to have some instruction from the developers about what and how to link external libraries. For instance, when you compile NetCDF you need also HDF5. HDF5, in turn, need zlib and szlib. Each of these libraries can be compiled with a lot of option: single or double precision, static or not, with or without fortran support, in parallel or serial way. You can see that the number of combinations become huge and try to guess which is appropriate to Yambo is almost impossible without the support from the developers team. Maybe there is some instruction somewhere, but I didn't find it...
Let me know, please.
Fabio
- I succesfully compiled yambo linking netcdf, so I would suppose your problem is runtime, isn't it?
- I suppose the problem is not in the NetCDF itself because, as you report, there many users that link NetCDF without reporting this kind of problems.
I just would like to add a consideration to your problem.
It would be extremely useful to have some instruction from the developers about what and how to link external libraries. For instance, when you compile NetCDF you need also HDF5. HDF5, in turn, need zlib and szlib. Each of these libraries can be compiled with a lot of option: single or double precision, static or not, with or without fortran support, in parallel or serial way. You can see that the number of combinations become huge and try to guess which is appropriate to Yambo is almost impossible without the support from the developers team. Maybe there is some instruction somewhere, but I didn't find it...
Let me know, please.
Fabio
Fabio Affinito, PhD
CINECA - SCAI group
InterUniversity Computer Center
Via Magnanelli, 6/3
Casalecchio di Reno (Bologna) ITALY
+39/051/6171794 (Phone)
CINECA - SCAI group
InterUniversity Computer Center
Via Magnanelli, 6/3
Casalecchio di Reno (Bologna) ITALY
+39/051/6171794 (Phone)
-
- Posts: 31
- Joined: Tue Apr 07, 2009 4:39 pm
Re: rev 16: LFS support absent on CINECA sp6 !!
On my side, I can suggest also the people with errors to attach their compilation option used on sp6 in terms of: modules loaded and configure options used. Like that we can try to have an historical of all the compilation flavours. what do you say?
- myrta gruning
- Posts: 240
- Joined: Tue Mar 17, 2009 11:38 am
- Contact:
Re: rev 16: LFS support absent on CINECA sp6 !!
Hallo Fabio,faffinito wrote:. For instance, when you compile NetCDF you need also HDF5.
HDF5 is not needed by Yambo. At present Yambo does not use any of the hdf5 featues.
Also in case Yambo will implement the use of NETCDF+HDF5 we will left the option of having NETCDF without HDF5.
I do not think that the options for libraries linked to a library linked to NETCDF are influenced by yambo.Each of these libraries can be compiled with a lot of option: single or double precision, static or not, with or without fortran support, in parallel or serial way. ou can see that the number of combinations become huge and try to guess which is appropriate to Yambo is almost impossible without the support from the developers team. Maybe there is some instruction somewhere, but I didn't find it...
The standard options to compile those libs should be fine. If there is some special requirements may be because of netcdf, not because of yambo. Also I do not think that e.g. serial or parallel, static or dynamic compilation should in principle matter.
What we know is that is crucial to use the same compiler as for yambo. Having e.g. netcdf compiled with gcc/gfortran and yambo with icc/ifort will not work.
Unfortunately it is true that netcdf is becoming more and more complicated, and changing rapidily and linking more and more things. Note that all new features of netcdf4, that usually need to link to other libraries, are not needed by Yambo.
Please note we are not great experts in netcdf and related libraries, we are using the library and most of the time learning by experience as other users. Our experience is limited to some machine/compilers/version of netcdf. Through this forum one can in principle access to the experiences of other users that may have had and solved the same problem.
Regards,
Myrta
Dr Myrta Grüning
School of Mathematics and Physics
Queen's University Belfast - Northern Ireland
http://www.researcherid.com/rid/B-1515-2009
School of Mathematics and Physics
Queen's University Belfast - Northern Ireland
http://www.researcherid.com/rid/B-1515-2009
-
- Posts: 9
- Joined: Tue Aug 31, 2010 9:55 am
Re: rev 16: LFS support absent on CINECA sp6 !!
Hi Myrta,
following your suggestions, I decided to restart from scratch.
First I compiled NetCDF4.0.1 (there's a bug affecting 4.1.1, and I didn't want to waste time on that) with:
Then I simply installed Yambo 3.2.3 r.696 with:
obtaining this error in compilation :
Any suggestion about that?
Fabio
following your suggestions, I decided to restart from scratch.
First I compiled NetCDF4.0.1 (there's a bug affecting 4.1.1, and I didn't want to waste time on that) with:
Code: Select all
export CFLAGS="-g -O2 -q64 -qarch=pwr6 -qtune=pwr6 -qmaxmem=-1 -qsuppress=1501-245"
export CXXFLAGS=$CFLAGS
export FFLAGS=$CFLAGS
export FCFLAGS=$CFLAGS
./configure --prefix=/gpfs/scratch/userinternal/cin0644a/netcdf4 \
Code: Select all
module load blacs
module load scalapack
export CPP=cpp
export CC=xlc_r
export F77=xlf_r
export FC=xlf90_r
export FCFLAGS='-O2 -q64 -qstrict -qarch=pwr6 -qtune=pwr6 -qmaxmem=-1 -qautodbl=dbl4 -qsuffix=f=f -qrealsize'
./configure --prefix=/sp6/userinternal/cin0644a/yambo \
--with-fftw=/cineca/prod/libraries/fftw/3.2.2/xl--10.1/lib \
--with-netcdf-lib=/gpfs/scratch/userinternal/cin0644a/netcdf4/lib \
--with-netcdf-include=/gpfs/scratch/userinternal/cin0644a/netcdf4/lib/include \
--with-p2y=4.0 \
--with-iotk=/cineca/prod/build/applications/QuantumESPRESSO/4.1/xl--10.1/BA_WORK/QuantumESPRESSO-4.1/iotk \
--build=powerpc-ibm \
--enable-netcdf-LFS
Code: Select all
"mod_itm.f", line 437.34: 1513-062 (S) Generic procedure reference can not be resolved due to incorrect actual argument attributes.
"mod_itm.f", line 471.34: 1513-062 (S) Generic procedure reference can not be resolved due to incorrect actual argument attributes.
"mod_itm.f", line 536.21: 1513-062 (S) Generic procedure reference can not be resolved due to incorrect actual argument attributes.
"mod_itm.f", line 551.21: 1513-062 (S) Generic procedure reference can not be resolved due to incorrect actual argument attributes.
"mod_itm.f", line 566.20: 1513-062 (S) Generic procedure reference can not be resolved due to incorrect actual argument attributes
Fabio
Fabio Affinito, PhD
CINECA - SCAI group
InterUniversity Computer Center
Via Magnanelli, 6/3
Casalecchio di Reno (Bologna) ITALY
+39/051/6171794 (Phone)
CINECA - SCAI group
InterUniversity Computer Center
Via Magnanelli, 6/3
Casalecchio di Reno (Bologna) ITALY
+39/051/6171794 (Phone)
- andrea marini
- Posts: 325
- Joined: Mon Mar 16, 2009 4:27 pm
- Contact:
Re: rev 16: LFS support absent on CINECA sp6 !!
Fabio, anyway you can provide us access on SP ?It is really hard for me to understand where the error is without having access to the source itself.
Andrea
Andrea
Andrea MARINI
Istituto di Struttura della Materia, CNR, (Italy)
Istituto di Struttura della Materia, CNR, (Italy)
-
- Posts: 31
- Joined: Tue Apr 07, 2009 4:39 pm
Re: rev 16: LFS support absent on CINECA sp6 !!
PROBLEM FIXED:
report on
the error previously reported
has been fixed for the moment.
The database ndb.pp that yambo was creating during the plasmon pole calculation was larger than 2Gb.
With Daniele support, I've launch yambo -S (with fragmentation option) for the plasmon pole calculation.
It works. In the end I obtain a ndb.pp database fragmented in n-part (less than 2Gb) as the number of q-point I have in my calculation.
So, I can say that this error I got was due to database larger than 2 Gb.
Problem fixed for the moment.
Thanks to all.
report on
Code: Select all
yambo_3.2.3_rev16 compiled linking Netcdf and --enable-netcdf-LFS=no (default)
the error previously reported
Code: Select all
[ERROR] STOP signal received while in :[09] Dynamic Dielectric Matrix (PPA)
[ERROR][NetCDF] NetCDF: One or more variable sizes violate format constraints
The database ndb.pp that yambo was creating during the plasmon pole calculation was larger than 2Gb.
With Daniele support, I've launch yambo -S (with fragmentation option) for the plasmon pole calculation.
It works. In the end I obtain a ndb.pp database fragmented in n-part (less than 2Gb) as the number of q-point I have in my calculation.
So, I can say that this error I got was due to database larger than 2 Gb.
Code: Select all
SOLUTION: run with yambo -S
Thanks to all.