ifort mpi compilation options

[matthieu]

Hello,

I have finally succeeded in compiling yambo on a xeon openmpi platform with intel fortran 11, so here are:

1) my configure options and

./configure FC=mpif90 FCFLAGS="-O3 -g -nofor_main" --enable-dp --enable-largedb --with-mpi=yes --with-fftw=/cvos/shared/apps/fftw/gcc/64/3.1.2/lib/ --with-etsf=~/CODES/ABINIT/6.0.0-private/tmp-ifort/plugins/etsf_io/ --with-netcdf-include=~/CODES/ABINIT/6.0.0-private/tmp-ifort/plugins/netcdf/ --with-scalapack=/cvos/shared/apps/scalapack/intel/current/lib64/ --with-blacs=/cvos/shared/apps/blacs/openmpi/intel/current/lib64/ --with-lapack=/cvos/shared/apps/lapack/intel/64/3.0/liblapack.a --with-blas=/cvos/shared/apps/gotoblas/penryn/64/1.26/libgoto.so


2)the big nuisance:
you need to add -nofor_main to the FCFLAGS to keep ifort from adding a default main() to the executable (yambo has its own). Then the ifort complains that main is defined twice.

- My etsfio detection did not work: don't know why.
- The rest look ok. Does anyone have a test for the different parts of the parallelization? Eg something for the normal mpi scaling, and something for the scalapack one?

ciao tutti

Matthieu

[andrea marini]

Hello,

Hi matthieu, nice to see you on board A minor request... of course most of us know you but could you include your complete affiliation in the signature for all other memebers ? Thanks !

I have finally succeeded in compiling yambo on a xeon openmpi platform with intel fortran 11, so here are:

1) my configure options and

./configure FC=mpif90 FCFLAGS="-O3 -g -nofor_main" --enable-dp --enable-largedb --with-mpi=yes --with-fftw=/cvos/shared/apps/fftw/gcc/64/3.1.2/lib/ --with-etsf=~/CODES/ABINIT/6.0.0-private/tmp-ifort/plugins/etsf_io/ --with-netcdf-include=~/CODES/ABINIT/6.0.0-private/tmp-ifort/plugins/netcdf/ --with-scalapack=/cvos/shared/apps/scalapack/intel/current/lib64/ --with-blacs=/cvos/shared/apps/blacs/openmpi/intel/current/lib64/ --with-lapack=/cvos/shared/apps/lapack/intel/64/3.0/liblapack.a --with-blas=/cvos/shared/apps/gotoblas/penryn/64/1.26/libgoto.so

Careful with the -enable-largedb and -enable-dp option. The first is needed only if any of the fragments produced by Yambo during the I/O get larger then 2Gb. This is quite unlikely unless you are running huge systems. The second option enables double precision, so all your memeory usage will be moltiplied by 2. I have never found till now a case where the use of DP makes an important difference. Even my old calculation of total energies did not show any difference (and we are talking about mHa/electron !)

2)the big nuisance:
you need to add -nofor_main to the FCFLAGS to keep ifort from adding a default main() to the executable (yambo has its own). Then the ifort complains that main is defined twice.

The configure should already check for the correct flag to link the C driver. As you have passed your own FCFLAGS this step is skipped. Try leaving unset the FCFLAGS, and you should get the correct options from teh configure.
[/quote]

- My etsfio detection did not work: don't know why.

Conor is the ETSF I/O responsible. Conor, could you have a look ?

- The rest look ok. Does anyone have a test for the different parts of the parallelization? Eg something for the normal mpi scaling, and something for the scalapack one?

Hmmmm ... we have some scalability test made by the guys in Barcellona. But the input databases are huge. If it is fine for you I will give you access to the results so to have an idea of the performances. The tests, however, are very basical and they have been made without the scalapack support.

[Conor Hogan]

./configure FC=mpif90 FCFLAGS="-O3 -g -nofor_main" --enable-dp --enable-largedb --with-mpi=yes --with-fftw=/cvos/shared/apps/fftw/gcc/64/3.1.2/lib/ --with-etsf=~/CODES/ABINIT/6.0.0-private/tmp-ifort/plugins/etsf_io/ --with-netcdf-include=~/CODES/ABINIT/6.0.0-private/tmp-ifort/plugins/netcdf/ --with-scalapack=/cvos/shared/apps/scalapack/intel/current/lib64/ --with-blacs=/cvos/shared/apps/blacs/openmpi/intel/current/lib64/ --with-lapack=/cvos/shared/apps/lapack/intel/64/3.0/liblapack.a --with-blas=/cvos/shared/apps/gotoblas/penryn/64/1.26/libgoto.so

2)the big nuisance:
you need to add -nofor_main to the FCFLAGS to keep ifort from adding a default main() to the executable (yambo has its own). Then the ifort complains that main is defined twice.

- My etsfio detection did not work: don't know why.

Hi Matthieu,
I second the comment of Andrea: best to leave FCFLAGS untouched. If you absolutely need to do some tweaking after configure, edit the config/setup file.

ETSF: wow, I never thought I'd see the day. Following discussions some years back with the ETSF_IO people, I assumed the default installation to look like (/opt/etsf being replaced by the user defined path):

Code: Select all

/opt/etsf/include/intel/*.mod
/opt/etsf/lib/*.a

since there were some incompatibility problems with (intel) compiler specific mod files, etc. Look for "etsf" in config/configure.ac to see the paths.
This can all surely be updated now that someone other than me is trying it out. This is probably what's going wrong.
Otherwise, check you have also linked the netcdf library properly: you are missing --netcdf-lib= ...
Note that etsf support is only rudimentory, due to lack of interest so far

Hope it helps,
Conor

[myrta gruning]

Hello Matthieu

Code: Select all

export FC=ifort
export CC=icc
export CPP=/lib/cpp
export NETCDFHOME=$HOME/share/netcdf

./configure --with-fftw=$FFTWDIR  --with-netcdf-include=$NETCDFHOME/include --with-netcdf-lib=$NETCDFHOME/lib

This was the configure I used for compiling yambo on a xeon cluster (green) with intel 10.1 and openmpi.

About scalapack I just ran small tests on few cpus to see it was working (distributing the work), but nothing to test seriously the scaling.

m