Error in compilation of PETSC

[vjhalani]

Hello!

I've recently tried to recompile Yambo v4.4. I had a working compilation of the same exact branch before, but perhaps some of the compilers changed since I last did it, and now I am having issues when compiling PETSc, getting the following error:

===============================================================================
Configuring PETSc to compile on your system
===============================================================================
===============================================================================
***** WARNING: MAKEFLAGS (set to w -- VPATH=/global/homes/v/vatsal/codes/yambo-devel/compile_devel-excitonlifetimes/../lib/petsc) found in environment variables
use ./configure MAKEFLAGS=$MAKEFLAGS if you really want to use that value ****** =============================================================================== ===============================================================================
It appears you do not have valgrind installed on your system.
We HIGHLY recommend you install it from http://www.valgrind.org
Or install valgrind-devel or equivalent using your package manager.
Then rerun ./configure ===============================================================================
TESTING: checkLib from config.packages.BlasLapack(config/BuildSystem/config/packages/BlasLapack.py:106) *******************************************************************************
UNABLE to CONFIGURE with GIVEN OPTIONS (see configure.log for details):
-------------------------------------------------------------------------------
You set a value for --with-blas-lapack-lib=<lib>, but ['/opt/intel/compilers_and_libraries_2018.1.163/linux/mkl/lib/intel64/libmkl_scalapack_lp64.a', '-Wl,--start-group', '/opt/intel/compilers_and_libraries_2018.1.163/linux/mkl/lib/intel64/libmkl_intel_lp64.a', '/opt/intel/compilers_and_libraries_2018.1.163/linux/mkl/lib/intel64/libmkl_intel_thread.a', '/opt/intel/compilers_and_libraries_2018.1.163/linux/mkl/lib/intel64/libmkl_core.a', '/opt/intel/compilers_and_libraries_2018.1.163/linux/mkl/lib/intel64/libmkl_blacs_intelmpi_lp64.a', '-Wl,-zmuldefs,--end-group', '-lpthread', '-lm', '-ldl', '/opt/intel/compilers_and_libraries_2018.1.163/linux/mkl/lib/intel64/libmkl_scalapack_lp64.a', '-Wl,--start-group', '/opt/intel/compilers_and_libraries_2018.1.163/linux/mkl/lib/intel64/libmkl_intel_lp64.a', '/opt/intel/compilers_and_libraries_2018.1.163/linux/mkl/lib/intel64/libmkl_intel_thread.a', '/opt/intel/compilers_and_libraries_2018.1.163/linux/mkl/lib/intel64/libmkl_core.a', '/opt/intel/compilers_and_libraries_2018.1.163/linux/mkl/lib/intel64/libmkl_blacs_intelmpi_lp64.a', '-Wl,-zmuldefs,--end-group', '-lpthread', '-lm', '-ldl'] cannot be used
*******************************************************************************

Here are the options I am using for the configure:

export MKLLIBS="$MKLROOT/lib/intel64/libmkl_scalapack_lp64.a -Wl,--start-group $MKLROOT/lib/intel64/libmkl_intel_lp64.a $MKLROOT/lib/intel64/libmkl_intel_thread.a $MKLROOT/lib/intel64/libmkl_core.a $MKLROOT/lib/intel64/libmkl_blacs_intelmpi_lp64.a -Wl,-zmuldefs,--end-group -lpthread -lm -ldl"

../configure \
--enable-dp \
--enable-msgs-comps \
--enable-memory-profile \
--enable-open-mp \
--enable-iotk \
--enable-slepc-linalg \
--with-extlibs-path="/global/homes/v/vatsal/codes/yambo-libs/" \
--with-blas-libs="$MKLLIBS" \
--with-lapack-libs="$MKLLIBS" \
--with-blacs-libs="$MKLLIBS" \
--with-scalapack-libs="$MKLLIBS" \
--with-fft-libs="/usr/common/software/mkl/2017.up2/interfaces/hsw/intel/libfftw3xf_intel.a" \
--with-fft-includedir="-I${MKLROOT}/include/fftw" \
--with-libxc-path="/global/common/sw/cray/cnl6/haswell/libxc/3.0.0/intel/17.0.2.174/uff3mhq" \
--with-netcdf-path="/opt/cray/pe/netcdf/4.4.1.1.3/INTEL/16.0" \
--with-hdf5-path="/opt/cray/pe/hdf5/1.10.0.3/INTEL/16.0" \
--with-mpi-path="/opt/cray/pe/mpt/7.6.2/gni/mpich-intel/16.0/" \
MPIFC=ftn FCFLAGS="-O3 -qopenmp" MPICC=cc CC=icc FC=ifort CPP="cpp -P"

I've also attached the config.log if that can provide any information.

Thanks so much!

[Davide Sangalli]

I think the problem was already experienced compiling PETSC on cori.

A solution was to edit the file

Code: Select all

lib/petsc/Makefile.loc

adding the following configure options:

Code: Select all

         --with-clib-autodetect=0 \
         --with-cxxlib-autodetect=0 \
         --with-fortranlib-autodetect=0 \

Hope it is still working.

Best,
D.

[vjhalani]

Unfortunately no
(Also, did you mean lib/petsc/Makefile.loc and not lib/petscf/Makefile.loc?)

For reference this is what I did: add the configure options to "lib/petsc/Makefile.loc" under AUXFLAGS
then "make distclean | ./configure ... | make yambo"

Then I still get the same error.

Vatsal

[Davide Sangalli]

did you mean lib/petsc/Makefile.loc and not lib/petscf/Makefile.loc?

Yeah I meant "lib/petsc/Makefile.loc" (I'll correct it in the original message)

Unfortunately no

For reference this is what I did: add the configure options to "lib/petsc/Makefile.loc" under AUXFLAGS
then "make distclean | ./configure ... | make yambo"

Then I still get the same error.

That's strange.
Last time this was the fix which solved the problem. I pushed it in the repo and the compilation on cori worked.
Than I had to remove it because it broke the compilation on many other machines.
I'll further check

[Davide Sangalli]

Can you paste here the edited Makefile.loc and attach the new config.log ?

Best,
D.

[vjhalani]

Here is the edited Makefile.loc and I've attached the config.log

#
#===============================
# Yambo package
#===============================
#
include ../../config/setup
include ../archive/package.list
#
LIBNAME=libpetsc.a
LIBPATH=$(libs_prefix)/$(fc_kind)/${fc}/${yprecision}
LIBRARY=$(LIBPATH)/lib/$(LIBNAME)
#
PACKAGE=$(pkgname_petsc)
TARBALL=$(tarball_petsc)

#
# redefinitions
#
PETSC_ARCH=yambo_${yprecision}_complex
AUXFLAGS=--prefix=$(LIBPATH) \
--PETSC_ARCH=${PETSC_ARCH} \
--with-ssl=0 \
--with-x=0 \
--with-cxx=0 \
--with-shared-libraries=0 \
--with-blas-lapack-lib="$(llapack) $(lblas)" \
--with-scalar-type="complex" \
--with-precision="${yprecision}" \
--with-clib-autodetect=0 \
--with-cxxlib-autodetect=0 \
--with-fortranlib-autodetect=0 \
--known-64-bit-blas-indices
#
# MAIN target
#
all: $(LIBRARY)
#
uncompress-stamp:
( cd ../archive ; if ! test -e $(TARBALL) ; then \
cp Makefile.loc Makefile ; $(make) $(TARBALL) ; fi )
gunzip < ../archive/$(TARBALL) | ../../config/missing --run tar xf -
touch uncompress-stamp

configure-stamp: uncompress-stamp
if test -d $(PACKAGE) ; then \
cd $(PACKAGE); \
unset PETSC_DIR; \
unset PETSC_ARCH; \
./configure $(AUXFLAGS) \
--with-cc=$(cc) \
--with-fc=$(fc) ; \
fi
touch configure-stamp

package-ready-stamp: uncompress-stamp configure-stamp
if test -d $(PACKAGE) ; then \
( cd $(PACKAGE); $(make) -j1 PETSC_DIR=$(PWD)/$(PACKAGE) PETSC_ARCH=$(PETSC_ARCH) ) ; fi
#
touch package-ready-stamp

$(LIBRARY): uncompress-stamp configure-stamp package-ready-stamp
if test -d $(PACKAGE) ; then \
( cd $(PACKAGE); $(make) PETSC_DIR=$(PWD)/$(PACKAGE) PETSC_ARCH=$(PETSC_ARCH) install ) ; fi
#
chmod u+x $(LIBPATH)/lib/*petsc*.a

#
# cleaning
#
clean:
@if test -d $(PACKAGE) ; then ( cd $(PACKAGE); rm -rf $(PETSC_ARCH) ) ; fi
@- rm -rf package-ready-stamp configure-stamp

clean_all:
@if test -d $(PACKAGE) ; then ( rm -rf $(PACKAGE) ) ; fi
@- rm -rf uncompress-stamp

[vjhalani]

Another thing, it was suggested that I try linking to already compiled PETSc and SLEPc libraries on cori. When doing that though (using v3.9.1), I get the following error when compiling:

mod_interfaces.f90(1703): error #6457: This derived type name has not been declared. [TMAT]
type(tMat) :: A ! matrix to diagonalize of stype from slepc
----------^
compilation aborted for mod_interfaces.f90 (code 1)
Makefile:176: recipe for target 'mod_interfaces.o' failed

I'm assuming issues like this though are why yambo prefers to download a different version of petsc (3.7.4)?

Vatsal

[Davide Sangalli]

Indeed it would be better to link the pre-compiled libraries which are likely more efficient.
Unfortunately we still have problems linking external petsc and slepc. The detection is not working properly and should be fixed.
Thus yambo configure accepts also options which may not work in practice.
It's not related to the petsc version, but likely to non proper linking of the external libraries
It can be fixed in principle properly specifying headers and libraries manually, but the process maybe cumbersome.

For the log, sorry, I meant the log of the petsc library generated while compiling the internal ones

Code: Select all

lib/petsc/petsc-3.7.4/configure.log

Best,
D.

[vjhalani]

Ah ok I see... I will see if I can work with NERSC staff on it in parallel.

I've attached the compressed petsc configure.log