Efficiency decreased when openmp is enabled

[yunhailiseu]

Dear Developers,

Recently I managed to build yambo-3.4.1-rev3187 with openmp support. However, the benchmark showed that enabling openmp will decrease the efficiency. In my test, the "MPI+openmp" version of yambo took 2 hours and 40 minutes to finish, while the pure MPI version took only 2 hours.

The benchmark was performed on an AMD Opteron SMP computer. The two versions of yambo aforementioned were run simultaneously, each with 12 cores.

What may be the reason? I consulted the documentation page, but couldn't find any variables related to openmp. I also tried the "pstree" command and found each yambo process held only one process.

In the attachment are the "config.log" file and job report file for two versions of yambo.

All suggestions are appreciated.

Best,
Yunhai

[andrea.ferretti]

Dear Yunhai,
∫˜µ∫b∫though flag --enable-open-mp is listed in the configure help of yambo 3.4.1, there is not yet any explicit openmp support in that version.
Such explicit omp support is at the moment only in the development version under testing and will be made available with the next release.
The only openmp support you can get at the moment is the one that comes through the use of optimized libraries (eg mkl or alike), but you should not expect to get an extensive scaling from that. Explicit omp tends to improve on the situation but will come out later.

In any case one would like to see some improvement when using openmp, instead you are getting none.
In general there are many reasons for openmp non working well, and one would need some more info to better understand.
From your logs it seems you are running two calculations, each using 12 MPI tasks, one of them also using some OMP threads...
How many threads were you using ? you also mention you were running these two calculations on the same time... what do you mean here ?
on the same node ? how big (how many cores) are the node(s) you were running on ?
This is just to better understand the situation...

take care
Andrea

[yunhailiseu]

Dear Andrea,

Thank you for the quick reply.

The machine where I ran the tests has 48 cores. By then it had only 24 cores available since the other 24 cores had been occupied already by other users. As this machine were also running other tasks at that time, I had to run the two versions of yambo simultaneously in order to minimize the environmental affects, as the disk I/O or inter-process communication of the other tasks may slow down the computer, leading to false efficiency decrease of yambo.

I tried the “pstree –e PID” command to watch into the yambo process started by mpirun and found each yambo process held only one thread. I am not familiar with OpenMP.

As you mentioned above, currently the only OpenMP support is the one that comes through the use of optimized libraries. I think the decrease of efficiency may result from the incorrect linking of MKL, since I linked the sequential version (-lmkl_sequential), not the multi-threaded one (-lmkl_intel_thread).

Anyway, I will fall back to the MPI version and wait for the new release.

Best,
Yunhai