Re: numpy with icc/MKL

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: numpy with icc/MKL

rex-7
> No, they heavily changed how to link against mkl in 10. There is a whole
> chapter about it in the releases notes.

Yes, I read it, but it appears to me that the new layer libraries
are an option, and that the legacy link format still works. From
chapter 3:

"Pure layered libraries give more flexibility to choose the appropriate combination of
libraries but do not have backward compatibility by library names in link lines. Dummy
libraries are introduced to provide backward compatibility with earlier version of Intel MKL,
which did not use layered libraries.
Dummy libraries do not contain any functionality, but only dependencies on a set of layered
libraries. Placed in a link line, dummy libraries enable omitting dependent layered libraries,
which will be linked automatically. Dummy libraries contain dependency on the following
layered libraries (default principle):
·    Interface: Intel, LP64
·    Threading: Intel compiled
·    Computational: the computation library.
So, if you employ the above interface and use OpenMP* threading provided by the Intel®
compiler, you may not change your link lines."

> I don't have the same thing at all. You have many more libraries than I
> have, and that's because you were using the intel compiler, I think
> (libguide is from intel cc, imf is also used by icc and ifort).

>> Remove the installer numpy AND the build directory, and retry building
>> numpy with the mkl.

I always remove the build directory (if I forget the much faster
compilation reminds me). Do you mean remove the installed numpy?
Did that, built numpy again, and it fails numpy.test() exactly as before.

I changed site.cfg to:

[mkl]
library_dirs = /opt/intel/mkl/10.0.3.020/lib/32
lapack_libs = mkl_lapack
mkl_libs = mkl, mkl_vml_p4m, guide

(the vml is appropriate for my CPU). The build log shows:

F2PY Version 2_5189
blas_opt_info:
blas_mkl_info:
  FOUND:
    libraries = ['mkl', 'mkl_vml_p4m', 'guide', 'pthread']
    library_dirs = ['/opt/intel/mkl/10.0.3.020/lib/32']
    define_macros = [('SCIPY_MKL_H', None)]
    include_dirs = ['/opt/intel/mkl/10.0.3.020/include']

  FOUND:
    libraries = ['mkl', 'mkl_vml_p4m', 'guide', 'pthread']
    library_dirs = ['/opt/intel/mkl/10.0.3.020/lib/32']
    define_macros = [('SCIPY_MKL_H', None)]
    include_dirs = ['/opt/intel/mkl/10.0.3.020/include']

lapack_opt_info:
lapack_mkl_info:
mkl_info:
  FOUND:
    libraries = ['mkl', 'mkl_vml_p4m', 'guide', 'pthread']
    library_dirs = ['/opt/intel/mkl/10.0.3.020/lib/32']
    define_macros = [('SCIPY_MKL_H', None)]
    include_dirs = ['/opt/intel/mkl/10.0.3.020/include']

  FOUND:
    libraries = ['mkl_lapack', 'mkl', 'mkl_vml_p4m', 'guide', 'pthread']
    library_dirs = ['/opt/intel/mkl/10.0.3.020/lib/32']
    define_macros = [('SCIPY_MKL_H', None)]
    include_dirs = ['/opt/intel/mkl/10.0.3.020/include']

  FOUND:
    libraries = ['mkl_lapack', 'mkl', 'mkl_vml_p4m', 'guide', 'pthread']
    library_dirs = ['/opt/intel/mkl/10.0.3.020/lib/32']

So it's finding the vector math library, but checking with ldd
shows that of all the *.so libs in site-packages/numpy only
lapack_lite.so is using vml. Any thoughts on why none of the other
libs are using it?

ldd /usr/local/lib/python2.5/site-packages/numpy/linalg/lapack_lite.so
        linux-gate.so.1 =>  (0xffffe000)
        libmkl_lapack.so => /opt/intel/mkl/10.0.3.020/lib/32/libmkl_lapack.so (0xb7a48000)
        /opt/intel/mkl/10.0.3.020/lib/32/libmkl_intel.so (0xb790a000)
        /opt/intel/mkl/10.0.3.020/lib/32/libmkl_intel_thread.so (0xb770d000)
        /opt/intel/mkl/10.0.3.020/lib/32/libmkl_core.so (0xb76a9000)
        libmkl_vml_p4m.so => /opt/intel/mkl/10.0.3.020/lib/32/libmkl_vml_p4m.so (0xb73bd000)
        libguide.so => /opt/intel/mkl/10.0.3.020/lib/32/libguide.so (0xb735a000)
        libpthread.so.0 => /lib/tls/i686/cmov/libpthread.so.0 (0xb732f000)
        libimf.so => /opt/intel/fc/10.1.015/lib/libimf.so (0xb70ff000)
        libm.so.6 => /lib/tls/i686/cmov/libm.so.6 (0xb70da000)
        libgcc_s.so.1 => /lib/libgcc_s.so.1 (0xb70cf000)
        libintlc.so.5 => /opt/intel/fc/10.1.015/lib/libintlc.so.5 (0xb708c000)
        libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0xb6f5a000)
        libdl.so.2 => /lib/tls/i686/cmov/libdl.so.2 (0xb6f56000)
        /lib/ld-linux.so.2 (0x80000000)

As you can see from the above, it uses libmkl_vml_p4m, but it's
the only *.so that does. It would be interesting to see the output
of these commands below on a box running BlAS and Lapack to see how many
of the *.so libs use them.

ldd /usr/local/lib/python2.5/site-packages/numpy/fft/fftpack_lite.so
ldd /usr/local/lib/python2.5/site-packages/numpy/lib/_compiled_base.so
ldd /usr/local/lib/python2.5/site-packages/numpy//core/multiarray.so
ldd /usr/local/lib/python2.5/site-packages/numpy//core/_dotblas.so
ldd /usr/local/lib/python2.5/site-packages/numpy//core/_sort.so
ldd /usr/local/lib/python2.5/site-packages/numpy/core/scalarmath.so
ldd /usr/local/lib/python2.5/site-packages/numpy/core/umath.so
ldd /usr/local/lib/python2.5/site-packages/numpy/random/mtrand.so
ldd /usr/local/lib/python2.5/site-packages/numpy/numarray/_capi.so

> > Interestingly, the above shows libmkl_lapack.so is being used even
> > though it is not in the site.cfg. Apparently, mkl and guide are
> > sufficient in site.cfg.  

> I am not sure I understand: there is a mkl_lapack section in the
> site.cfg, and you need it.

Sorry, I'm going blind from fatigue. :(

> > So why do Erik and I get failures (with both gcc & icc) when MKL
> > is used and you don't?

> I don't know. I would ask Intel about this error if the above does not
> work, maybe you did not install it correctly, or there was a bug in your
> version (my version is a bit more recent, I downloaded it a few days
> ago).

In your list post you show mkl/10.0.1.014/. I'm using 10.0.3.020,
but I've tried an older 10.0.X version with no better luck. BTW,
my build runs, i.e., I can run some programs that use NumPy.

I have posted to the Intel MKL forum. No responses yet.

-rex
I am not goin' to buy my kids an encyclopedia. Let them walk to school the way I did.
_______________________________________________
Numpy-discussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: numpy with icc/MKL

cdavid
rex wrote:
>
> I always remove the build directory (if I forget the much faster
> compilation reminds me). Do you mean remove the installed numpy?
>  

Yes.

> Did that, built numpy again, and it fails numpy.test() exactly as before.
>
> I changed site.cfg to:
>
> [mkl]
> library_dirs = /opt/intel/mkl/10.0.3.020/lib/32
> lapack_libs = mkl_lapack
> mkl_libs = mkl, mkl_vml_p4m, guide
>  

Please follow exactly my instruction, otherwise, we cannot compare what
we are doing: use exactly the same site.cfg as me.

>
> ldd /usr/local/lib/python2.5/site-packages/numpy/linalg/lapack_lite.so
>         linux-gate.so.1 =>  (0xffffe000)
>         libmkl_lapack.so => /opt/intel/mkl/10.0.3.020/lib/32/libmkl_lapack.so (0xb7a48000)
>         /opt/intel/mkl/10.0.3.020/lib/32/libmkl_intel.so (0xb790a000)
>         /opt/intel/mkl/10.0.3.020/lib/32/libmkl_intel_thread.so (0xb770d000)
>         /opt/intel/mkl/10.0.3.020/lib/32/libmkl_core.so (0xb76a9000)
>         libmkl_vml_p4m.so => /opt/intel/mkl/10.0.3.020/lib/32/libmkl_vml_p4m.so (0xb73bd000)
>         libguide.so => /opt/intel/mkl/10.0.3.020/lib/32/libguide.so (0xb735a000)
>         libpthread.so.0 => /lib/tls/i686/cmov/libpthread.so.0 (0xb732f000)
>         libimf.so => /opt/intel/fc/10.1.015/lib/libimf.so (0xb70ff000)
>         libm.so.6 => /lib/tls/i686/cmov/libm.so.6 (0xb70da000)
>         libgcc_s.so.1 => /lib/libgcc_s.so.1 (0xb70cf000)
>         libintlc.so.5 => /opt/intel/fc/10.1.015/lib/libintlc.so.5 (0xb708c000)
>         libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0xb6f5a000)
>         libdl.so.2 => /lib/tls/i686/cmov/libdl.so.2 (0xb6f56000)
>         /lib/ld-linux.so.2 (0x80000000)
>  

You still have libraries linked to intel compilers (libintlc, libimf).
If you remove both the build directory and installed numpy, I don't why
those are used. As a last resort, can you put aside /opt/intel/fc
temporally ?

> As you can see from the above, it uses libmkl_vml_p4m, but it's
> the only *.so that does. It would be interesting to see the output
> of these commands below on a box running BlAS and Lapack to see how many
> of the *.so libs use them.
>
> ldd /usr/local/lib/python2.5/site-packages/numpy/fft/fftpack_lite.so
> ldd /usr/local/lib/python2.5/site-packages/numpy/lib/_compiled_base.so
> ldd /usr/local/lib/python2.5/site-packages/numpy//core/multiarray.so
> ldd /usr/local/lib/python2.5/site-packages/numpy//core/_dotblas.so
> ldd /usr/local/lib/python2.5/site-packages/numpy//core/_sort.so
> ldd /usr/local/lib/python2.5/site-packages/numpy/core/scalarmath.so
> ldd /usr/local/lib/python2.5/site-packages/numpy/core/umath.so
> ldd /usr/local/lib/python2.5/site-packages/numpy/random/mtrand.so
> ldd /usr/local/lib/python2.5/site-packages/numpy/numarray/_capi.so
>  

Only _dotblas and lapack_lite use blas/lapack on numpy, AFAIK. They may
be linked to other extensions, but won't be used.

> In your list post you show mkl/10.0.1.014/. I'm using 10.0.3.020,
> but I've tried an older 10.0.X version with no better luck. BTW,
> my build runs, i.e., I can run some programs that use NumPy.
>
>  

You are not using the free version, right ? The problem is that the MKL
error has no clear meaning, and google did not return anything
meaningful. Maybe your environment is corrupted in some way, because
something is likely to influence how MKL is initialized. But what
exactly, I have no idea.

cheers,

David
_______________________________________________
Numpy-discussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: numpy with icc/MKL

rex-7
> Please follow exactly my instruction, otherwise, we cannot compare what
> we are doing: use exactly the same site.cfg as me.

OK, I used the same MKL version you did (10.0.1.014), the same
site.cfg, and set .bashrc to do:

source /opt/intel/mkl/10.0.1.014/tools/environment/mklvars32.sh

and compiled 1.1.0rc1 with:

python setup.py config build_clib build_ext install \
 prefix=/usr/local/ > buildXX

Then I ran:

# python
Python 2.5 (release25-maint, Dec  9 2006, 14:35:53)
[GCC 4.1.2 20061115 (prerelease) (Debian 4.1.1-20)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
>>> numpy.test()
Numpy is installed in /usr/local/lib/python2.5/site-packages/numpy
Numpy version 1.1.0rc1
Python version 2.5 (release25-maint, Dec  9 2006, 14:35:53) [GCC 4.1.2 20061115 (prerelease) (Debian 4.1.1-20)]
  Found 18/18 tests for numpy.core.defmatrix
  Found 3/3 tests for numpy.core.memmap
  Found 283/283 tests for numpy.core.multiarray
  Found 70/70 tests for numpy.core.numeric
  Found 36/36 tests for numpy.core.numerictypes
  Found 12/12 tests for numpy.core.records
  Found 0/0 tests for numpy.lib.format
  Found 53/53 tests for numpy.lib.function_base
  Found 5/5 tests for numpy.lib.getlimits
  Found 6/6 tests for numpy.lib.index_tricks
  Found 15/15 tests for numpy.lib.io
  Found 1/1 tests for numpy.lib.machar
  Found 4/4 tests for numpy.lib.polynomial
  Found 49/49 tests for numpy.lib.shape_base
  Found 15/15 tests for numpy.lib.twodim_base
  Found 43/43 tests for numpy.lib.type_check
  Found 1/1 tests for numpy.lib.ufunclike
  Found 89/89 tests for numpy.linalg
  Found 94/94 tests for numpy.ma.core
  Found 15/15 tests for numpy.ma.extras
  Found 7/7 tests for numpy.random
  Found 16/16 tests for numpy.testing.utils
  Found 0/0 tests for __main__
..............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
 ..............
----------------------------------------------------------------------
Ran 1004 tests in 1.429s

OK
<unittest._TextTestResult run=1004 errors=0 failures=0>
>>>

So it works. :)

ldd /usr/local/lib/python2.5/site-packages/numpy/linalg/lapack_lite.so
        linux-gate.so.1 =>  (0xffffe000)
        libmkl_lapack.so => /opt/intel/mkl/10.0.1.014/lib/32/libmkl_lapack.so (0xb7a53000)
        /opt/intel/mkl/10.0.1.014/lib/32/libmkl_intel.so (0xb791f000)
        /opt/intel/mkl/10.0.1.014/lib/32/libmkl_intel_thread.so (0xb7735000)
        /opt/intel/mkl/10.0.1.014/lib/32/libmkl_core.so (0xb76d9000)
        libguide.so => /opt/intel/mkl/10.0.1.014/lib/32/libguide.so (0xb767f000)
        libpthread.so.0 => /lib/tls/i686/cmov/libpthread.so.0 (0xb7653000)
        libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0xb7522000)
        libdl.so.2 => /lib/tls/i686/cmov/libdl.so.2 (0xb751e000)
        /lib/ld-linux.so.2 (0x80000000)

No more links to icc libs.


>> ldd /usr/local/lib/python2.5/site-packages/numpy/linalg/lapack_lite.so
>>         linux-gate.so.1 =>  (0xffffe000)
>>         libmkl_lapack.so => /opt/intel/mkl/10.0.3.020/lib/32/libmkl_lapack.so (0xb7a48000)
>>         /opt/intel/mkl/10.0.3.020/lib/32/libmkl_intel.so (0xb790a000)
>>         /opt/intel/mkl/10.0.3.020/lib/32/libmkl_intel_thread.so (0xb770d000)
>>         /opt/intel/mkl/10.0.3.020/lib/32/libmkl_core.so (0xb76a9000)
>>         libmkl_vml_p4m.so => /opt/intel/mkl/10.0.3.020/lib/32/libmkl_vml_p4m.so (0xb73bd000)
>>         libguide.so => /opt/intel/mkl/10.0.3.020/lib/32/libguide.so (0xb735a000)
>>         libpthread.so.0 => /lib/tls/i686/cmov/libpthread.so.0 (0xb732f000)
>>         libimf.so => /opt/intel/fc/10.1.015/lib/libimf.so (0xb70ff000)
>>         libintlc.so.5 => /opt/intel/fc/10.1.015/lib/libintlc.so.5 (0xb708c000)


> You still have libraries linked to intel compilers (libintlc, libimf).
> If you remove both the build directory and installed numpy, I don't why
> those are used. As a last resort, can you put aside /opt/intel/fc
> temporally ?

They are linked because that build was with the Intel compiler. As
you can see from the new ldd above, those links are not there when
I build with gcc.

> You are not using the free version, right ? The problem is that the MKL
> error has no clear meaning, and google did not return anything
> meaningful. Maybe your environment is corrupted in some way, because
> something is likely to influence how MKL is initialized. But what
> exactly, I have no idea.

AFAIK, there is one MKL version for each release. The only
difference is the licensing.

Next step is to try icc instead of gcc, and if that works, try
the latest MKL (10.0.3.020).

Thanks,

-rex
--
I almost stole another tagline. I'm so ashamed.
_______________________________________________
Numpy-discussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: numpy with icc/MKL

rex-7
> Next step is to try icc instead of gcc, and if that works, try
> the latest MKL (10.0.3.020).

OK, either I've got a corrupted copy of MKL 10.0.3.020, or it has
a problem. Building with icc & MKL 10.0.1.014 works.

Erik, are you reading this? If so, roll back to MKL 10.0.014 and it
should work, both with gcc and icc.

root@c2d0:/usr/local/src/1.1.0rc1# python setup.py config --compiler=intel build_clib --compiler=intel build_ext --compiler=intel install --prefix=/usr/local/ >build25
Running from numpy source directory.
root@c2d0:/usr/local/src/1.1.0rc1# cd ..
root@c2d0:/usr/local/src# python
Python 2.5 (release25-maint, Dec  9 2006, 14:35:53)
[GCC 4.1.2 20061115 (prerelease) (Debian 4.1.1-20)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
>>> numpy.test()
Numpy is installed in /usr/local/lib/python2.5/site-packages/numpy
Numpy version 1.1.0rc1
Python version 2.5 (release25-maint, Dec  9 2006, 14:35:53) [GCC 4.1.2 20061115 (prerelease) (Debian 4.1.1-20)]
  Found 18/18 tests for numpy.core.defmatrix
  Found 3/3 tests for numpy.core.memmap
  Found 283/283 tests for numpy.core.multiarray
  Found 70/70 tests for numpy.core.numeric
  Found 36/36 tests for numpy.core.numerictypes
  Found 12/12 tests for numpy.core.records
  Found 7/7 tests for numpy.core.scalarmath
  Found 16/16 tests for numpy.core.umath
  Found 5/5 tests for numpy.ctypeslib
  Found 5/5 tests for numpy.distutils.misc_util
  Found 2/2 tests for numpy.fft.fftpack
  Found 3/3 tests for numpy.fft.helper
  Found 24/24 tests for numpy.lib._datasource
  Found 10/10 tests for numpy.lib.arraysetops
  Found 1/1 tests for numpy.lib.financial
  Found 0/0 tests for numpy.lib.format
  Found 53/53 tests for numpy.lib.function_base  
  Found 5/5 tests for numpy.lib.getlimits
  Found 6/6 tests for numpy.lib.index_tricks
  Found 15/15 tests for numpy.lib.io
  Found 1/1 tests for numpy.lib.machar
  Found 4/4 tests for numpy.lib.polynomial
  Found 49/49 tests for numpy.lib.shape_base
  Found 15/15 tests for numpy.lib.twodim_base
  Found 43/43 tests for numpy.lib.type_check
  Found 1/1 tests for numpy.lib.ufunclike
  Found 89/89 tests for numpy.linalg
  Found 94/94 tests for numpy.ma.core
  Found 15/15 tests for numpy.ma.extras
  Found 7/7 tests for numpy.random
  Found 16/16 tests for numpy.testing.utils
  Found 0/0 tests for __main__
..............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
 ..............
----------------------------------------------------------------------
Ran 1004 tests in 1.041s

OK
<unittest._TextTestResult run=1004 errors=0 failures=0>
>>>
_______________________________________________
Numpy-discussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpy-discussion
Reply | Threaded
Open this post in threaded view
|

Re: numpy with icc/MKL

Erik Nugent
I'm here... i am rolling back now and will post my results...

e

On Mon, May 19, 2008 at 9:22 AM, rex <[hidden email]> wrote:

>> Next step is to try icc instead of gcc, and if that works, try
>> the latest MKL (10.0.3.020).
>
> OK, either I've got a corrupted copy of MKL 10.0.3.020, or it has
> a problem. Building with icc & MKL 10.0.1.014 works.
>
> Erik, are you reading this? If so, roll back to MKL 10.0.014 and it
> should work, both with gcc and icc.
>
> root@c2d0:/usr/local/src/1.1.0rc1# python setup.py config --compiler=intel build_clib --compiler=intel build_ext --compiler=intel install --prefix=/usr/local/ >build25
> Running from numpy source directory.
> root@c2d0:/usr/local/src/1.1.0rc1# cd ..
> root@c2d0:/usr/local/src# python
> Python 2.5 (release25-maint, Dec  9 2006, 14:35:53)
> [GCC 4.1.2 20061115 (prerelease) (Debian 4.1.1-20)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>>>> import numpy
>>>> numpy.test()
> Numpy is installed in /usr/local/lib/python2.5/site-packages/numpy
> Numpy version 1.1.0rc1
> Python version 2.5 (release25-maint, Dec  9 2006, 14:35:53) [GCC 4.1.2 20061115 (prerelease) (Debian 4.1.1-20)]
>  Found 18/18 tests for numpy.core.defmatrix
>  Found 3/3 tests for numpy.core.memmap
>  Found 283/283 tests for numpy.core.multiarray
>  Found 70/70 tests for numpy.core.numeric
>  Found 36/36 tests for numpy.core.numerictypes
>  Found 12/12 tests for numpy.core.records
>  Found 7/7 tests for numpy.core.scalarmath
>  Found 16/16 tests for numpy.core.umath
>  Found 5/5 tests for numpy.ctypeslib
>  Found 5/5 tests for numpy.distutils.misc_util
>  Found 2/2 tests for numpy.fft.fftpack
>  Found 3/3 tests for numpy.fft.helper
>  Found 24/24 tests for numpy.lib._datasource
>  Found 10/10 tests for numpy.lib.arraysetops
>  Found 1/1 tests for numpy.lib.financial
>  Found 0/0 tests for numpy.lib.format
>  Found 53/53 tests for numpy.lib.function_base
>  Found 5/5 tests for numpy.lib.getlimits
>  Found 6/6 tests for numpy.lib.index_tricks
>  Found 15/15 tests for numpy.lib.io
>  Found 1/1 tests for numpy.lib.machar
>  Found 4/4 tests for numpy.lib.polynomial
>  Found 49/49 tests for numpy.lib.shape_base
>  Found 15/15 tests for numpy.lib.twodim_base
>  Found 43/43 tests for numpy.lib.type_check
>  Found 1/1 tests for numpy.lib.ufunclike
>  Found 89/89 tests for numpy.linalg
>  Found 94/94 tests for numpy.ma.core
>  Found 15/15 tests for numpy.ma.extras
>  Found 7/7 tests for numpy.random
>  Found 16/16 tests for numpy.testing.utils
>  Found 0/0 tests for __main__
> ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
 ..

>  ..............
> ----------------------------------------------------------------------
> Ran 1004 tests in 1.041s
>
> OK
> <unittest._TextTestResult run=1004 errors=0 failures=0>
>>>>
> _______________________________________________
> Numpy-discussion mailing list
> [hidden email]
> http://projects.scipy.org/mailman/listinfo/numpy-discussion
>



--
Erik Nugent
System Administrator
Department of Computer Science
The University of Montana - Missoula
(406) 243-2812 V
(406) 243-5139 F
_______________________________________________
Numpy-discussion mailing list
[hidden email]
http://projects.scipy.org/mailman/listinfo/numpy-discussion