Discover MakerZone

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn more

Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply Today

Thread Subject:
Mex and external MKL LAPACK

Subject: Mex and external MKL LAPACK

From: Jonathan Hogg

Date: 7 Apr, 2011 15:30:22

Message: 1 of 5

I'm having difficulty compiling to use an external LAPACK. Despite linking against it in a way I know works for independent programs stack trace clearly shows I am using the libmwlapack (when the program crashes out).

Reason for not using libmwlapack: Don't want to alter code that calls LAPACK, but forcing an ILP64 model as mathworks use for their LAPACK results in significant performance hit for program due to extra (unneeded) memory traffic.

mex command line:
/usr/local/MATLAB/R2010b/bin/mex CC='gcc-4.3' FC='g95 -O3 -Wno=155 -fno-second-underscore' FFLAGS='-fexceptions -fPIC -fno-omit-frame-pointer' LD='g95 -O3 -Wno=155 -fno-second-underscore -fPIC -I/usr/local/MATLAB/R2010b/extern/include' -g -I/usr/local/MATLAB/R2010b/extern/include -largeArrayDims -L/opt/intel/composerxe-2011/mkl/lib/intel64/ -lmkl_gf_lp64 -lmkl_sequential -lmkl_core -lmkl_def -lmkl_gf_lp64 -lmkl_sequential -lmkl_core -lmkl_def -L/usr/local/metis-4.0-shared -lmetis -output hsl_ma87_expert hsl_ma87_expert.f90 hsl_ma87d.o ddeps90.o hsl_ma87z.o zdeps90.o common90.o hsl_matlab.o

valgrind output indicates stack trace of first failure:
==14322== at 0x25678BAA: dpotrf_ (in /home/local/MATLAB/R2010b/bin/glnxa64/mllapack.so)
==14322== by 0x221B97A9: hsl_ma87_double_MP_factor_diag_block (hsl_ma87d.f90:3332)
...

this is clearly caused by passing 32-bit integer arguments rather than 64-bit. problem goes away if compiling with default real as integer*8.

Any idea why its not using the MKL dpotrf? (it seems to use the right BLAS)

nm /opt/intel/mkl/lib/intel64/libmkl_gf_lp64.a | grep -i dpotrf
clapack_dpotrf_work_lp64.o:
0000000000000000 T LAPACKE_dpotrf_work
                 U dpotrf_
clapack_dpotrf_lp64.o:
0000000000000000 T LAPACKE_dpotrf
                 U LAPACKE_dpotrf_work
_dpotrf_lp64.o:
0000000000000000 T DPOTRF
0000000000000000 T DPOTRF_
0000000000000000 T dpotrf
0000000000000000 T dpotrf_
                 U mkl_lapack_dpotrf

Thanks for any ideas you might have,

Jonathan.

Subject: Mex and external MKL LAPACK

From: Christopher Hulbert

Date: 7 Apr, 2011 16:50:18

Message: 2 of 5

On 4/7/11 11:30 AM, Jonathan Hogg wrote:
> I'm having difficulty compiling to use an external LAPACK. Despite linking
> against it in a way I know works for independent programs stack trace clearly
> shows I am using the libmwlapack (when the program crashes out).
>

Did you set the BLAS version before starting matlab? Here is an example of what
I use that seems to work (adjust your paths/libraries as necessary).

export
BLAS_VERSION="../../../../../opt/intel/composerxe-2011/lib/intel64/libiomp5.so
../../../../../opt/intel/composerxe-2011/lib/intel64/libimf.so
../../../../../opt/intel/composerxe-2011/lib/intel64/libirc.so
../../../../../opt/intel/composerxe-2011/lib/intel64/libsvml.so
../../../../../opt/intel/composerxe-2011/mkl/lib/intel64/libmkl_core.so
../../../../../opt/intel/composerxe-2011/mkl/lib/intel64/libmkl_intel_thread.so
../../../../../opt/intel/composerxe-2011/mkl/lib/intel64/libmkl_intel_lp64.so"

Chris

> Thanks for any ideas you might have,
>
> Jonathan.

Subject: Mex and external MKL LAPACK

From: Christopher Hulbert

Date: 7 Apr, 2011 16:58:51

Message: 3 of 5

On 4/7/11 12:50 PM, Christopher Hulbert wrote:
> On 4/7/11 11:30 AM, Jonathan Hogg wrote:
>> I'm having difficulty compiling to use an external LAPACK. Despite linking
>> against it in a way I know works for independent programs stack trace clearly
>> shows I am using the libmwlapack (when the program crashes out).
>>
>
> Did you set the BLAS version before starting matlab? Here is an example of what
> I use that seems to work (adjust your paths/libraries as necessary).
>
> export
> BLAS_VERSION="../../../../../opt/intel/composerxe-2011/lib/intel64/libiomp5.so
> ../../../../../opt/intel/composerxe-2011/lib/intel64/libimf.so
> ../../../../../opt/intel/composerxe-2011/lib/intel64/libirc.so
> ../../../../../opt/intel/composerxe-2011/lib/intel64/libsvml.so
> ../../../../../opt/intel/composerxe-2011/mkl/lib/intel64/libmkl_core.so
> ../../../../../opt/intel/composerxe-2011/mkl/lib/intel64/libmkl_intel_thread.so
> ../../../../../opt/intel/composerxe-2011/mkl/lib/intel64/libmkl_intel_lp64.so"

I forgot to mention the "../../../../.." gets out of the
/opt/matlab/R2007b/bin/glnxa64 directory. You will likely have another
installation location and may need to adjust the number of levels to go up.

Chris

>
> Chris
>
>> Thanks for any ideas you might have,
>>
>> Jonathan.
>

Subject: Mex and external MKL LAPACK

From: Jonathan Hogg

Date: 19 Apr, 2011 12:43:04

Message: 4 of 5

"Jonathan Hogg" <jonathan.removespam.hogg@stfc.ac.uk> wrote in message <inklae$9vb$1@fred.mathworks.com>...
> I'm having difficulty compiling to use an external LAPACK. Despite linking against it in a way I know works for independent programs stack trace clearly shows I am using the libmwlapack (when the program crashes out).
>
> Reason for not using libmwlapack: Don't want to alter code that calls LAPACK, but forcing an ILP64 model as mathworks use for their LAPACK results in significant performance hit for program due to extra (unneeded) memory traffic.
>
> mex command line:
> /usr/local/MATLAB/R2010b/bin/mex CC='gcc-4.3' FC='g95 -O3 -Wno=155 -fno-second-underscore' FFLAGS='-fexceptions -fPIC -fno-omit-frame-pointer' LD='g95 -O3 -Wno=155 -fno-second-underscore -fPIC -I/usr/local/MATLAB/R2010b/extern/include' -g -I/usr/local/MATLAB/R2010b/extern/include -largeArrayDims -L/opt/intel/composerxe-2011/mkl/lib/intel64/ -lmkl_gf_lp64 -lmkl_sequential -lmkl_core -lmkl_def -lmkl_gf_lp64 -lmkl_sequential -lmkl_core -lmkl_def -L/usr/local/metis-4.0-shared -lmetis -output hsl_ma87_expert hsl_ma87_expert.f90 hsl_ma87d.o ddeps90.o hsl_ma87z.o zdeps90.o common90.o hsl_matlab.o
>
> valgrind output indicates stack trace of first failure:
> ==14322== at 0x25678BAA: dpotrf_ (in /home/local/MATLAB/R2010b/bin/glnxa64/mllapack.so)
> ==14322== by 0x221B97A9: hsl_ma87_double_MP_factor_diag_block (hsl_ma87d.f90:3332)
> ...
>
> this is clearly caused by passing 32-bit integer arguments rather than 64-bit. problem goes away if compiling with default real as integer*8.
>
> Any idea why its not using the MKL dpotrf? (it seems to use the right BLAS)
>
> nm /opt/intel/mkl/lib/intel64/libmkl_gf_lp64.a | grep -i dpotrf
> clapack_dpotrf_work_lp64.o:
> 0000000000000000 T LAPACKE_dpotrf_work
> U dpotrf_
> clapack_dpotrf_lp64.o:
> 0000000000000000 T LAPACKE_dpotrf
> U LAPACKE_dpotrf_work
> _dpotrf_lp64.o:
> 0000000000000000 T DPOTRF
> 0000000000000000 T DPOTRF_
> 0000000000000000 T dpotrf
> 0000000000000000 T dpotrf_
> U mkl_lapack_dpotrf
>
> Thanks for any ideas you might have,
>
> Jonathan.

Setting BLAS_VERSION is not an option as the end result must be simple to use for "normal" matlab users.

I eventually got to the bottom of the problem. It is as follows:
dpotrf is defined in libmkl_gf_lp64 and implemented in libmkl_core It calls dgemm which is defined in libmkl_gf_lp64.
So I pick up the LP64 dpotrf but it picks up an ILP64 dgemm.

The only way I found to solve this is to statically link the MKL. You need to repeat the libraries twice to avoid a link error relating to the above problem.

Intel recommend using -Wl,--start-group and -Wl,--end-group to solve this problem, but this is non-trivial to get mex to invoke. Futher, dynamically linking the librraies twice doesn't seem to solve the problem either.

Jonathan.

Subject: Mex and external MKL LAPACK

From: Harrison Smith

Date: 14 Feb, 2012 15:04:44

Message: 5 of 5

I'm having a similar problem, and I think setting the BLAS_VERSION will work for me. However, the 'export' command you suggest doesn't make much sense to me... are those lines separated by carriage returns? semicolons?

Thanks!
-Ben

Christopher Hulbert <cchgroupmail@gmail.com> wrote in message <inkq1e$6hl$1@dont-email.me>...
> On 4/7/11 11:30 AM, Jonathan Hogg wrote:
> > I'm having difficulty compiling to use an external LAPACK. Despite linking
> > against it in a way I know works for independent programs stack trace clearly
> > shows I am using the libmwlapack (when the program crashes out).
> >
>
> Did you set the BLAS version before starting matlab? Here is an example of what
> I use that seems to work (adjust your paths/libraries as necessary).
>
> export
> BLAS_VERSION="../../../../../opt/intel/composerxe-2011/lib/intel64/libiomp5.so
> ../../../../../opt/intel/composerxe-2011/lib/intel64/libimf.so
> ../../../../../opt/intel/composerxe-2011/lib/intel64/libirc.so
> ../../../../../opt/intel/composerxe-2011/lib/intel64/libsvml.so
> ../../../../../opt/intel/composerxe-2011/mkl/lib/intel64/libmkl_core.so
> ../../../../../opt/intel/composerxe-2011/mkl/lib/intel64/libmkl_intel_thread.so
> ../../../../../opt/intel/composerxe-2011/mkl/lib/intel64/libmkl_intel_lp64.so"
>
> Chris
>
> > Thanks for any ideas you might have,
> >
> > Jonathan.

Tags for this Thread

No tags are associated with this thread.

What are tags?

A tag is like a keyword or category label associated with each thread. Tags make it easier for you to find threads of interest.

Anyone can tag a thread. Tags are public and visible to everyone.

Contact us