Why do I get a "File format not recognized" error when creating a MEX-file on Solaris with MATLAB?

I am trying to create a MEX-file. I am using MATLAB on a Solaris machine. When I try to create the MEX-file, I receive the following error message at the end of the compilation process (with -v option):

 /tools/mathworks/matlab/6.5/extern/lib/sol2/mexFunction.map: file not recognized: File format not recognized

 collect2: ld returned 1 exit status

     mex: link of 'loadsig.mexsol' failed.

 ??? Error using ==> mex

 Unable to complete successfully

Answer 1

[MathWorks Support Team]

The problem may be caused by a discrepancy between the linker you are using and the default linker options specified in the MEX options file. For a further explanation, open the following file:

1. In MATLAB, type "cd(prefdir)"

2. View the mexopts.sh file, by typing "edit mexopts.sh"

Find the sol2 section of the file. For CC, if the entry is "gcc", MATLAB is using the gcc compiler. Several lines later, there is the LD entry. If this is also "gcc", then MATLAB is using the GNU linker, which may be the cause of the problem you are seeing. The map file is specified differently for the GNU linker as opposed to the Solaris system linker (ld). In many cases the LD entry will be set to $COMPILER; this is normal.

Changing the LDFLAGS variable in the sol2 section of mexopts.sh should fix the problem. The exact setting for the LDFLAGS variable will depend upon the version of MATLAB you are using. In all versions you will need to replace "-M" with "--version-script".

In MATLAB 6.5 (R13) you will change

 LDFLAGS="-shared -Wl,-M,$TMW_ROOT/extern/lib/$Arch/$MAPFILE"

to:

 LDFLAGS="-shared -Wl,--version-script,$TMW_ROOT/extern/lib/$Arch/$MAPFILE"

In MATLAB 7.3 (R2006b) you will change

 LDFLAGS="-shared -Wl,-M,$TMW_ROOT/extern/lib/$Arch/$MAPFILE,-R,$GCC_LIBDIR"

to

 LDFLAGS="-shared -Wl,--version-script,$TMW_ROOT/extern/lib/$Arch/$MAPFILE,-R,$GCC_LIBDIR"

The --version-script option is the GNU linker option for telling the linker to specify a print map while -M is this option for the Solaris ld linker.