MEX compilation with external libraries errors - .lib needed?

I want to call external C code (in particular, MPIR libraries) via mex on my (Windows 7 64bit) machine.

The gcc compilation outside Matlab works fine, but translating it into a workable mex compilation command gives me mex errors for seven days now. The compiler complains that it does not see any ".lib" files, and says, upon adding "-lmpir", as is required in the gcc compilation, that it looks for a file "libmpir.lib" which it does not find, and which also I do not see anywhere in the MPIR directories.

Should such files be there after a successful installation (i.e. do I have to set the MPIR build options differently)? What are there are the files mpir.h, libmpir.la, libmpir.dll.a, libmpir-3.dll.def, among many others, but eg. no file "libmpir.lib", what is mentioned in the mex documentation. The only file ending on ".lib" or ".dll" is "libmpir-16.dll" in the mpir-2.7.0/.libs directory.

I conjecture this because on this post, you can read "I was trying to link my program with ftd2xx.dll. However, for MEX files you need to link them with lib library, ftd2xx.lib. So I found and downloaded lib version of the library..."

Details and settings:

mex compile commands I tried:

mex -IC:/MPIR/mpir-2.7.0/ -LC:/MPIR/mpir-2.7.0/  -LC:/MPIR/mpir-2.7.0/.libs  -LC:/MPIR/mpir-2.7.0/mpf/.libs -LC:/MPIR/mpir-2.7.0/mpf/ mexlib.cpp myfile.cpp 
mex -IC:/MPIR/mpir-2.7.0/ -LC:/MPIR/mpir-2.7.0/  -LC:/MPIR/mpir-2.7.0/.libs  -LC:/MPIR/mpir-2.7.0/mpf/.libs -LC:/MPIR/mpir-2.7.0/mpf/ -lmpir mexlib.cpp myfile.cpp 
mex -IC:/MPIR/mpir-2.7.0/ -LC:/MPIR/mpir-2.7.0/  -LC:/MPIR/mpir-2.7.0/.libs  -LC:/MPIR/mpir-2.7.0/mpf/.libs -LC:/MPIR/mpir-2.7.0/mpf/ -lmpir.dll mexlib.cpp myfile.cpp

Here is mexlib.cpp:

#include "mex.h"
#include "mpir.h"
#include "myfile.h"
#include <stdio.h>

void mexFunction(
     int          nlhs,
     mxArray      *plhs[],
     int          nrhs,
     const mxArray *prhs[]
     )
{
    double a;
    a        = mxGetScalar(prhs[0]);        /* create pointer to the real data in the input arguments  */
    myfile(a);                              /* call the computational routine */    
    return;    
}

and here is myfile.cpp:

#include "mex.h"
#include "mpir.h"
#include "myfile.h"
#include <stdio.h>

void myfile(double a)
{
    mpf_t b;
    mp_bitcnt_t bct = 350;
    double f = 10.0, g = 3.0;

      mpf_set_default_prec(bct);  
      mpf_init(b);
      mpf_set_d (b, f);
      gmp_printf("b is %.*Ff \n",120,b );
      mpf_clear(b);      
  }

Or is this a problem from wrong flags with mex? Or should I try to install the library (MPIR) differently, such that .lib files are produced?

I have R2015a and also tried it with R2011b. Thank you.

Answer 1

[dpb]

Haven't ever used it myself, but...

The web site installation says...

"MPIR has an autoconf/automake/libtool based conguration system. ... Important note: by default MPIR produces libraries named libmpir, etc., and the header file mpir.h."

If you can compile (and link) externally, then the needed libraries do exist somewhere; your problem is that mex setup doesn't know about external non-TMW libraries and so it doesn't set the LIB environment variable to include that location (nor the INCLUDE path to find header files if they're not in local directory either).

To build this way under mex, you'll have to modify the default mexopts.bat file manually to include those locations needed or set them in an autostart routine so they're globally already included in the startup. Remember that mex spawns a new console process to execute under and that will be a virgin environment that only has those environment variables set that are either explicitly set in the batch file itself or were inherited from the parent.

ADDENDUM

Or, you could use the $COMPFLAGS$/$LINKFLAGS$ option variables when you call mex but that means specifying them every time which is why I suggested first to either modify the mexopts.bat file or add the flags to the base environment if you're going to be using this library a lot.

[Attila Kolekrov]

I tried this suggestion: I added to "LIB" the paths MPIRROOT, MPIRROOT/.libs and to "INCLUDE" also MPIRROOT, where MPIRROOT corresponds to the MPIR installation C:/MPIR/mpir-2.7.0 . Nothing changed - still the same error.

I tried another suggestion: Since mex looks for library files named "lmpir.lib" when I add the -lmpir flag in the mex command, and a file "libmpir.lib" or "libmpir.dll" is not present anywhere in the MPIR directories, I tried to convert the existing "libmpir-16.dll" to a "libmpir-16.lib" file using the MS lib tool; then using the flag "-lmpir-16" gave the same error message "...unresolved external symbol...", however.

When using the flag "-lmpir", it said

"MEX could not find the library "mpir" specified with -l option. MEX looked for a file with one of the names: mpir.lib, libmpir.lib ." Thanks.

[dpb]

Again, if, as you say, you can build and link externally, your mission (should you choose to undertake it) is to find the specific librar[y|ies] needed and make them available to the linker via one or the other options available with mex as outlined above.

What, specifically, that is is indeterminate from here (I can't see your terminal), but given the presumption your initial statement is so, the library must exist somewhere (and the install doc's says if you install per the default it will build libraries).

Now, if you can't actually link outside mex because it can't find the librar[y|ies] then there's certainly no hope that it's going to build under mex, either.

If, indeed, the library isn't there at all, that would imply two things--

1. You can't build outside mex either, and
2. The install didn't succeed in building the libraries it says it should.

If that's the case, go back to the installation and get that working. I presume there are probably some test case examples available to ensure things are working correctly after installation; until those work, you're spittin' in the wind re: mex.

[dpb]

Oh, you are using the -verbose option when using mex, correct? If not, do so and it will echo the commands passed to the compiler linker and you can thereby see exactly what the values of the environment variables seen by the batch file are.

[Attila Kolekrov]

Yes I do. In the following I am pasting the output after using mex with "-v". Which values would I have to set/change?

->    Options file           = C:\Users\kolekrov\AppData\Roaming\MathWorks\MATLAB\R2011b\mexopts.bat 
...
->    Link directives: 
       LINKFLAGS           = ... /LIBPATH:"C:\Program Files\MATLAB\R2011b\extern\lib\win64\microsoft" libmx.lib libmex.lib libmat.lib ... kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib uuid.lib odbc32.lib odbccp32.lib ...

...

I can build and link the code externally (in MinGW) (with

gcc  mytest.c -Lc:/MPIR/mpir-2.7.0/.libs -lmpir  -o mytest) ...

How exactly should I edit the COMPFLAGS or LINKFLAGS lines in mexopt.bat? Thanks!

[dpb]

OK I trimmed the mex output down to the LIBFLAGS library path and libraries and compare to the LIB gcc switches. NB: w/ gcc you've specified

-Lc:/MPIR/mpir-2.7.0/.libs -lmpir

which library location and library are missing in mexopts.bat

[Attila Kolekrov]

Please clarify your comment. How should the COMPFLAGS or LINKFLAGS lines in mexopt.bat look like? Should "C:/MPIR/mpir-2.7.0/.libs" be added in the 'set LINKFLAGS' line?

BTW, are you a list administrator or how did you remove the rest of my comment? It was much longer and contained the entire mexopts.bat file.

[dpb]

What's to clarify? As is clear in comparing the library path and libraries passed to LINK you're missing the reference to MPIR. Either edit the mexopts.bat file or add the missing reference via the optional flags -L for path and -l for the library. I'd recommend creating the revised batch file as would assume you'll be doing a compilation more than once and specifying the switches every time gets old. See doc mex and the section on "MEX script switches" for more details.

As for the other comment, when a contributor gets enough of a "Reputation" one can edit other questions and answers beyond one's own. I did this to make it simpler to see where the discrepancy in the two cases is.

BTW, you likely don't need that great big long list of Windoes libraries; not sure why they're all there, but having ones you don't need listed doesn't hurt; it's missing the one in particular that you do need that's the real problem.

[dpb]

BTW, I routinely do NOT crop a posting, simply reformat code and/or clean up obvious grammatical kinds of errors but in the present case it seemed appropriate to remove the superfluous clutter since you were apparently having such difficulty in "seeing the forest for the trees" in picking out what was the key difference in what worked and what didn't.

[Attila Kolekrov]

I meant: what is the syntax to edit the LINKFLAGS variable in the mexopts.bat file?

I tried now adding "C:/MPIR/mpir-2.7.0/.libs" to the 'set LINKFLAGS' line in there, which I was asking, but it did not change anything. So I need a documentation of the syntax of the mexopts.bat file.

As stated initially, the mex commands I tried contained the library name and location, "-LC:/MPIR/mpir-2.7.0/ -LC:/MPIR/mpir-2.7.0/.libs -lmpir". It apparently does find the library. Upon other hints I created the "libmpir-16.lib" (stub?) file from the (only existing dll) file "libmpir-16.dll", and now no error complaining about mex not finding the mpir library shows up. The problem is now only the compiler error

"myfile.obj : error LNK2019: unresolved external symbol __imp___gmpf_init referenced in function "void __cdecl myfile(double)" ",

where mexlib.c is the mex-file (interface to c code) and myfile.c uses some basic mpir commands (mpf_init, mpf_set, mpf_clear).

It seems to me that the exported functions (like mpf_init) are listed in some unsupported format in the libmpir-16.dll (and now libmpir-16.lib) , because there are functions listed "__gmpf_set" and "__gmpf_init = ___gmpf_set_prec_raw", but not in the format "__imp___gmpf_init" what the mex compiler error reported. Thus the dll would have been created in a way unsuitable for use with mex and/or my C compiler (MS SDK 7.1).

Thus my main route of thought is: Is there a way to build the dlls in a way supported by mex?

Re editing: Alright, I didn't know that.

[dpb]

"xyntax [of] LINKFLAGS variable in the mexopts.bat file?"

If you look at the [compiler]OPTS.BAT file (where [compiler] is the shorthand name TMW uses for the compiler you're using, that's the base file mex -setup uses when you run the setup option. Setup then copies that file to the $MATLAB$ location for your installation and renames it MEXOPTS.BAT and that then is the file that mex uses for a given compilation.

If you edit that file, you'll see a LINKFLAGS variable that contains the LIBPATH for the default libraries and the list of those; use that pattern to add the additional path you need for LIPMR and the base library it needs.

Again, alternatively, you can use the -L and -l options each time you invoke the base mex opts file.

The other issue appears to be one of name-mangling. I can't really tell from the snippets you've given where what is coming from but it doesn't seem as though there should be any need for any library other than LIPMR if you can build and link with the shown gcc line.

I don't "know nuthink!" much at all about the C compilers (I'm a Fortran guy) but I'm certain there's a way to build a working mex file; it is just a DLL and I'm certain gcc or whatever C compiler you're using can build a DLL.

I don't know what "other hints" you're speaking of, but looking at the gcc compile and link step I don't see any reference to any other libraries so I'm thinking that is a red herring and the problem is you've got a name-mangling issue that you're getting an extra underscore probably.

Have you successfully been able to build and run one or the sample mex routines first? If so, ensure that the compilation switches for that routine are the same as the ones used in the compilation of the additional code.

[dpb]

ADDENDUM

Note that in the previous comment re: syntax for mex -- all that the mex file does is dispatch a batch file with the various commands for the compiler and linker set in environment variables to be used by them. Hence, the question isn't one of "*mex* syntax" at all, but what is the form and information needed by the compiler and linker to build the desired output?

One of the drawbacks of the TMW mex route is that it makes what is basically a simple process into something that seems totally arcane and unique as though there was some real magic going on behind the scenes. It isn't; it is as the above says simply a front end pseudo-editor to create a set of switches and list of source and various object a library files for the selected compiler and linker. There's a non-default naming convention for the extension but that's basically it as for anything out of the ordinary.

Hence, you should be able to use the command line tools and directly compile and then link the files (not even worrying about the output DLL name initially). When you can build that, then you know the settings required to be incorporated into the appropriate xxxOPTS.BAT file which would then be copied over by mex -setup to the MEXOPTS.BAT file that is the one that mex actually invokes after setup.

The process is explained in more detail at the end of the chapter on external interfaces where it does actually talk some about what to do to use a non-supported compiler; there it outlines the actual calling process and subsequently the way to build a mex file so that you can get a better appreciation of what it is you're trying to accomplish rather than just trying "hit'n miss" tactics not really knowing what is the objective.

[Attila Kolekrov]

Ok, thanks again, this seems too complicated to be solved by setting a path or adding flags to the mex command. I guess the libs are creating in a way that mex does not understand. I will try to find some detailed mex documentation; the one in the help does not explain this issue in a depth sufficient for this issue. Best.

[dpb]

No, if you can build from the command line, you can build a mexopts.bat file that does the same thing and then use mex.

The key is you have to progress logically and figure out how to build the DLL with that additional library after you've built a working demonstration mex file without it. At that point it really should be simply adding the reference to the additional library.