Upgrade MEX-Files to Use 64-Bit API

MATLAB Support for 64-Bit Indexing

MATLAB® Version 7.3 (R2006b) added support for 64-bit indexing. With 64-bit indexing, you can create variables with up to 248-1 elements on 64-bit platforms. Before Version 7.3, the C/C++ and Fortran API Reference library functions used int in C/C++ and INTEGER*4 in Fortran to represent array dimensions. These types limit the size of an array to 32-bit integers. Simply building and running MEX-files on a 64-bit platform does not guarantee you access to the additional address space. You must update your MEX source code to take advantage of this functionality.

The following changes to the MX Matrix Library support 64-bit indexing:

  • New types, mwSize and mwIndex, enabling large-sized data.

  • Updated MX Matrix Library functions use mwSize and mwIndex types for inputs and outputs. These functions are called the 64-bit API or the large-array-handling API.

  • New -largeArrayDims flag for mex build command enabling use of the 64-bit API.

To help transition your MEX-files to the 64-bit API, MATLAB maintains an interface, or compatibility layer. To build MEX-files with this interface, use the -compatibleArrayDims flag.

    Note:   Only variables representing array size or index value require the mwSize or mwIndex types. The C-language int data type is valid for variables representing, for example, the number of fields or arrays.

MEX Uses 32-Bit API by Default

The mex command uses the -compatibleArrayDims flag (32-bit API) by default. In a future version of MATLAB, the mex command will change to use the large-array-handling API. Then, the -largeArrayDims option will be the default. This topic describes how to upgrade your MEX-files now in preparation for that transition.

Can I Run Existing Binary MEX-Files?

You can run existing binary MEX-files without upgrading the files for use with the 64-bit API. However, unrelated incompatibilities that prevent execution of an existing MEX-file can occur. If your MEX-file does not execute properly, review the MEX Compatibility Considerations topics in the Release Notes for this release. To find MEX topics, check the External Interfaces section of the Compatibility Summary for MATLAB release notes table for each relevant version.

Must I Update Source MEX-Files on 64-Bit Platforms?

If you build MEX-files on 64-bit platforms or write platform-independent applications, you must upgrade your MEX-files when the default changes. To upgrade, review your source code, make appropriate changes, and rebuild using the mex command.

Previous versions of the External Interfaces Release Notes provide instructions for updating your MEX-files. What action you take now depends on whether your MEX-files currently use the 64-bit API. The following table helps you identify your next actions.

State of Your Source CodeNext Action

I do not plan to update my code.

You have chosen to opt out and you must build using the -compatibleArrayDims flag.

I want to update my code. Where do I start?

See How to Upgrade MEX-Files to Use the 64-Bit API.

I use MEX-files, but do not have access to the source code.

Ask the owner of the source code to follow the steps in How to Upgrade MEX-Files to Use the 64-Bit API.

I use third-party libraries.

Ask the vendor if the libraries support 64-bit indexing. If not, you cannot use these libraries to create 64-bit MEX-files. Build your MEX-file using the -compatibleArrayDims flag.

If the libraries support 64-bit indexing, review your source code, following the steps in How to Upgrade MEX-Files to Use the 64-Bit API, and then test.

I updated my code in a previous release.

Review your source code, following the steps in How to Upgrade MEX-Files to Use the 64-Bit API, and then test.

Must I Update Source MEX-Files on 32-Bit Platforms?

There are no changes to building 32-bit MEX-files. However, in a future version of MATLAB, the compatibility layer, with the -compatibleArrayDims flag, might be unsupported and you then would need to upgrade your MEX-files.

If you build MEX-files exclusively on 32-bit platforms, but want to write platform-independent code, you still can upgrade your code. If possible, build on a 64-bit system to validate your changes.

What If I Do Not Upgrade?

On 32-bit platforms, you do not need to make any changes to build MEX-files.

On 64-bit platforms, you can build MEX-files by using the -compatibleArrayDims flag.

On 64-bit platforms, if you do not update your source files and you build without the -compatibleArrayDims flag, the results are unpredictable. One or more of the following could occur:

  • Increased compiler warnings and/or errors from your native compiler

  • Run-time errors

  • Wrong answers

How to Upgrade MEX-Files to Use the 64-Bit API

To review and update MEX-file source code, use the following checklist.

  1. Prepare your code before editing — see Back Up Files and Create Tests.

  2. Iteratively change and test code.

    Before building your MEX-files with the 64-bit API, refactor your existing code by checking for the following conditions:

    1. Update Variables.

    2. Replace Unsupported Functions.

    3. If necessary, Update Fortran Source Code.

    After each change, build and test your code:

  3. Compile using the 64-bit API. To build myMexFile.c, type:

    mex -largeArrayDims myMexFile.c
  4. Resolve failures and warnings — see Resolve -largeArrayDims Build Failures and Warnings.

  5. Compare Results — see Execute 64-Bit MEX-File and Compare Results with 32-Bit Version.

  6. Check memory — see Experiment with Large Arrays.

The following procedures use C/C++ terminology and example code. Fortran MEX-files share the same issues, with more tasks described in Update Fortran Source Code.

Back Up Files and Create Tests

Before adapting your code to handle large arrays, verify the MEX-file works with the traditional 32-bit array dimensions. At a minimum, build a list of expected inputs and outputs, or create a full test suite. To compare the results with the upgraded source code, use these tests. The results should be identical.

Back up all source, binary, and test files.

Update Variables

To handle large arrays, convert variables containing array indices or sizes to use the mwSize and mwIndex types instead of the 32-bit int type. Review your code to see if it contains the following types of variables:

Update Arguments Used to Call Functions in the 64-Bit API

Identify the 64-bit API functions in your code that use the mwSize / mwIndex types. For the list of functions, see Using the 64-Bit API. Search for the variables that you use to call the functions. Check the function signature, shown under the Syntax heading on the function reference documentation. The signature identifies the variables that take mwSize / mwIndex values as input or output values. Change your variables to use the correct type.

For example, suppose that your code uses the mxCreateDoubleMatrix function, as shown in the following statements:

int nrows,ncolumns;
...
y_out = mxCreateDoubleMatrix(nrows, ncolumns, mxREAL);

To see the function signature, type:

doc mxCreateDoubleMatrix

The signature is:

mxArray *mxCreateDoubleMatrix(mwSize m, mwSize n, 
    mxComplexity ComplexFlag)

The type for input arguments m and n is mwSize. Change your code as shown in the table.

Replace:With:
int nrows,ncolumns;
mwSize nrows,ncolumns;

Update Variables Used for Array Indices and Sizes

If your code uses intermediate variables to calculate size and index values, use mwSize / mwIndex for these variables. For example, the following code declares the inputs to mxCreateDoubleMatrix as type mwSize:

mwSize nrows,ncolumns;	/* inputs to mxCreateDoubleMatrix */
int numDataPoints;
nrows = 3;
numDataPoints = nrows * 2;
ncolumns = numDataPoints + 1;
...
y_out = mxCreateDoubleMatrix(nrows, ncolumns, mxREAL);

This example uses the intermediate variable, numDataPoints (of type int), to calculate the value of ncolumns. If you copy a 64-bit value from nrows into the 32-bit variable, numDataPoints, the resulting value truncates. Your MEX-file could crash or produce incorrect results. Use type mwSize for numDataPoints, as shown in the following table.

Replace:With:
int numDataPoints;
mwSize numDataPoints;

Analyze Other Variables

You do not need to change every integer variable in your code. For example, field numbers in structures and status codes are of type int. However, you need to identify variables used for multiple purposes and, if necessary, replace them with multiple variables.

The following example creates a matrix, myNumeric, and a structure, myStruct, based on the number of sensors. The code uses one variable, numSensors, for both the size of the array and the number of fields in the structure.

mxArray *myNumeric, *myStruct;
int numSensors;
mwSize m, n;
char **fieldnames;
...
myNumeric = mxCreateDoubleMatrix(numSensors, n, mxREAL);
myStruct = mxCreateStructMatrix(m, n, numSensors, fieldnames);

The function signatures for mxCreateDoubleMatrix and mxCreateStructMatrix are:

mxArray *mxCreateDoubleMatrix(mwSize m, mwSize n,
    mxComplexity ComplexFlag)
mxArray *mxCreateStructMatrix(mwSize m, mwSize n,
    int nfields, const char **fieldnames);

For the mxCreateDoubleMatrix function, your code uses numSensors for the variable m. The type for m is mwSize. For the mxCreateStructMatrix function, your code uses numSensors for the variable nfields. The type for nfields is int. Replace numSensors with two new variables to handle both functions, as shown in the following table.

Replace:With:
int numSensors;
/* create 2 variables   */
/* of different types */
mwSize numSensorSize;
int numSensorFields;
myNumeric = 
    mxCreateDoubleMatrix(
    numSensors,
    n, mxREAL);
/* use mwSize variable */
/* numSensorSize       */
myNumeric = 
    mxCreateDoubleMatrix(
    numSensorSize,
    n, mxREAL);
myStruct = 
    mxCreateStructMatrix(
    m, n,
    numSensors,
    fieldnames);
/* use int variable */
/* numSensorFields  */
myStruct = 
    mxCreateStructMatrix(
    m, n,
    numSensorFields,
    fieldnames);

Replace Unsupported Functions

While updating older MEX-files, you could find calls to unsupported functions, such as mxCreateFull, mxGetName, or mxIsString. MATLAB removed support for these functions in Version 7.1 (R14SP3). You cannot use unsupported functions with 64-bit array dimensions. For the list of unsupported functions and the recommended replacements, see Obsolete Functions No Longer Documented.

Update your code to use an equivalent function, if available. For example, use mxCreateDoubleMatrix instead of mxCreateFull.

Test, Debug, and Resolve Differences After Each Refactoring Iteration

To build myMexFile.c with the 32-bit API, type:

mex -compatibleArrayDims myMexFile.c

Use the tests you created at the beginning of this process to compare the results of your updated MEX-file with your original binary file. Both MEX-files should return identical results. If not, debug and resolve any differences. Differences are easier to resolve now than when you build using the 64-bit API.

Resolve -largeArrayDims Build Failures and Warnings

After reviewing and updating your code, compile your MEX-file using the large array handling API. To build myMexFile.c with the 64-bit API, type:

mex -largeArrayDims myMexFile.c

Since the mwSize / mwIndex types are MATLAB types, your compiler sometimes refers to them as size_t, unsigned_int64, or by other similar names.

Most build problems are related to type mismatches between 32- and 64-bit types. Refer to http://www.mathworks.com/matlabcentral/answers/99144-how-do-i-update-mex-files-to-use-the-large-array-handling-api-largearraydims, Step 5 to identify common build problems for specific compilers, and possible solutions.

Execute 64-Bit MEX-File and Compare Results with 32-Bit Version

Compare the results of running your MEX-file compiled with the 64-bit API with the results from your original binary. If there are any differences or failures, use a debugger to investigate the cause. For information on the capabilities of your debugger, refer to your compiler documentation.

Refer to http://www.mathworks.com/matlabcentral/answers/99144-how-do-i-update-mex-files-to-use-the-large-array-handling-api-largearraydims, Step 6 to identify issues you might encounter when running your MEX-files, and possible solutions.

After you resolve any issues and upgrade your MEX-file, it replicates the functionality of your original code while using the large array handling API.

Experiment with Large Arrays

If you have access to a machine with large amounts of memory, you can experiment with large arrays. An array of double-precision floating- point numbers (the default in MATLAB) with 232 elements takes approximately 32 GB of memory.

For an example that demonstrates the use of large arrays, see the arraySize.c MEX-file in Handling Large mxArrays.

Update Fortran Source Code

All of the previous information applies to Fortran, as well as C/C++. Fortran uses similar API signatures, identical mwSize / mwIndex types, and similar compilers and debuggers. To make your Fortran source code 64-bit compatible, perform these additional tasks:

Use Fortran API Header File.  To make your Fortran MEX-file compatible with the 64-bit API, use the fintrf.h header file in your Fortran source files. Name your source files with an uppercase .F file extension. For more information about these requirements, see Components of Fortran MEX-File.

Declare Fortran Pointers.  Pointers are 32- or 64-bit addresses, based on machine type. This requirement is not directly tied to array dimensions, but you could encounter problems when moving 32-bit code to 64-bit machines as part of this conversion.

For more information, see Preprocessor Macros and mwPointer.

The C/C++ compiler automatically handles pointer size. In Fortran, MATLAB uses the mwPointer type to handle this difference. For example, mxCreateDoubleMatrix returns an mwPointer:

mwPointer mxCreateDoubleMatrix(m, n, ComplexFlag)
mwSize m, n
integer*4 ComplexFlag

Require Fortran Type Declarations.  Fortran uses implicit type definitions. This means undeclared variables starting with letters I through N are implicitly declared type INTEGER. Variable names starting with other letters are implicitly declared type REAL*4. Using the implicit INTEGER type could work for 32-bit indices, but is not safe for large array dimension MEX-files. To force you to declare all variables, add the IMPLICIT NONE statement to your Fortran subroutines. For example:

subroutine mexFunction(nlhs, plhs, nrhs, prhs)
implicit none

This statement helps identify 32-bit integers in your code that do not have explicit type declarations. Then, you can declare them as INTEGER*4 or mwSize / mwIndex, as appropriate. For more information on IMPLICIT NONE, refer to your Fortran compiler documentation.

Use Variables in Function Calls.  If you use a number as an argument to a function, your Fortran compiler could assign the argument an incorrect type. On a 64-bit platform, an incorrect type can produce Out of Memory errors, segmentation violations, or incorrect results. For example, definitions for the argument types for the mxCreateDoubleMatrix function are:

mwPointer mxCreateDoubleMatrix(m, n, ComplexFlag)
mwSize m, n
integer*4 ComplexFlag

Suppose that you have a C/C++ MEX-file with the following statement:

myArray = mxCreateDoubleMatrix(2, 3, mxREAL); 

Most C/C++ compilers interpret the number 2 as a 64-bit value. Some Fortran compilers cannot detect this requirement, and supply a 32-bit value. For example, an equivalent Fortran statement is:

myArray = mxCreateDoubleMatrix(2, 3, 0)

The compiler interprets the value of the ComplexFlag argument 0 correctly as type INTEGER*4. However, the compiler could interpret the argument 2 as a 32-bit value, even though the argument m is declared type mwSize.

A compiler-independent solution to this problem is to declare and use an mwSize / mwIndex variable instead of a literal value. For example, the following statements unambiguously call the mxCreateDoubleMatrix function in Fortran:

mwSize nrows, ncols
INTEGER*4 flag
nrows = 2
ncols = 3
flag = 0
myArray = mxCreateDoubleMatrix(nrows, ncols, flag)

Manage Reduced Fortran Compiler Warnings.  Some Fortran compilers cannot detect as many type mismatches as similar C/C++ compilers. This inability can complicate the step Resolve -largeArrayDims Build Failures and Warnings by leaving more issues to find with your debugger in the step Execute 64-Bit MEX-File and Compare Results with 32-Bit Version.

Was this topic helpful?