MATLAB Answers

Bruno Luong
4

mxCreateSharedDataCopy no longer supported in R2018a

Asked by Bruno Luong
on 19 Apr 2018
Latest activity Answered by Bruno Luong
on 22 Sep 2018

This is an error message I got when compiling my C-mex file

Error using mex Function mxCreateSharedDataCopy not supported. Either switch to using the more efficient MATLAB Data API available in C++ MEX Applications, or rerun MEX with the -R2017b command line option to use the R2017b API.

Can't see anything remotely equivalent in MATLAB Data API. This function is very important to me to make faster MEX code. What is the alternative if we want to use R2018a (interleaved complex) and create shared data copy ?

  2 Comments

I won't be able to update any of my most important mex code to run under R2018a without this function! This is bad ... very bad ...

I request TMW to reintroduce this function in the API.

Sign in to comment.

4 Answers

Answer by Ned Charles on 9 May 2018

We’d like to clarify that MEX-files built in R2018a using default options to the MEX command will not see a build error if they use the undocumented API mxCreateSharedDataCopy. Such MEX files work in both R2018a and older releases (and future releases). It is not possible to build MEX-files that both opt into the new interleaved complex API and use the undocumented mxCreateSharedDataCopy. MEX-Files that opt into Interleaved Complex only work in R2018a and future releases.
It is possible to build MEX files that both use interleaved complex data and have fully documented support for copy-on-write behavior. Arrays created with the C++ MATLAB Data API automatically use copy-on-write without any special handling and knowledge needed by the programmer. Such MEX files will work in R2018a and future releases. See https://www.mathworks.com/help/matlab/matlab_external/c-mex-functions.html for more information on C++ MEX. To Bruno’s question, not only can a C++ MEX function share data with input arguments, but copies of arrays created inside the MEX file will also use copy-on-write. See https://www.mathworks.com/help/matlab/matlab_external/copy-cpp-api-matlab-arrays.html.
We have not documented or supported copy-on-write in MEX-Files because we believe our internal C API would place too high a burden on the MEX-File author. Without the abstraction and implementation-hiding provided by C++, the C API would require cooperation among all the code using the arrays being shared so that copies are made before writing to the array data. With the C++ API, it is possible to fully support optimization without a MEX-File author needing to take any kind of special action to get correct behavior. MathWorks encourages the documented C++ API because it provides the performance of copy-on-write without the hassles of managing it and the risks of getting that management wrong which can lead to bugs that may be difficult to detect, diagnose, and fix.

  0 Comments

Sign in to comment.


Answer by Cris Luengo on 2 May 2018
Edited by Cris Luengo on 9 May 2018

Note that the new C++ interface (introduced in R2018a) has documented ways of creating shared data copies:
I think, going forward, we all need to move to the C++ interface starting with R2018a. Up to version R2017b we use the C interface with separate real and imaginary components, and starting with R2018a we use the C++ interface with interleaved complex values.
I have not gotten started with this new interface yet, but it seems to simplify use significantly. I'm actually looking forward to try it out! :)

  1 Comment

This article does seem to be related to the question, it merely state the rule to create a new copy when data is modified (because MATLAB can have another instant of variable that share the data with the input).
The question I ask is to be able to create a new instant of mxArray that share data with an existing variable, a mechanism TMW keeps for their internal uses. Pity. Many of my MEX codes are now become suddenly inefficient or obsolete.

Sign in to comment.


Answer by Jan
on 20 Apr 2018
Edited by Jan
on 20 Apr 2018

[More a comment than an answer:]
mxCreateSharedDataCopy was not documented. There might be an undocumented method in the 2018 Data API also.
It would be nice, if MathWorks provides an exhaustive documentation concerning the inplace access of array in the MEX level. I'm waiting for this since 1999.

  3 Comments

It certainly has, since MATLAB, regardless the version, shares internal data.
It is just pity that TMW keep for themselves. One of the main reasons of MATLAB slowness (compared to other language) is the intensive memory copying.
It is not clear to me wheras the newly implemented interleaved-complex and copying-in/out of mxGetPr/mxGetData breaks mxCreateSharedDataCopy() or not. Actually I don’t see the reason.
BTW, you don't answer my question Jan by saying mxCreateSharedDataCopy is not documented, this I knew.
I can accept answer that provides the workaround.
I do not want my answer to be accepted also, because the best answer will be written by MathWork's support team, preferably containing a link to the updated documentation.
This answer might be more accurate: "The drop of support shows, why this feature has not been documented yet. Replacing it by another undocumented method implies the danger of additional incompatibilities in the future. Therefore currently only deep data copies are reliable." But this would not match the needs of programmers.
Thanks, Bruno, for this important question. I hope that a discussions demonstrates the need of a documented method for shared data copies.
So, it is notable that the error message is not this
"... unresolved external symbol ..."
But it is this
"... mxCreateSharedDataCopy not supported ..."
So it is not the case that the function isn't necessarily there and the linker couldn't find it. What appears to be happening is that the mex command itself is stopping the build because it sees you are trying to link with this particular function.
The creation of shared data copies functionality is still present in MATLAB (easily verified by a few simple tests at the command line with "format debug"), and in fact the mxCreateSharedDataCopy function still looks like it is present in the libmx file, but the mex command is specifically preventing you from linking to it. So this appears to be a deliberate decision by TMW, not an inadvertent omission. This may be the result of the fact that the mxArray header was changed (e.g. the pi pointer was removed) and TMW not wanting to write a new function that is R2018a mxArray compatible, or perhaps TMW is trying to steer you towards their C++ DATA API. Either way this leaves the C programmers out in the cold with no alternative API functions to use.
As a side note, the undocumented mxCreateReference API function was actually deleted from the libmx file back in R2014a. This was very useful in mex functions when dealing with cell and struct variables.

Sign in to comment.


Answer by Bruno Luong
on 22 Sep 2018

An alternative solution would be to hack mxArray internal structure
/* File "mxInternals_R2018B.h" */
/* Hacked internal MXARRAY DEFINITION */
/* Matlab version: R2018b */
#define NumericalMask 0x00000080
#define ComplexMask 0x00000800
#define SparseMask 0x00000010
typedef struct mxArray_tag mxArray_tag;
struct mxArray_tag {
mxArray_tag *B_Xlink;
mxClassID classID;
unsigned int unknown1;
mxArray_tag *F_Xlink;
mwSize ndims;
unsigned int refcount; /* Number of mxArray* objects that point to this structure */
unsigned int Mask; /* See some of the mask values defined above */
union {
mwSize rows; /* Valid if the array has <= 2 dimensions */
mwSize *pSize; /* vector of size, in heap, Valid if the array has > 2 dimensions */
} size;
mwSize cols; /* If the array has > 2 dimensions, this is actually numel(X) / size(X,1) */
union {
struct {
void *Data;
unsigned int unknown0;
unsigned int unknown1;
unsigned int unknown2;
unsigned int unknown3;
unsigned int unknown4;
unsigned int unknown5;
} number_array;
struct {
void * Data;
mwIndex * pRowIndex;
mwIndex * pColFirstElement; /* pColFirstElement[j] is the index of the first entry in column j (see mxSetJc). The number of non-sparse elements is pColFirstElement[cols] */
mwSize nzmax; /* The number of non zero elements allocated for the sparse array */
} sparse_array;
struct {
mxArray_tag ** Data;
unsigned int unknown1;
unsigned int unknown2;
unsigned int unknown3;
unsigned int unknown4;
unsigned int unknown5;
unsigned int unknown6;
unsigned int unknown7;
} cell_array;
struct {
mxArray_tag ** Data;
void * pFieldNames; /* N.B. I have no idea how field names are stored */
unsigned int unknown1;
unsigned int unknown2;
unsigned int unknown3;
unsigned int unknown4;
unsigned int unknown5;
} struct_array;
} data;
unsigned int unknown3;
unsigned int unknown4;
unsigned int unknown5;
unsigned int unknown6;
unsigned int unknown7;
};
Then create shared mxArray like this (more complicated data can be handle along this line):
/*
* B = mxCreateSharedMatrix2018(A)
*
* INPUT: A is double matrix real or complex, (i.e., dim=2, not checked)
*
* Create an array B (mxArray differents than A) but shares data pointer with A
*
* >> mex -R2018a mxCreateSharedMatrix2018.c
*/
#include "mxInternals_R2018B.h"
#define A_IN prhs[0]
#define A_OUT plhs[0]
void mexFunction( int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[])
{
size_t m, n;
mxComplexity ComplexFlag;
mxArray_tag *Ain, *Aout;
if (nrhs < 1) mexErrMsgTxt("mxCreateSharedMatrix2018: one input required.");
ComplexFlag = mxIsComplex(A_IN) ? mxCOMPLEX : mxREAL;
m = mxGetM(A_IN);
n = mxGetN(A_IN);
A_OUT = mxCreateDoubleMatrix(0,0,ComplexFlag);
mxSetM(A_OUT,m);
mxSetN(A_OUT,n);
Ain = (mxArray_tag*)A_IN;
Aout = (mxArray_tag*)A_OUT;
/* Update cross-link, insert Aout as back of Ain */
Aout->B_Xlink = Ain->B_Xlink;
Aout->F_Xlink = Ain;
if (Aout->B_Xlink) Aout->B_Xlink->F_Xlink = Aout;
else Aout->B_Xlink = Ain;
if (Ain->F_Xlink==NULL) Ain->F_Xlink = Aout;
Ain->B_Xlink = Aout;
/* Shared data */
Aout->data.number_array.Data = Ain->data.number_array.Data;
/* It seems I do not need to update refcount of Ain and Aout */
}

  0 Comments

Sign in to comment.