C++ API: How to convert ArrayElementRef to Array?

I have a C++ function that accepts an Array, directly from a mex-function input and results in a single C++ class. This works OK:

dataClass processStruct(const matlab::data::Array& matlabArray, const size_t index = 0)

{

...

const matlab::data::StructArray matlabStructArray = matlabArray;

// Process struct-fields using 'matlabStructArray', i.e. matlabStructArray[index]["field"]

...

}

And I have a function that does the same, but then returning a std::vector. This function must call the above function to do the conversion:

std::vector<dataClass> processStructVector(const matlab::data::Array& matlabArray)

{

std::vector<dataClass> result;

for (size_t i = 0; .....)

result[i] = processStruct(matlabArray[i]);

return result;

}

This gives me an error:

'Field does not exist in this struct.'

I can make it working by changing the call in the for-loop to

result[i] = processStruct(matlabArray, i);

but then the whole array is converted from Array to StructArray in processStruct() for each 'i' which is a huge amount of overhead. We don't want that...

How can I solve this?

Answer 1

[埃博拉酱]

In MATLAB, an element of an array is also an array. However, in C++, an element is not an array.

matlabArray[i] is taking an element of the array. However, when calling processStruct with this argument, you're casting this element to an array by implicitly casting it into a matlab:data::Array. And then in the function processStruct, you are indexing into the element that 'pretends' to be an array. Due to the lack of strict type checking, this may pass compilation, but there are actually logical errors that will result in unexpected behavior at runtime.

According to my guess, what you really want to do should be something like this:

using dataClass = int;
dataClass processStruct(const Struct& matlabStruct)
{
	return TypedArray<int>(matlabStruct["field"])[0];
}
std::vector<dataClass> processStructVector(const StructArray& matlabArray)
{
	std::vector<dataClass> result;
	for (size_t i = 0; i < matlabArray.getNumberOfElements(); ++i)
		result.push_back(processStruct(matlabArray[i]));
	return result;
}
void MexFunction::operator()(ArgumentList& outputs, ArgumentList& inputs)
{
	const StructArray SA(std::move(inputs[0]));
	std::vector<dataClass> result = processStructVector(SA);
}

Usually, the top-level caller is responsible for parsing the array to a specific type. Use std::move to avoid array copying. Then, the top-level caller retains ownership of the array and passes a const reference to the subroutine. Note that in C++, it is strictly necessary to distinguish between "array" and "element", and indexing operations on elements are not allowed (except for Cells).

[Jeroen Boschma]

Thank you for your answer. In the end I did things a bit different for several reasons that forced me so:

• given the functions available for doing the actual conversion of struct-fields (they accept a struct array plus index, not a struct)
• uniformity of this kind of converters that already were present in the project (I joined the project late)
• etc.

Nevertheless, you pushed me in the right direction. So now I have something like this (C++ details omitted, code only to show the dependencies...), bottom line is that I have 2 processStruct() functions overloaded:

dataClass processStruct(const StructArray& matlabStructArray, const size_t index)

{

// Conversion of all struct fields below.

...

}

dataClass processStruct(const Array& matlabArray, const size_t index)

{

const StructArray SA = matlabArray;

return processStruct(SA, index);

}

std::vector<dataClass> processStructVector(const Array& matlabArray)

{

std::vector<dataClass> result;

const StructArray SA = matlabArray; // std::move on const does not work?

for (size_t i = 0; i < SA.getNumberOfElements(); ++i)

result.push_back(processStruct(SA, i));

return result;

}

void MexFunction::operator()(ArgumentList& outputs, ArgumentList& inputs)

{

std::vector<dataClass> result = processStructVector(inputs[0]);

...

}

Because the mex-inputs may be used again, they are passed as const and std::move has no effect, I guess... On the other hand, if you have:

const Array A = ...;

const StructArray SA = std::move(A);

then it would be possible for a compiler to detect that SA and A are both const so it is safe to share the resources without a copy or even move?

[埃博拉酱]

@Jeroen Boschma Yes, constant resources will not be copied. However, strictly speaking, this is not a compiler optimization, but MEX API defines different function template specializations for copying constants and variables. For constants, a high-performance shallow copy is used, while a deep copy is used for variables. Even with a deep copy, it will only occur when attempting to access array elements. If array elements are not accessed, even variables will not undergo deep copying.

[埃博拉酱]

But if possible, I recommend that only one type conversion be done by the top-level caller, rather than relying entirely on copy optimizations of the MEX API. The MEX array is essentially a std::shared_ptr, and the so-called shallow copy actually not only copies this 24-byte shared_ptr, but also increases its use_count. In addition, the MEX array also needs to check the correctness of type conversion. If it is incorrect, an error should be reported. These are all performance overheads that will occur every time a shallow copy is made. If this conversion occurs frequently, it still incurs a significant performance overhead. So the best practice is still to have the top-level caller parse the array directly to the correct type, and then pass its constant reference to the subroutine: a constant reference is only 8 bytes (for a 64-bit CPU), and there is no reference counting overhead, which is bound to be faster than shared_ptr.

[Jeroen Boschma]

@埃博拉酱 Thanks for your explanations. I agree with you to do the required array-conversions as soon and as little as possible. Because the project at hand has a lot of legacy code I have to work with, things may be not as optimal as it could be. For now rewriting is not an option for several reasons, and I think processing all 5000 elements of the struct array, i.e. converting them from Matlab data to C++ classes, will be much more time-consuming than the Matlab array type conversion. I'll keep your advice in mind though for new projects.