MATLAB Answers

How to access raw data via the mex c++ api?

76 views (last 30 days)
I want to use the mex c++ api to interface existing external code. The API of the external application is fairly low level, but I cannot work out how I can access the double pointers I'd like to pass. As this is not specific to the code I have, assume I have a function
void foo(int n, const double *a, const double *b, double *c)
to which I pass the arrays a and b and their lengths n and it calculates something and puts it onto the array c.
In order to use the mex function with the c++ api I don't know how I can avoid copying the data into some dummy pod arrays along the lines of:
#include "mex.hpp"
#include "mexAdapter.hpp"
using namespace matlab::data;
using matlab::mex::ArgumentList;
class MexFunction : public matlab::mex::Function
{
private:
ArrayFactory factory;
std::shared_ptr<matlab::engine::MATLABEngine> matlabPtr;
void validateArguments(ArgumentList outputs, ArgumentList inputs)
{
if (inputs.size() != 2)
matlabPtr->feval(u"error", 0, std::vector<Array>({ factory.createScalar("Require 2 inputs")}));
if (inputs[0].getType() != ArrayType::DOUBLE || inputs[0].getType() == ArrayType::COMPLEX_DOUBLE )
matlabPtr->feval(u"error", 0, std::vector<Array>({ factory.createScalar("First input has to be a double array") } ));
if (inputs[1].getType() != ArrayType::DOUBLE || inputs[1].getType() == ArrayType::COMPLEX_DOUBLE )
matlabPtr->feval(u"error", 0, std::vector<Array>( {factory.createScalar("second input has to be a double array")}));
if (outputs.size() > 1)
matlabPtr->feval(u"error", 0, std::vector<Array>( {factory.createScalar("One output only")}));
}
void foo(int n, const double *a, const double *b, double *c){
int i;
for (i=0;i<n;i++)
c[i] = a[i] + b[i];
}
public:
MexFunction() : matlabPtr(getEngine()) {}
~MexFunction() {}
void operator() (ArgumentList outputs, ArgumentList inputs)
{
validateArguments(outputs,inputs);
double *retVal, *a, *b;
int i, n = inputs[0].getNumberOfElements();
retVal = new double[n];
a = new double[n];
b = new double[n];
for (i =0; i<n;i++){
a[i] = inputs[0][i];
b[i] = inputs[1][i];
}
foo(n, a, b, retVal);
ArrayDimensions dims = inputs[0].getDimensions();
outputs[0] = factory.createArray(dims, retVal, retVal+n);
delete a;
delete b;
delete retVal;
}
};
This copying and creating of placeholders seems unelegant and wasteful. What am I missing? Please help!
Thank you!
Manuel

  0 Comments

Sign in to comment.

Accepted Answer

Mihir Thakkar
Mihir Thakkar on 3 Jan 2019
There is a "release" member function that returns a unique pointer of the raw data: TypedArray Documentation.
Instead of copying data from the typed array, the following should just work:
double* a=inputs[0].release().get();

  2 Comments

Dominic Liao-McPherson
Dominic Liao-McPherson on 8 Nov 2019
The release documentation says that
"Release the underlying buffer from the Array. If the Array is shared, a copy of the buffer is made; otherwise, no copy is made. After the buffer is released, the array contains no elements."
What does this imply when using .release().get()? Won't the unique_ptr returned by release() immediately go out of scope (since its not being assigned to anything) and thus the memory pointed to by
```double* a = inputs[0].release().get();```
will become invalid? Or is my understanding of unique_ptr flawed?
Of course just assigning
std::unique_ptr<double> temp = inputs[0].release();
double* a = temp.get();
Would resolve the issue.
Torsten Knüppel
Torsten Knüppel on 4 Apr 2020
Mihir could you please elaborate your answer?
It would be really great if it was so simple to access the raw data, but I do share Dominic's concerns. Furthermore, I don't manage to get your code working, because matlab::data::Array has no release-function. Following the answers below, it should be something along the line
double* b = matlab::data::TypedArray<double>(inputs[0]).release().get()
Furthermore, I don't fully understand the documentation - under which cirumstances does this create a copy (because I think this is what everybody wants to avoid). According to the documentation this is done, when the array is -shared-. Is this to be understood in the sense of a shared-pointer as the existence of multiple references to the array? I have done a test: I passed an array to the Mex-function and then released the memory with code above. After returning to Matlab the array that I passed was unchanged (which I was glad to see). However, I would have expected that somehow the array is destroyed by calling release, because this somehow detaches the data from the array (at least that's how I understand it).
Does this mean, that Matlab is passing the data by copy or does it keep a reference to the array (so that the data is copied when I call release). Wouldn't that mean, that I can't avoid a copy when using the mechanism you suggest?
Thanks in advance!

Sign in to comment.

More Answers (1)

Savyasachi Singh
Savyasachi Singh on 14 Feb 2020
Edited: Savyasachi Singh on 14 Feb 2020
I am extracting the pointer to the underlying data for TypedArray<T> using the following code. The trick is to extract the pointer from TypedIterator<T>.
//! Extracts the pointer to underlying data from the non-const iterator (`TypedIterator<T>`).
/*! This function does not throw any exceptions. */
template <typename T>
inline T* toPointer(const matlab::data::TypedIterator<T>& it) MW_NOEXCEPT {
static_assert(std::is_arithmetic<T>::value && !std::is_const<T>::value,
"Template argument T must be a std::is_arithmetic and non-const type.");
return it.operator->();
}
/*! Extracts pointer to the first element in the array.
* Example usage:
* \code
* ArrayFactory factory;
* TypedArray<double> A = factory.createArray<double>({ 2,2 }, { 1.0, 3.0, 2.0, 4.0 });
* auto ptr = getPointer(A);
* \endcode
* \note Do not call `getPointer` with temporary object. e.g., the following code is ill-formed.
* auto ptr=getPointer(factory.createArray<double>({ 2,2 },{ 1.0, 3.0, 2.0, 4.0 }));
*/
template <typename T>
inline T* getPointer(matlab::data::TypedArray<T>& arr) MW_NOEXCEPT {
static_assert(std::is_arithmetic<T>::value, "Template argument T must be a std::is_arithmetic type.");
return toPointer(arr.begin());
}
template <typename T>
inline const T* getPointer(const matlab::data::TypedArray<T>& arr) MW_NOEXCEPT {
return getPointer(const_cast<matlab::data::TypedArray<T>&>(arr));
}
Add the following #includes
  1. type_traits
  2. MatlabDataArray.hpp
Use the function getPointer as follows
ArrayFactory factory;
TypedArray<double> A = factory.createArray<double>({ 2,2 }, { 1.0, 3.0, 2.0, 4.0 });
auto ptr = getPointer(A); // double*
const TypedArray<double> B = factory.createArray<double>({ 2,2 }, { 1.0, 3.0, 2.0, 4.0 });
auto ptr = getPointer(B); // const double*
// Assuming that inputs[0] is a double array
const TypedArray<double> inArr(inputs[0]);
auto ptr = getPointer(inArr); // const double*
This approach has been working fine for me. Do not call getPointer function on a temporary TypedArray object.

  2 Comments

Manuel Schaich
Manuel Schaich on 15 Feb 2020
This seems like a pretty neat solution! I've adopted the buffer_ptr_t to deal with the Matlab facing IO. It is effectively just a typedef for std::unique_ptr but it allows some separation between library api and matlab api.
I would do something along the lines of
TypedArray<Real> xVec(std::move(inputs[0]));
buffer_ptr_t<Real> x = bVec.release();
buffer_ptr_t<Real> retVal = factory.createBuffer<Real>(n);
call_my_library(x.get(), retVal.get(), n);
if (outputs.size() > 0)
outputs[0] = factory.createArrayFromBuffer<Real>({n, 1}, std::move(retVal));
I guess the separation of apis is similar, but I get away not having to deal with any raw pointers in the intermediary layer of my mex wrapper.
Savyasachi Singh
Savyasachi Singh on 16 Feb 2020
I like your solution using the move semantics. You are removing the guts from TypedArray which helps in the separation of apis.
My motivation of writing the getPointer function was two fold. Firstly I needed the pointer to the MATLAB arrays so that I can pass them to 3rd party routines.
Secondly and most importantly, the indexing of TypedArray (via operator[]) is painfully slow as compared to indexing on the raw pointer. As an example perform a matrix multiplication in mex using TypedArray's operator[] and then repeat the same operation by using the pointer returned by getPointer method. You will find that the version that uses TypedArray's operator[] is significantly slower. I am disappointed by the latest MathWorks C++ api.
Hence I wrote the getPointer function which is a lightweight implementation since it is extracting the pointer from TypedIterator.

Sign in to comment.

Sign in to answer this question.

Products


Release

R2018b