MATLAB Engine for Python is very slow

I have a main python script and I want to call a Matlab function (spm_filter.m) on an array.

I use something like the following in a jupyter notebook:

import matlab.engine
eng = matlab.engine.start_matlab()
y_filtered = np.asarray(eng.spm_filter(filter_kernel, matlab.double(data.tolist()) ,nargout=1))

This works fine but it is extremely slow compared to the case where I use only Matlab to do the same thing e.g. by doing:

y_filtered = spm_filter(filter_kernel, data)

Why is the matlab engine so slow? How can I make it faster?

Answer 1

[Alan Frankel]

The performance in R2022a is orders of magnitude better than in previous releases, as described in the release notes. See the item "MATLAB Engine API for Python: Improved performance with large multidimensional arrays in Python" under the "Performance" section. As mentioned there, converting a Python array of size 10^8 to a MATLAB double array is about 12x faster than in the previous release, and executing the sum() function (including data conversion between Python and MATLAB) is about 110x faster than in the previous release.

[Sam Bilson]

I think the OP means it takes a while to start the engine. Any way to speed this up?

[Alan Frankel]

You can connect Python to a running instance of MATLAB: Connect Python to Running MATLAB Session

[Mitchell Tillman]

@Alan Frankel I was having some performance issues with sending a 30MB variable from Python to MATLAB and so just upgraded from R2021b to R2023b and performance was way worse - there's a memory leak now (RAM useage shot up over 20GB in just a minute or two, which did not happen before) and a function that used to take ~0.2 seconds now took 0.5 - 1.5 seconds to run (this is untenable because I need to run it many times).

Is there any reason to suspect that R2022a performs better than R2023b? Going to downgrade back to R2021b so I can benchmark the two versions.

Edit: the attached images shows memory useage over time running many iterations of the same function I mentioned above, in R2021b and R2023b. I also posted a question expanding on my issue.

[Alan Frankel]

As I posted on your question, I'm unable to reproduce your results.