When should you start to consider using MEX files?

19 views (last 30 days)
Hi
Whilst following some discussion forums about project euler problems and the best languages to use, someone asked was Matlab any good for solving all problems. The reply said that some of the problems were designed to take considerable more time running a Matlab function than a compiled language.
So my question. How do I decide that a problem is best suited to Matlab or C/C++. Other than writing a program/function and my dinner getting cold while I wait for an answer. Should I teach myself building MEX files or would I be better just writing, compiling and running from command line.

Answers (2)

Walter Roberson
Walter Roberson on 14 Nov 2011
One of the first things you have to ponder is execution time compared to development time (including debugging.)
In my work, we develop new algorithms, and compare them to see which algorithm produces the best answer. For our purposes, ease of development and debugging is important: R&D is a lot of hard thinking and thinking is expensive. Once we have settled on some particular algorithms then might be time to make them substantially faster using a different language -- or it might be time to license the intellectual property and let some other organization worry about that.
And since R&D often turns out to be about economic politics rather than about best product, sometimes it turns out that taking the algorithms to market is financially impractical -- it costs pretty much $3 million to get through all of the regulatory assessments and field trials for anything the US considers a "medical device". In such a case, we may just continue to use the prototyped version in-house without the rewrite in to a faster language, knowing that the expense of rewriting would be far more than the in-house expense of waiting a little longer for the MATLAB version to finish.
Now, if "everyone" in-house was using what we develop, then the total waiting time might make it worth-while to spend the time doing a rewrite.
  3 Comments
Walter Roberson
Walter Roberson on 15 Nov 2011
A) You have educational costs.
C is not bad to learn "competently", but even real C experts can get caught on the minute details.
C++ is harder to learn than C, but what really bogs you down with C++ is learning the C++ library, which you are expected to know.
MATLAB itself, at least without the object oriented programming, is in many respects easier to learn than C: just don't hope to understand the subtle parts of the I/O subsystem unless you _do_ have C experience.
Where MATLAB really gets people is that the MATLAB libraries are huge and cover such a wide variety of very different fields, that it is fairly difficult for anyone to learn all the facilities of the standard libraries, let along the optional toolboxes. You may spend years learning all the best ways to use one-part and two-part expoxy / resin mixes under varying circumstances, only to have someone with different MATLAB experience come by and tackle the problem with the MATLAB equivalent of the staple gun.
Personal bias: back when I studied the C++ standard library, it was fairly large (and yet much smaller than it is now!), and I had a heck of a time trying to figure out the differences between multiple routines with seemingly the same functionality. It was a like pouring a mix of mud and treacle through my brain.
MATLAB probably has a much larger library than C++ does, but one can be a very productive developer by sticking to the parts relevant to your field and your tasks. If there is a graph theory solution routine to your load-flow problem, you don't have to care much if the problem in front of you is "get the load-flow working": you do what you can, and as your time and interests develop, may choose to branch off to other parts of MATLAB.
There is no shame around here for not knowing the details of the MATLAB communications theory libraries because you were concentrating on getting your IIR filters designed properly -- just as there is no shame for not being simultaneously a physicist, brain surgeon, test pilot, and rock star. (Bonus points for those who recognize the reference ;-) ) That is not at all the same welcome as I found in the C++ community.
B) You will find that you understand one of the languages better than the other, or have more "affinity" with it. If using that language (which-ever one it turns out to be) results in you coming up with a decent solution in (say) 2 hours, but would have taken you (say) 3 hours of harder cursing with a supposedly "better" language, then in the absence of externally imposed standards, you might as well use what you know best.Unless, that is, the running time is going to be quite long and could be substantially shorter with another language: then it might be worth the extra thinking and debugging in the other language.
C and C++ will usually be faster than MATLAB, but not invariably. I do not, for example, encounter many people who bother to routinely link LAPAK in to their programs for fastest computation, not when they are just trying to solve a puzzle.
Scragmore
Scragmore on 15 Nov 2011
Wow, thanks. I've never come across that quote or film. Looks like a must see for any geek, grabbing copy now, will watch later. Can't wait ;D
The libraries thing does seem to be like learning the concise oxford dictionary in order to read HGTTG. I did like C (love pointers and the possibilities) but didn't like my conversion to C++. I started to learn because I wanted a better understanding of the tools I have been using all my life. Up until now all I knew was 10 print "Hello world" 20 goto 10. :) When I get work I am employed to use excel (yes I feel dirty and wash regularly) but finding that I think Matlab is more were I want to be. So please expect more naive questions as play with this great tool.
Will let you know what I think of the film.

Sign in to comment.


Jan
Jan on 15 Nov 2011
The allocation of large temporary arrays can consume more time than the actual calculations. Example:
x = rand(1, 1e7);
v = max(abs(sqrt(sin(x))) > 5);
Required temporary arrays:
tmp1 = sin(x);
tmp2 = sqrt(tmp1);
tmp3 = abs(tmp2);
tmp4 = (tmp3 > 5);
Each memory block does neither match into the internal data cache of the processor nor in the 2nd level cache. Therefore you need a time-consuming interchange with the RAM.
Using a FOR loop is made much faster since Matlab 6.5 (R13). The JIT acceleration improves the speed massively. For this example, I expect a C-mex to be even faster. But the main benefit of the FOR loop in Matlab or C is avoiding the need of large temporary arrays.
I've published some C-mex files in the FEX, which benefit from this method: DNorm2, anyExceed, anyEq, Shuffle, DGradient. Another surprising field for C-mex functions are some built-in functions, which seem to have a suboptimal implementation, as e.g. filter or horzcat (see FilterM and Cell2Vec).
But before you start to implement some functions in C, use the profiler or other speed measurements to find the bottlenecks of your program. As Walter has said already: Spending hours to optimze a function, which runs for seconds only, is a waste of time - except if real-time programming is necessary or the total runtime of the program can be squeezed under 5 seconds. This is the magic limit, which provokes physical stress for human users.

Categories

Find more on Programming in Help Center and File Exchange

Products

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!