Path: news.mathworks.com!not-for-mail
From: "Steve Amphlett" <Firstname.Lastname@Where-I-Work.com>
Newsgroups: comp.soft-sys.matlab
Subject: Re: much faster MATLAB
Date: Tue, 7 Oct 2008 19:11:01 +0000 (UTC)
Organization: Ricardo UK Ltd
Lines: 23
Message-ID: <gcgc85$7dc$1@fred.mathworks.com>
References: <gcebi7$otk$1@fred.mathworks.com>
Reply-To: "Steve Amphlett" <Firstname.Lastname@Where-I-Work.com>
NNTP-Posting-Host: webapp-03-blr.mathworks.com
Content-Type: text/plain; charset="ISO-8859-1"
Content-Transfer-Encoding: 8bit
X-Trace: fred.mathworks.com 1223406661 7596 172.30.248.38 (7 Oct 2008 19:11:01 GMT)
X-Complaints-To: news@mathworks.com
NNTP-Posting-Date: Tue, 7 Oct 2008 19:11:01 +0000 (UTC)
X-Newsreader: MATLAB Central Newsreader 43398
Xref: news.mathworks.com comp.soft-sys.matlab:494126


"Kevin Johnson" <defer.jof@gmail.com> wrote in message <gcebi7$otk$1@fred.mathworks.com>...
> All,
> 
> I have been using MATLAB for many years now but finally its speed has become a major limiting factor.  As time has gone on, large segments of my code are no longer in need of further development (though other segments are). These same segments are the ones that consume most of the computation time. 
> 
> Mex files are the solution I suppose; however the last time I programmed in Fortran was 30 years ago, and in C, never. In reading about mex files, it sounds daunting for a basically amateur (in terms of skills) programmer like me. 
> 
> In a broad sense, what would be the best approach to substantially (at least 10x) speed up these unchanging swaths of code, for someone who is not a programmer?  I'd be able to invest in a solution within reason.


Some additional comments that may be relevant.

Files: Matlab didn't use to be that clever at file I/O.  Or at data types other than double precision. Given some random file to read (e.g. from a simulation program or data analyser), MEX was the only sensible option.  That may have changed now - I've not recently tried to read in huge amounts of data from a binary file.  Many vendors will provide you with A C API to read their files, pretty much forcing you to use MEX.

Memory: Most data analysis I used to do involved very large datasets.  The analysis was usually limited my memory.  Fine-grained control over memory use was something that was absolutely necessary, but not possible with Matlab - too many intermediate copies of variables.  To a certain extent, this is still true today.

System interaction:  If you want access to base OS functions, you need to code in their language.  In the UNIX environment, this is C.  What you are really doing is extending Matlab rather than simply making it faster.

As far as speed goes, I've never really been involved with matrix maths.  Most of my analysis is/was with very long 1D arrays.  It was always simple to get good speedup over the built-in functions.  Massive if you "crossed to the dark side" and started to modify things you weren't supposed to.

An anecdote:  One of our departments based its whole analysis core on a load of MEX functions, using simple m-file scripts to shovel data between them.  The original author left many years ago, leaving them unsupported and to a large extend, not even understood (neither the theory nor implementation).  After a few years with their heads in the sand they seem to have decided to go back to the safer position of using MEX only where required rather than for fun.

My view: MEX files should not be used unless there is a competent (hopefully better than that) C programmer on board, preferably more than one (see above).  Outsourcing them sounds like a suicidal tactic.