Path: news.mathworks.com!not-for-mail
From: "Bruno Luong" <b.luong@fogale.findmycountry>
Newsgroups: comp.soft-sys.matlab
Subject: Re: matlab typecast function very slow
Date: Tue, 3 Nov 2009 12:49:02 +0000 (UTC)
Organization: FOGALE nanotech
Lines: 56
Message-ID: <hcp8ru$d11$1@fred.mathworks.com>
References: <05e24510-1633-4d87-9cbb-39230a6efeec@n35g2000yqm.googlegroups.com>
Reply-To: "Bruno Luong" <b.luong@fogale.findmycountry>
NNTP-Posting-Host: webapp-02-blr.mathworks.com
Content-Type: text/plain; charset="ISO-8859-1"
Content-Transfer-Encoding: 8bit
X-Trace: fred.mathworks.com 1257252542 13345 172.30.248.37 (3 Nov 2009 12:49:02 GMT)
X-Complaints-To: news@mathworks.com
NNTP-Posting-Date: Tue, 3 Nov 2009 12:49:02 +0000 (UTC)
X-Newsreader: MATLAB Central Newsreader 390839
Xref: news.mathworks.com comp.soft-sys.matlab:582029


roger <northsolomonsea@gmail.com> wrote in message <05e24510-1633-4d87-9cbb-39230a6efeec@n35g2000yqm.googlegroups.com>...
> Hi,
> 
> I have a function that reads large binary files by reading a couple of
> numbers per time in a complicated loop using fread.
> 
> Since the function isn't too fast I thought I'd read the entire file
> at once as uint8 and then typecast byte by byte to the relevant
> datatypes.

Why doing byte-by-byte? Use typecast for the whole array. Many function Matlab function is not designed to called many times, but rather on an array. Example:

b=zeros(8*1024^2,1,'uint8');

tic;
d=typecast(b,'double');
toc % Elapsed time is 0.012408 seconds.

tic;
% a already allocated
k=1;
for i=1:8:length(b)
    d(k)=typecast(b(i+(0:7)),'double');
end
toc % 40 sec

> 
> To my surprise the typecast method is actually a _lot_ slower than
> consecutively using fread, on inspection the profiler shows that the
> typecast function is the main culprit.
> 
> The function that serves the typecast.m function is a mex file called
> typecastc.mexw32 (on my 32 bit system that is). It seems to be a
> function which is compiled from an accompying c file that is located
> in the same private directory: typecastc.c
> 
> When I look at the c source file I see that the function itself
> doesn't do any typecasting, it just creates a relevant datatype with
> the mex mxCreateNumericMatrix and then leaves the actual typecasting
> to the mx libraries.
> 
> So all in all the typecasting slowness seems a result of matlab's
> inability to deal with c-style pointers from m-files and the penalty
> for this is the mx library overhead.
> 
> An odd situation.
> 
> Anyone have a better method (than the mathworks have ;) for reading/
> typecasting?
> 

Alternatively use MEMMAPFILE to read binary data.

or MEX it, but first try to use TYPECAST on the whole array.

Bruno