Path: news.mathworks.com!newsfeed-00.mathworks.com!nlpi057.nbdc.sbc.com!prodigy.net!border1.nntp.dca.giganews.com!nntp.giganews.com!postnews.google.com!w7g2000hsa.googlegroups.com!not-for-mail
From: Rune Allnor <allnor@tele.ntnu.no>
Newsgroups: comp.soft-sys.matlab
Subject: Re: how to save time loading data from file
Date: Mon, 4 Aug 2008 07:49:19 -0700 (PDT)
Organization: http://groups.google.com
Lines: 32
Message-ID: <4b983eef-8f6c-4eb8-87fd-961d5f24413d@w7g2000hsa.googlegroups.com>
References: <g75abr$40d$1@fred.mathworks.com>
NNTP-Posting-Host: 212.17.141.54
Mime-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
X-Trace: posting.google.com 1217861359 26356 127.0.0.1 (4 Aug 2008 14:49:19 GMT)
X-Complaints-To: groups-abuse@google.com
NNTP-Posting-Date: Mon, 4 Aug 2008 14:49:19 +0000 (UTC)
Complaints-To: groups-abuse@google.com
Injection-Info: w7g2000hsa.googlegroups.com; posting-host=212.17.141.54; 
User-Agent: G2/1.0
X-HTTP-UserAgent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET 
Bytes: 2198
Xref: news.mathworks.com comp.soft-sys.matlab:483524



On 4 Aug, 00:10, "ggk " <ggkm...@comcast.net> wrote:
> Hi,
>
> I've run a profile on my code and found that 90% of the
> time running my .m file is consumed by one line of code:
>
> x = load('data.txt');
>
> I'm using 1GB RAM (Windows XP 3GHz Pentium4). The file
> contains ascii data -- one data point (double precision)
> per row, and many rows.

How many rows?

> Can anyone share their insight how to speed this up?
>
> - Will more RAM help?

No. The time-consuming step is to convert data from
ASCII to binary. How you read the file from disk is
insignificant in comparision.

> - Any other means to load data into matlab that is faster?

The only thing that will speed things *significantly*
is to store the data on binary format. Not too long ago
I sped up the loading of 90 MB of ASCII files from ~45 s
to ~0.2 s by changing the storage format to binary. Not
only were the loading some 250x faster, I also saved some
20% disk space by storing the data on binary format.

Rune