Path: news.mathworks.com!not-for-mail
From: <HIDDEN>
Newsgroups: comp.soft-sys.matlab
Subject: Re: Need advice on File I/O
Date: Wed, 5 Sep 2007 15:39:15 +0000 (UTC)
Organization: The MathWorks, Inc.
Lines: 61
Message-ID: <fbmij3$s6c$1@fred.mathworks.com>
References: <fbaa6i$3pu$1@fred.mathworks.com> <fbcpdd$b7$1@fred.mathworks.com> <fbd23f$jt2$1@fred.mathworks.com> <muy7in6qxky.fsf@G99-Boettcher.llan.ll.mit.edu>
Reply-To: <HIDDEN>
NNTP-Posting-Host: webapp-06-blr.mathworks.com
Content-Type: text/plain; charset="ISO-8859-1"
Content-Transfer-Encoding: 8bit
X-Trace: fred.mathworks.com 1189006755 28876 172.30.248.36 (5 Sep 2007 15:39:15 GMT)
X-Complaints-To: news@mathworks.com
NNTP-Posting-Date: Wed, 5 Sep 2007 15:39:15 +0000 (UTC)
X-Newsreader: MATLAB Central Newsreader 1030060
Xref: news.mathworks.com comp.soft-sys.matlab:427089



Peter Boettcher <boettcher@ll.mit.edu> wrote in message
<muy7in6qxky.fsf@G99-Boettcher.llan.ll.mit.edu>...
> "G.A.M. " <x0zero@gmail.com> writes:
> 
> > "Bill " <william.nospam.a.cobb@gm.com> wrote in message
> > <fbcpdd$b7$1@fred.mathworks.com>...
> >
> > It seems like my options are coming down to either Excel or
> > a database. But I was hoping for a solution that involved
> > CSV/ASCII files. My past experience tells me that using a
> > database will probably be slower than writing to CSV files.
> 
> Unless you are talking about bulk saving or loading, I
very much doubt
> that!

I was on a data-intensive application development project
where we collected extensive performance data on various
persistence options and we were all surprised how fast the
ASCII file I/O option was in a variety of situations. We
ended up going with the ASCII file I/O option because it was
several orders of magnitude faster for every I/O operation
we used. That said, another application might have produced
very different results.


> 
> > (I don't know how using one of the options to write to
> > Excel-formatted files will compare in speed.) I also like
> > the fact that ASCII text files, besides being generally fast
> > than a db, are really easy to deal with.
> 
> Well, you are discovering precisely the things that make
an ASCII file
> difficult to deal with.  They are very bad at random
access, and
> nearly impossible to do single-entry modifications. 
Higher-level
> wrappers like csvwrite, etc, will not change this.  It may
hide some
> of the nastiness, but it will still probably rewrite the
entire file.
> When I say bad, I mean slow and complex to implement.

These points are all good reminders for me. Thanks for
bringing them up.

> 
> Databases are very good at randomly accessing elements, and at
> modifying, inserting, etc.


I will seriously consider a database now.


> 
> A third option not mentioned here is a binary file.  

I think I'll stay away from this option. If I'm not going to
enjoy the advantages of a database, I at least want to have
the advantages of ASCII.