Path: news.mathworks.com!newsfeed-00.mathworks.com!newsfeed2.dallas1.level3.net!news.level3.com!postnews.google.com!z3g2000prd.googlegroups.com!not-for-mail
From: Praetorian <ashish.sadanandan@gmail.com>
Newsgroups: comp.soft-sys.matlab
Subject: Re: read large text files
Date: Tue, 27 Oct 2009 21:04:56 -0700 (PDT)
Organization: http://groups.google.com
Lines: 54
Message-ID: <fa09ddda-8988-47b7-bb6a-5c33b24d9bcd@z3g2000prd.googlegroups.com>
References: <hc85sr$f4q$1@fred.mathworks.com> <hc88hj$k9a$1@news.eternal-september.org> 
	<b976d3af-5074-4042-8ca3-d96de0e02de8@g1g2000vbr.googlegroups.com>
NNTP-Posting-Host: 71.56.239.95
Mime-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
X-Trace: posting.google.com 1256702696 6167 127.0.0.1 (28 Oct 2009 04:04:56 GMT)
X-Complaints-To: groups-abuse@google.com
NNTP-Posting-Date: Wed, 28 Oct 2009 04:04:56 +0000 (UTC)
Complaints-To: groups-abuse@google.com
Injection-Info: z3g2000prd.googlegroups.com; posting-host=71.56.239.95; 
	posting-account=uxNuEQoAAABwxsrGxAKeLxAj2hdO1tck
User-Agent: G2/1.0
X-HTTP-UserAgent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.3) 
	Gecko/20090824 Firefox/3.5.3,gzip(gfe),gzip(gfe)
Xref: news.mathworks.com comp.soft-sys.matlab:580521


On Oct 27, 9:39 pm, anandhi <anandhi.san...@gmail.com> wrote:
> On Oct 27, 9:54 pm, dpb <n...@non.net> wrote:
>
>
>
> > Anandhi wrote:
>
> > ...> When I use this prog I am able to get upto 100000 rows. How to get
> > > the rows beyond this till the end of file?
>
> > > block_size = 100000;
> > > format = '%f %f %f %f %f %f';
> > > file_id = fopen(fno{i});
> > > cnt=0;
> > > segarray = textscan(file_id, format, block_size);
>
> > ...
>
> > Don't specify N and textscan() should read to EOF
>
> > Alternatively, see
>
> > doc textscan
>
> > and note one can call textscan repeatedly on the same fid and continue
> > from where left off.
>
> > Doc doesn't indicate it, but N=-1 in textread() is a flag for "read to
> > end of file"; one would presume that would have been implemented in
> > textscan() as well.  Also, I'd presume inf would have the same effect.
> > I can't test these hypotheses as my version predates textscan().
>
> > --
>
> Thanks for the response, however
>
> when i call textscan repeatedly on the same fid and continue
>  it does continue upto 100000 lines only after which it does not
> continue.
>
> eg the file has 1179919 lines
>
> segarray = textscan(file_id, format);
> segarray1 = textscan(file_id, format);
>
> I still get the size of segarray1 empty

You could try using my CSVIMPORT submission from FEX (http://
tinyurl.com/yjctr57).

HTH,
Ashish.