Path: news.mathworks.com!not-for-mail
From: <HIDDEN>
Newsgroups: comp.soft-sys.matlab
Subject: Re: Problems with importing large text files
Date: Wed, 17 Dec 2008 10:50:19 +0000 (UTC)
Organization: ErasmusMC
Lines: 45
Message-ID: <gialhb$d8t$1@fred.mathworks.com>
References: <giajg4$f3t$1@fred.mathworks.com>
Reply-To: <HIDDEN>
NNTP-Posting-Host: webapp-02-blr.mathworks.com
Content-Type: text/plain; charset="ISO-8859-1"
Content-Transfer-Encoding: 8bit
X-Trace: fred.mathworks.com 1229511019 13597 172.30.248.37 (17 Dec 2008 10:50:19 GMT)
X-Complaints-To: news@mathworks.com
NNTP-Posting-Date: Wed, 17 Dec 2008 10:50:19 +0000 (UTC)
X-Newsreader: MATLAB Central Newsreader 1095751
Xref: news.mathworks.com comp.soft-sys.matlab:507477


You can write your own import function using fopen:

Assuming your data looks something like this:
Comments
2.35 93.1 294.1 593.1
23.1 345.4 127812947.12834 1282881.1

Then:
A = zeros(100000, 101);  % This is ~ 800MB, are you sure you have enough memory?

fid = fopen('data.txt', 'r');
row = 1;

% Skip first line
tline = fgetl(fid);

while 1
  tline = fgetl(fid);
  if ~ischar(tline), break, end
  A(row, :) = str2double(regexp(tline, ' ', 'split'));
end
fclose(fid);

If you have binary data, use fread instead.

Good luck.


"Sven B?" <sven.boehm@contiautomotive.com> wrote in message <giajg4$f3t$1@fred.mathworks.com>...
> Hi,
> from my PSPICE-simulation I get a text-file with the simulated data.
> The text file is approx. 500MB.
> The data in the file consists of 101 columns and approx. 100000 rows.
> For Matlab it is not possible to import this file via the Import Wizard.
> The first row is normal text, the others should be numbers.
> The first row has no special delimiter. Sometimes there are 2 sometimes 4 blanks as delimiter.
> Each column is the result of a Monte Carlo analysis.
> 
> The Import wizard starts up an then hangs on the point "creating preview".
> 
> Does anyone can help me?
> 
> Regards,
> 
> Sven