importing large amounts of text and numerical data from csv or txt files
1 view (last 30 days)
Show older comments
Hi everybody
I have a very large csv or txt file with numerical as well as text data. I am looking for a way to import this data, beginning from the third row. The text strings in the data ("NA") should be represented as NaN.
The original csv data represents approx. 1000 columns and 500 rows, so specifying the handling of each column is not feasible.
Any ideas?
Thank you, best,
Chris
0 Comments
Accepted Answer
Walter Roberson
on 26 Nov 2011
numcols = 1000;
fmt = repmat('%f',1,numcols);
fid = fopen('YourFile.csv', 'rt');
indata = textscan(fid, fmt, 'Headerlines', 2, 'Delimiter', ',', 'TreatAsEmpty', 'NA', 'CollectOutput', 1);
fclose(indata);
indata = indata{1};
5 Comments
Sven
on 27 Nov 2011
When working with reading/writing files, by default the "pointer" to the current location in the file is always at the end of the last piece that was read. This means that you can do things like reading a file line-by-line without explicitly needing to move the pointer of the current location to "current location plus whatever-the-length-of-the-last-line-was". In other words, it's easier under normal operation, but needs an extra piece of code for your case where you want to go back and re-read that portion.
Walter Roberson
on 27 Nov 2011
fseek(fid, 0, 'bof')
will move the file pointer back to the beginning of the file.
More Answers (0)
See Also
Categories
Find more on Data Import and Export in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!