How to use textscan to properly read a csv file?

Hi everybody, I'm having a problem with trying to read a csv file. The file is 18 columns by 234 rows. The first 34 rows are either strings or blank, and I don't need them. I am interested in 4 specific columns filled with decimal numbers, as I need to take that data and make some calculations and plot it. But I can't seem to get over the initial step of correctly putting those 4 specific columns into an array or saving them as variables. Any help? Sorry if my question is confusing.

An example file is now attached.

[Stephen23]

@Zak Kessel: without any example of your file it will be difficult to help you more than say "read the documentation". When you edit your question and upload a sample file using the paperclip button then we can help you.

[dpb]

The previous answer/comment still holds and you didn't yet tell us which of the columns it is you're actually interested in--none look terribly interesting from the sample data file so it'd be pretty hard to guess.

But, make the format string as outlined in my previous post to fit and you're "good to go"; you've got an exact analogy the the previous situation outlined there.

[Zak Kessel]

Sorry I'm interested in columns 1, 8, 9, and 16. I can't put any real data into the spreadsheet so I tried making it as general as possible. So essentially I need to pull out 4 columns that are filled with numbers. I'm currently trying to go through the situation you posted below. I'm a relative matlab beginner so I'm pretty slow at understanding everything that is going on.

[dpb]

Well, to skip a column you really don't care particularly whether it's numeric or character so to read numeric in the above columns it's

1. Read a number
2. Skip six columns, read two numbers
3. Skip six columns, read a number
4. Skip remainder of line (or two columns)

By analogy,

fmt=['%f' repmat('%*s',1,6)'%f %f' repmat('%*s',1,6)'%f' '%*s %*s'];

Your desired columns didn't lead to much symmetry so it's fairly bulky but still straightforward.

[Zak Kessel]

fileID = fopen('C:\Users\zk04478\Desktop\QuestionExample.csv')
data=cell2mat(textscan(fileID,'','delimiter',',','headerlines',34,'collectoutput',3))
fmt=['%f' repmat('%*s',1,6) '%f' '%f' repmat('%*s',1,6) '%f' '%*s' '%*s'];

This is my code above so far, and the resulting command window and value for "data" when I run it. Still unsure why it doesn't seem to be skipping the headers or the columns that I need. I'm very sorry, but I appreciate the help!

Answer 1

[dpb]

Well, you don't say which columns, but it's simple enough...and actually, if after the header/text lines it is all numeric then just use csvread --

data=csvread('yourfile.csv',34);  % read all the numeric data
idx=[2 7 12 13];                  % arbitrary for columns
data=data(:,idx);                 % save those for use

With textscan the above would be almost the same (after the fopen preamble, of course)

data=cell2mat(textscan(fid,'','delimiter',',','headerlines',34,'collectoutput',1));

NB: I went ahead and wrapped the textscan call in cell2mat to return a double array directly instead of the cell array one otherwise gets from textscan

[Zak Kessel]

The 4 columns I need to read aren't numeric but there are actually several columns that are numeric, so is it still possible to use csvread in that situation?

[dpb]

No, csvread is numeric only.

Also, then you'll need to unwrap the textscan call to return the cell array and if it is a mixed content by record in various columns you'll get two cell arrays; one with the numeric and another with the character data.

In this case if all you do really need is the specific columns, it may be just as well as to go ahead and build the specific format string skipping the unwanted columns, but to write a specific format string need to know which columns but the general idea would be to skip N, read M, skip, read, ... until defined the record. I just was involved in an extensive discussion in which such a pattern was built for numeric values, but the same basic logic is universal with specific fields matching what is in the file--

<answer_224353>

There's a detailed explanation of building the (apparently arcane) format string in one of the follow-up comments that I'd commend to your attention :) ) <fmt string detail>

[Zak Kessel]

I attempted this code below:

 fileID = fopen('C:\Users\zk04478\Desktop\QuestionExample.csv');

 fmt=['%f' repmat('%s',1,6) '%f' '%f' repmat('%s',1,6) '%f' '%*s' '%*s'];
 data= cell2mat(textscan(fileID, fmt,'delimiter',',','headerlines',34,'collectoutput',1))

And it gave me this error:

Error using cell2mat (line 45) All contents of the input cell array must be of the same data type.

Error in test3 (line 4) data=cell2mat(textscan(fileID, fmt,'delimiter',',','headerlines',34,'collectoutput',1))

[dpb]

Missing the '*' on some (if not all) of the '%s' fields to skip them...oh, I see I missed at least one initially, but will caution the user needs to carefully read and understand the posted code as well... :)

>> fmt=['%f' repmat('%*s',1,6) '%f %f' repmat('%*s',1,6) '%f %*s %*s']
fmt =
%f%*s%*s%*s%*s%*s%*s%f %f%*s%*s%*s%*s%*s%*s%f %*s %*s
>> fid=fopen('zak.csv')
fid =
   4
>> cell2mat(textscan(fid,fmt,'delimiter',',','headerlines',34','collectoutput',1))
ans =
   0     0     0     3
   1     0     0     3
   2     0     0     3
   3     0     0     3
   4     0     0     3
   5     0     0     3
   6     0     0     3
   7     0     0     3
   8     0     0     3
   9     0     0     3
  10     0     0     3
  11     0     0     3
  12     0     0     3
>> fid=fclose(fid);
>>

QED :)