Why can't MATLAB open this csv file properly?

I've got a .csv file containing data that I'd like to read into MATLAB. When I use csvread(filename, 1, 0) to tell it to skip the header line, I get this error:

Error using dlmread (line 138)
Mismatch between file and format string.
Trouble reading number from file (row 1u, field 1u) ==>

Error in csvread (line 47)
    m=dlmread(filename, ',', r, c);

While troubleshooting, I think I've discovered part of the problem. Opening the .csv in notepad shows me this text for the first two lines:

Time(s),Absorbance (AU)
1.5,0.0681447982788086

All other entries look as I'd expect. However, opening the same file in MATLAB shows me this:

ÿþT i m e ( s )   A b s o r b a n c e   ( A U )

All of the other entries are blank.

What is causing this issue, and how can I resolve it?

[dpb]

You've got some non-ASCII characters in the file, apparently...

>> double('ÿþ')
ans =
 255   254
>>

How best to resolve it depends largely on how the file was generated and whether you can go back and create it again without those issues.

Otherwise you're forced into trying to cleanup a bum file format and that's a quagmire of unknown depth and complexity depending on just what is in it and where and whether it's consistent between files and the phase of the moon and...

[sadc]

Looks like you're right. Apparently those two characters indicate the file is in unicode. I'm a little surprised MATLAB can't handle that gracefully but we'll see if I can find a way to output in ASCII instead.

[Walter Roberson]

No need, see my code.

Answer 1

[Star Strider]

The dlmread function and its friends (such as csvread) read only numeric data. You can try reading it with xlsread (if it’s Excel-compatible), or with textscan.

The function you use depends on what’s in the file and how it’s organised.

[Walter Roberson]

Current versions of dlmread() can handle text headers -- the requirement for pure numeric was dropped around R2015a or so.

[Star Strider]

You’re correct. I didn’t catch that change, but then I usually use either xlsread or textscan, in my own work and here in MATLAB Answers.

(Points should be awarded for reading the Release Notes!)

Answer 2

[Walter Roberson]

The 'ÿþ' is the characteristic sign of UTF-16-LE ("little endian") encoding of a file.

Depending on your version of MATLAB, dlmread() might not be able to skip the text header.

If you open the file as a text file, MATLAB should be able to figure out the encoding based upon the byte order marks.

Try:

fid = fopen(filename, 'rt');
datacell = textscan(fid, '%f,%f', 'HeaderLines', 1, 'CollectOutput', 1);
fclose(fid);
data = datacell{1};