First colum of CSV-imported table has "x___" added to its name

I have a bunch of automatically-generated CSV files with headers, which I'd like to import into Matlab as a table. I used code such as

T = readtable('d:\test.csv', 'readvariablenames', true);

However, even though the name of the CSV's first column is runNr, the first column in the Matlab table gets named "x___runNr"

This clearly has something to do with the CSV files being in a slightly format different from that expected by Matlab. For instance, if I manually change the name of that first cell in Excel to something else, then reopen the CSV, the cell contents are all merged into a cell, on every row.

Still, I am not sure what to do to fix this, since I cannot change the format of the CSVs.

Any help?

[Star Strider]

It imports correctly for me in R2019b (Update 1):

T =

      runNr    repNr    trialNr            file        
    _____    _____    _______    ____________________

     NaN      NaN       NaN      {0×0 char          }
     NaN      NaN       NaN      {0×0 char          }
     NaN      NaN       NaN      {0×0 char          }
       1        1         1      {'stimuli/12C1.wav'}

(Posting only a relevant subset of the columns.)

Answer 1

[Jeremy Hughes]

The file probably has a Byte Order Mark in the beginning.

If you read the raw bytes using fopen/fread you can see the mark at the beginning.

[Stephen23]

The file is encoded as UTF-8 with BOM.

Surely READTABLE et al can cope with BOMs... !

[Jeremy Hughes]

According to Star Strider, it can in 19b.

[z8080]

Thanks! The strange thing is that these files were all imported correctly (without the x___ at the start of the first variable's name) a few days ago, even though nothing has changed in them, nor in the line of code pasted above.

I guess I will just need to upgrade to 2019b then, if no immediate workaround is available with readtable. :(

[z8080]

I forgot to mention that upon calling readtable, the following warning is issued, even though, as I say, the variable names still looked fine until very recently:

"Warning: Variable names were modified to make them valid MATLAB identifiers. "

Some of my CSVs (perhaps produced by a different version of the software that outputs them) are still read OK, and for those CSVs also the warning above is displayed.

Is that at all a helpful hint as to how I can go round the problem?

[Steven Lord]

I would check with whomever generated the file to learn if the process of generating those files and/or the software used to generate the files changed between the last time they were imported without the "x_" prefix and the first time they were imported with the prefix. It's possible they made a change that they thought would not affect you but it turns out to have affected you.

[z8080]

Makes sense Steven, thanks. I now used the most recent version of that software (psychoPy), and the same problem occurs, thus it is not a question of the version of it. Beyond that, I'm not sure how much I can find about the structure of these CSVs, and have to just import them as they are. For now I will just hard-code "x___VariableName" as an acceptable variable name...

[Steven Lord]

You could post-process the imported table to change the variable name. Something like this should work, though I haven't tried it:

mytable.Properties.VariableNames{1} = ...
    strrep(mytable.Properties.VariableNames{1}, 'x___', '');

[z8080]

Perfect, thank you!