How to ignore comment character from header/variable names line

Hi!

I'm importing a text file using readtable with row/column data and the first couple of lines are commented which includes the variable names. When I specify a line to get variable names from, it takes the comment character as a variable name and shifts my variable names by one column.

How do I have readtable ignore the comment character for that line or use other methods align my variable names correctly?

Thank you!

Code:

data_spray = readtable('spray.out', FileType='text', CommentStyle='#', VariableNamesLine=2);

Data File Snippet: spray.out

# column        1                2                3                4                
#           Crank      tot_parcels    spray_parcels   liq_spray_mass           
#           (DEG)      (drop+film)           (drop)             (kg)           
#
    -1.5200000e+02                0                0    0.0000000e+00
    -1.5200000e+02                0                0    0.0000000e+00    
    -1.5188523e+02                0                0    0.0000000e+00    
    -1.5177080e+02                0                0    0.0000000e+00   
    -1.5163891e+02                0                0    0.0000000e+00
 

And this is what the imported table looks like in Matlab:

Answer 1

[Cris LaPierre]

When an import function has an input that allows you to specify CommentStyle, that is so that it knows to ignore everything after the comment character. So even if you used that option in readtable, it still wouldn't help.

My first thought would be to add your comment character as a Delimiter, and then tell readtable to ignore leading delimiters. Perhaps something like this. Note that I had to change the file extension to upload the file here, but it works on my desktop using 'spray.out'.

opts = detectImportOptions("spray.txt","FileType","text","VariableNamesLine",2);
opts.Delimiter{end+1} = '#';
opts.LeadingDelimitersRule = "ignore";
data_spray = readtable("spray.txt", opts)

data_spray = 5×4 table
     Crank     tot_parcels    spray_parcels    liq_spray_mass
    _______    ___________    _____________    ______________

       -152         0               0                0       
       -152         0               0                0       
    -151.89         0               0                0       
    -151.77         0               0                0       
    -151.64         0               0                0       

[Musharrat Chowdhury]

Thank you, Cris!

That works really well; the only caveat with this is now it reads the commented line which sometimes have numerical values (like line 1).

I don't suppose there is any way for '#' to be a both a comment character and a delimiter?

N.B.: Keeping the question open for discussion or any further solutions.

[Cris LaPierre]

Just a reminder that making the comment symbol a delimiter is a workaround to fix this particular issue. There are many more situations where this will not work than where it will. The fix to reading other header lines would be to include additional options like the number of header lines, the data range, etc.

opts = detectImportOptions("spray.out","FileType","text","VariableNamesLine",2,...
    "NumHeaderLines",4);
opts.Delimiter{end+1} = '#';
opts.LeadingDelimitersRule = "ignore";
data_spray = readtable("spray.out", opts)

or

opts = detectImportOptions("spray.out","FileType","text","VariableNamesLine",2,...
    'Range',5);
opts.Delimiter{end+1} = '#';
opts.LeadingDelimitersRule = "ignore";
data_spray = readtable("spray.out", opts)

I do not believe there is a way to have something be both a comment and a delmiter.

[Musharrat Chowdhury]

Thank you! This solution works nicely for this specific issue.