opts.VariableTypes for any number of variables

Hi, I want to be able to import a table with ANY number of columns into my MATlab code.

opts.VariableTypes = ["double", "double", "double", "double", "double", "double", "double", "double", "double", "double", "double", "double", "double", "double", "double", "double", "double", "double", "double", "double", "double", "double", "double", "double", "double"];

With the above line, readtable works perfectly on my 25-variable (25-column) .csv file. My entire gui works exactly as I need, but only on this exact number of variables.

However, I need to be able to import ANY number of variables, all as doubles. Everything I've tried has given me error messages when I try to plot the imported table in my GUI later on. Three of the many things I tried are below. I'm sure some of them look really dumb to anyone who knows MATlab- I apologize, I was just trying some things out on my own.

This one did not work:

opts.VariableTypes = "double";

Neither did this one, which I tried because all of my variable names start with D:

opts = setvartype(opts,{'D*'},{'single','string'});

Nor did this one:

opts.VariableTypes = ["double", ... ];

[Simon]

I have the same problem. The help content of 'htmlImportOptions' recommends this method, and it works for me:

opts = setvartype(opts,"string");

This also works:

opts = setvaropts(opts, 'type','string');

One method you mentioned also works for me when delimited..... line is commented out and keep

numVariables = length(opts.VariableNames);
opts.VariableTypes = repmat("string", 1 , numVariables);

Answer 1

[Steven Lord]

opts.VariableTypes = repmat("double", 1, 25);

Change the number 25 in that line of code as appropriate for your file.

[Corrine Rybarski]

Is there a way to have it input the number of variables automatically? I just want the user to be able to browse for the file and not have to change anything else.

[Steven Lord]

If you're creating opts via a call to detectImportOptions then the number of variables MATLAB thinks your file has is the length of the VariableNames, VariableTypes, or VariableOptions properties of opts or the variable options returned by getvaropts.

numVariables = length(opts.VariableNames)

[Corrine Rybarski]

yay!

opts = detectImportOptions("C:\Users\corrine\Desktop\scaracsv.csv");
numVariables = length(opts.VariableNames);
opts = delimitedTextImportOptions;
opts.VariableTypes = repmat("double", 1 , numVariables);

works perfectly!

thank you so much!

[Steven Lord]

I'm a little confused why you overwrite the opts variable created by detectImportOptions with the output of delimitedTextImportOptions (which I suspect your opts variable already is.)

From description of the output argument in the detectImportOptions documentation:

"For text files (.txt, .dat, or .csv), the detectImportOptions function returns a DelimitedTextImportOptions or FixedWidthImportOptions object."

From the delimitedTextImportOptions documentation:

"opts = delimitedTextImportOptions creates a DelimitedTextImportOptions object with one variable."

Why not eliminate that third line, so you overwrite the detected variable types but leave the rest of the detected options in place?

[Yulong Li]

Hi Steven, in my case, the variable names in the csv are not in the first row but in row 20, and "detectImportOptions" did not detect it corrrectly but used the first row. So "length(opts.VariableNames)" does not return the correct length. What can I do in this case to get the correct length? Thanks.

[Steven Lord]

What type of file are you passing into detectImportOptions? Which name-value arguments are allowed depends on the type of the file. If you have a text or spreadsheet file, have you tried specifying NumHeaderLines to skip the lines in the file prior to the row that contains variable names? Or for a spreadsheet file, perhaps VariableNamesRange would be useful as well or instead?

[Yulong Li]

It's a CSV file. I tried NumHeaderLines below, but still the VariableNames only took the first column of row 20.

Filename = 'xxx.csv';
opts = detectImportOptions(Filename, 'NumHeaderLines', 19); 

[Yulong Li]

Following up on this one, any other solutions as NumHeaderLines not making it detect all the variable names?

[Walter Roberson]

@Yulong Li

In the case where you have some headers to be ignored completely, then a line which gives the variable names, and then the data starts after that, then set NumHeaderLines to be the number of lines before the line that gives the variable names.

In the case where there are (possibly some headers to be ignored) then a line which gives the variable names, and then exactly one line that gives the units, and then the data, then you might be able to configure to read units.

In the case where there are (possibly some headers to be ignored) then a line which gives the variable names, then one or more lines to be ignored, then the data, then you will need to set the NumHeaderLines to be the number of lines before the data, but you must also use detectImportOptions (or equivilent) and set the variable names line number specifically.

In the case where there are a number of header lines that each have the same leading character, such as % or #, but the line with the variable names does not have that character, but any lines after the variable names and before the data do have that character, then you might be able to handle the situation by adjusting the CommentStyle option.

[Yulong Li]

@Walter Roberson Thanks for the response.

I have no problem setting the data line and variable line with DataLines and VariableNamesLine, and I'm getting the correct data reading in the table. However, I have problem setting the read in data type. I have tried to use setvartype(opts, 'double'), but it only sets the first column to double. I also tried

numVariables = length(opts.VariableNames);
opts.VariableTypes = repmat("double", 1 , numVariables);

But the length returns 1, because opts.VariableNames only detects the first variable name even if I set NumHeaderLines to the line before variable line when detecting options. My variable names do not have special format and all have "," as delimiter. I'm just wondering why the rest of the variable names can't be detected. Or if there's another way to set data type for all variables. Thanks.

[Walter Roberson]

@Yulong Li do you have a fixed number of variables, or a varying number?

In the case where you happen to have a fixed set of variable names, I would suggest readmatrix(), possibly followed by array2table() depending what you are doing.

[Yulong Li]

@Walter Roberson

Unfortunately the number of variables and the sequence of variables changes depending on the tests performed (those are RAM data logs from a debugger).

I did have a work around which is reading in the table as is and then converting the data type in the table. But it's not very efficient and can take a long time to process a big data log, that's why I'm trying to see if I can decide the data type at the time of readtable.

Answer 2

[Tamara del Águila]

opts = detectImportOptions("C:\Users\corrine\Desktop\scaracsv.csv");
numVariables = length(opts.VariableNames);
opts = delimitedTextImportOptions;
opts.VariableTypes = repmat("double", 1 , numVariables);

This does not work for me : (

I am using version R2019a. I am importing data from a csv with 'readtable' and also need to import a previously unknown number of variables of type 'double'. If I delete the line 'opts.VariableTypes', instead of a numeric matrix, I get a cell array...

[Walter Roberson]

opts = detectImportOptions("C:\Users\corrine\Desktop\scaracsv.csv");
opts = setvartype(opts, 'double');  %in the case of setting all variables

data = readtable(filenames, opts);