Readtable loop extreme slow

Hi,

I am trying to import 60 excel sheets of the same format from a single file into matlab.

F = " .xlsx"
S = SheetNames(F)
N = numel(S)
C = cell (1, N)
for k = 1:N
C{k} = readtable (F, 'Sheet',S(k)))
end

It took almost 40minutes to load a single file. I would really appreciate if someone could suggest a more efficient way.

[Matt J]

Why should we think anything is inefficient? Maybe your file is just really big.

Answer 1

[Walter Roberson]

If the sheets are potentially different formats (different columns, or different datatypes for some of the columns), there is not a lot you can do -- though telling to to use Excel might possibly speed things up in this case.

If the sheets are intended to all be the same format, then before the loop, use detectImportOptions() once, and pass the detected options to readtable().

When you do not supply import options to readtable(), readtable does a pre-scan of the sheet, trying to guess the correct variable type for each column. It is, for example, not uncommon to have cases where the first several thousand rows of a column are empty, and then has either a numeric or text entry, and that first entry might determine the column type. readtable() cannot just guess that an empty variable in the first row will be text or numeric.

When you detectImportOptions once and pass the result in, then readtable() can skip all of those scans to try to figure out what the columns are.