Table versus Cell Array to append a disp function that logs steps in a script file

Hello, I am new to Matlab and was hoping to get some opinions on the best approach of creating a log file that will ultimately be inserted into SQL at the very end. Throughout the script file I have been writing disp which is great when you use the matlab compiler as it adds it to a log file. But you can't append to it and you can probably use fprintf to append. But, I decided to store in SQL instead. So the question. I would ideally like to create a table in Matlab with Columns defined and append values (will no longer use disp) to that table as the script is run. At the end of the script I will use runstoredprocedure to insert the values from the Matlab table into SQL table. Any ideas? Table or cell array?

Answer 1

[Walter Roberson]

Cell array should be more efficient, especially if you can pre-allocate the cell array to the maximum size. (If you know the maximum number of entries, then you can allocate that many cells even if you might not use them all, as you can remove the unused ones somewhat efficiently at the end.)

Even if you have to transform into a table at the end for the purpose of inserting into the database, the cell array should be more efficient.

If you have a new enough MATLAB (R2016b or later) you could also consider using string arrays. String arrays use less memory then cell arrays of strings do.

[Mark Mcgrath]

Thanks. I will have a preset number of entries per script file. I just wanted to use a table as I will have a mismatch of data types. Strings, numbers and datetimes.

[Peter Perkins]

"Efficient" is kind of a broad term. It's difficult to give recommendations without knowing more about the specifics of what you have, but in general terms ...

If your data are mostly numeric, and tall not wide, the memory required by a table will be much smaller than that for a cell array, because the table stores each of its variables as one MATLAB array whereas a cell array stores each of its cells as one MATLAB array:

>> t = array2table(randn(1000,10)); >> c = table2cell(t); >> whos Name Size Bytes Class Attributes

c         1000x10            1200000  cell               
t         1000x10              82952  table

If possible (and it sounds like it is possible), you want to preallocate the table and overwrite existing rows, rather than writing to end+1, and that's true anywhere in MATLAB.

Walter, I think what you're referring to is that the actual assignment for a row of a cell array is less expensive than assignment to a row of a table. True, but whether or not that matters depends on what else is going on. And the syntax would be essentially the same:

>> t = table([0;0;0],["";"";""])
t =
  3×2 table
    Var1    Var2
    ____    ____
    0       ""  
    0       ""  
    0       ""  
>> t(1,:) = {1 "one"};
>> t(2,:) = {2 "two"};
>> t(3,:) = {3 "three"};
>> t
t =
  3×2 table
    Var1     Var2  
    ____    _______
    1       "one"  
    2       "two"  
    3       "three"