Convert large cell array to double

Hi, I have problem with conversation cell array to double. I have this format:

 'gfc' '0'  '0'  '0.100000000000e+01'  '0.000000000000e+00'
 'gfc' '1'  '0'  '0.000000000000e+00'  '0.000000000000e+00'
 'gfc' '1'  '1'  '0.000000000000e+00'  '0.000000000000e+00'
 'gfc' '2'  '0'  '-.484165343771e-03'  '0.000000000000e+00'
 'gfc' '2'  '1'  '-.316745955888e-09'  '0.141736103677e-08'
 'gfc' '2'  '2'  '0.243935242430e-05'  '-.140031280033e-05'

and I use this code:

 for i = 1:4
     model(:, i) = sscanf(sprintf('%s\v',model_c{:,i+1}),'%f\v');
 end

When I convert a small model which it has degree and order 100 (about 5 500 lines), my code works. But when I want convert large model with degree and order for example 2200 (about 2 500 000 lines), code crashed on the third column. And Matlab writes 'Subscripted assignment dimension mismatch'. Can someone help me?

Answer 1

[Stephen23]

"But when I want convert large model with degree and order for example 2200 (about 2 500 000 lines), code crashed on the third column."

One or more of the cells are not being converted to scalar numeric values, and so the number of elements that you have on the RHS differs from what is required by the indexing on the LHS. You can try to find out which cells by doing something like this:

[row,col] = find(isnan(str2double(C(:,2:5))))

Also note that the for loop is not required, as you can easily convert all values at once, e.g.:

   >> C = {...
   'gfc' '0'  '0'  '0.100000000000e+01'  '0.000000000000e+00'
   'gfc' '1'  '0'  '0.000000000000e+00'  '0.000000000000e+00'
   'gfc' '1'  '1'  '0.000000000000e+00'  '0.000000000000e+00'
   'gfc' '2'  '0'  '-.484165343771e-03'  '0.000000000000e+00'
   'gfc' '2'  '1'  '-.316745955888e-09'  '0.141736103677e-08'
   'gfc' '2'  '2'  '0.243935242430e-05'  '-.140031280033e-05'
  };
  >> M = sscanf(sprintf('%s\v',C{:,2:5}),'%f\v');
  >> M = reshape(M,size(C,1),[]);
  M =
     0.00000   0.00000   1.00000   0.00000
     1.00000   0.00000   0.00000   0.00000
     1.00000   1.00000   0.00000   0.00000
     2.00000   0.00000  -0.00048   0.00000
     2.00000   1.00000  -0.00000   0.00000
     2.00000   2.00000   0.00000  -0.00000

[Richard Kratochvíl]

Thank you for your advice. One line had format 1.12345D-13 instead of 1.12345e-13. I'm stupid I did not check the input data.

Answer 2

[Guillaume]

Yes that seems overly complicated. Wouldn't

model = str2double(model_c(:, 2:5));
%or
model = cellfun(@(c) sscanf('%f', c), model_c(: 2:5));

be faster and certainly simpler.

Even better would be to fix whatever is generating the cell array so that it creates a numeric matrix in the first place. If you're importing that from a text file, fixing the import would be better.

[Richard Kratochvíl]

Thank you for your investigation. But str2double was my the first try. It doesnt work for large cellarray and it is very, very and very desperately slow. If I used str2double, convert time is aprox 0.25 sec. When I use my code, convert time is aprox 0.019 sec. I tried it on model with degree and order 100. The cellfun is slow and doesnt work too.

[Stephen23]

"Wouldn't ... be faster and certainly simpler."

Not faster. Actually this code:

sscanf(sprintf('%s\v',model_c{:,i+1}),'%f\v');

is likely the fastest way to convert multiple cells of char vectors to numeric. str2double, and any other inbuilt functions will be slower than this. See:

https://www.mathworks.com/matlabcentral/answers/18509-cell-conversion-to-double

I did a lot of experimentation on this for my FEX submission natsort, and the results were quite clear that sprintf and sscanf is hard to beat.

[Guillaume]

Algorithmically, it makes no sense to first concatenate all the strings to then parse them instead of just parsing them. The concatenation is an unneeded step.

Since str2double is slower it clearly is not implemented efficiently. What you need is a sscanf that supports cell arrays. Then the concatenation would not be necessary.

Thankfully, I don't usually have to deal with this as all the import I do is binary. Much more efficient!