Discover MakerZone

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn more

Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply Today

New to MATLAB?

Import CSV file to 2012b make Windows 8 Entire UI hang!

Asked by mingming wang

mingming wang (view profile)

on 16 Oct 2012

I just try to import CSV file, which has 1054419 rows and 16 cols. However, during import, windows 8 has no response at all, the entire UI hang.

I try many times but fail. 2012b preformance is too bad.

I already try to use csvread(), however, it also return error.

Error using dlmread (line 139) Mismatch between file and format string. Trouble reading number from file (row 1u, field 3u) ==> x0000,0,0x0012,1,0,0,3,66,20,2012,6,21,10,44,4\n

Error in csvread (line 48) m=dlmread(filename, ',', r, c);

here is the sample data: 2nd and 4th cols data type is HEX:

18251,0x0000,0,0x0012,1,0,0,3,66,20,2012,6,21,10,44,4 18251,0x0000,10,0x0010,1,1,38,0,226,2432,2012,6,21,10,44,4 18251,0x0000,12,0x000e,0,0,0,20,28,84,2012,6,21,10,44,4 18251,0x0000,13,0x0000,0,1,85,0,91,5440,2012,6,21,10,44,4 18251,0x0000,15,0x000e,0,1,23,0,25,1472,2012,6,21,10,44,4 18251,0x0000,16,0x0000,0,1,44,0,51,2816,2012,6,21,10,44,4 18251,0x0000,18,0x000e,0,0,0,15,17,72,2012,6,21,10,44,4 18251,0x0000,23,0x000e,0,0,0,15,21,68,2012,6,21,10,44,4 18251,0x0000,23,0x000e,0,1,1,0,1,64,2012,6,21,10,44,4 18251,0x0000,25,0x000e,0,1,55,0,225,3520,2012,6,21,10,44,4 18251,0x0000,30,0x000e,0,1,121,0,244,7744,2012,6,21,10,44,4 18251,0x0000,36,0x000e,0,0,1,20,30,92,2012,6,21,10,44,4 18251,0x0000,37,0x0000,0,1,47,0,56,3008,2012,6,21,10,44,4 18251,0x0000,38,0x000e,0,1,78,0,104,4992,2012,6,21,10,44,4 18251,0x0000,42,0x0000,0,0,86,0,87,2680,2012,6,21,10,44,4 18251,0x0000,42,0x0000,0,1,747,0,1490,47608,2012,6,21,10,44,4 18251,0x0000,43,0x0000,0,0,0,15,55,64,2012,6,21,10,44,4

2 Comments

Andreas Goser

Andreas Goser (view profile)

on 18 Oct 2012

Only a comment as not necessarily related to your issue. Win 8 is not a supported OS for R2012b: http://www.mathworks.com/support/sysreq/current_release/

mingming wang

mingming wang (view profile)

on 18 Oct 2012

However, Before I upgrade to 2012b, I use 2012a on windows 8 and import this csv with no issue. Though Matlab itself may has no response during import, it does not lead to the whole windows 8 UI hang, other programs run normally. For matlab, I always care performance first, since I only process large amount data. I also do not understand why Matlab use Jave to develop UI. Java == Slow.

mingming wang

mingming wang (view profile)

Products

3 Answers

Answer by mingming wang

mingming wang (view profile)

on 16 Oct 2012
Edited by Walter Roberson

Walter Roberson (view profile)

on 18 Oct 2012

I wish data import wizard could provide more powerful data convert rules. Not only replace Blanks with NaN, but also like Hex data convert to Decimal; replace specific string to other value...

I just try to import data by wizard, exclude 2nd and 4th Hex data cols only. Windows 8 UI still Hang. So this is still a performance issue.

0 Comments

mingming wang

mingming wang (view profile)

Answer by Lisa Justin

Lisa Justin (view profile)

on 18 Oct 2012
Edited by Lisa Justin

Lisa Justin (view profile)

on 18 Oct 2012

you can try fopen, feof, fgets,regexprep,sscanf and fclose

or

fid=fopen(filename.csv')

a = fread(fid,1,'uint32','l');

 fprintf('%X',a)  b = hex2dec(a); 
a=dec2hex(fread(fid)) 

1 Comment

mingming wang

mingming wang (view profile)

on 18 Oct 2012

I try your code, but meet error:

>> fid=fopen('csvnew.csv');
>> a = fread(fid,1,'uint32','l');
>> fprintf('%X',a)
35323831>> b = hex2dec(a); 
Error using hex2dec (line 38)
Input string found with characters other than 0-9, a-f, or A-F.

I open that hex2dec.m file and see the source code:

% Check for out of range values
if any(any(~((h>='0' & h<='9') | (h>='A'&h<='F'))))
   error(message('MATLAB:hex2dec:IllegalHexadecimal'));
end

You can see that if statement. It only judge 0-9, A-F, but not include a-f. However your error message show: a-f. I update my CSV file chars: a-f to upper case: A-F, then re-run the code. But I meet the same error :(

Lisa Justin

Lisa Justin (view profile)

Answer by Walter Roberson

Walter Roberson (view profile)

on 18 Oct 2012

fopen(), textscan(), fclose(). If you use textscan(), read the hex fields as strings and convert them after using sscanf() with a %i format item.

Alternately, fopen(), fscanf(), fclose(). Use %i for the format items: it will convert the 0x fields but leave the others as decimal.

0 Comments

Walter Roberson

Walter Roberson (view profile)

Contact us