Cannot import/load/read complete .dat file

Hi

I am trying to open my .dat file that has 785663 rows x 29 columns; each column separated by a comma (therefore, csv). I tried using textscan, importdata, csvread, load, etc in order to access my files and hence do some data analysis. The first 2 lines of my .dat file (excluding the headerline) look like this:

"2013-01-02 21:10:00",64359,18.33,20.41257,19.95976,11.60755,4.172891,56.92463,35.29119,10.93861,0,0,0,0,2.630428,0.4060569,0.7230025,0.3542804,-0.0027175,0.2566106,2.75231,2.727983,114.6311,15.36885,7.615118,99.7831,0,23.01129,12.45
"2013-01-02 21:15:00",64360,18.27,20.36502,19.92328,11.72347,4.227652,57.53248,35.50731,10.93866,0,0,0,0,2.427403,0.3657977,0.7309517,0.3855992,0.03224583,0.2124791,2.561898,2.535069,113.2416,16.75837,8.289121,99.78176,0,22.88334,12.47

The first column is date and the rest is all numeric.

I tried using the following code:

fid = fopen(filename,'r');

formatspec = ['%s' repmat('%f',1,29)];
A = textscan(fid, formatspec, 'HeaderLines',4,'Delimiter',',' ,inf, 'CollectOutput', true);

But my A cell array returns me only 11184 rows in some columns, and 11183 rows in other columns.

I also tried importdata, and again, it gave me only 11184 rows for all the columns which are numeric except the date column (11188 rows: because include the headerlines which are text).

In anyway, I couldn't get all my 785663 rows loaded. How can I get the remaining part of my dataset?

Thank you.

[Stephen23]

Did you look at rows 11184/11185? Is it possible that the data on these rows might be incomplete or not able to be parsed with either of the format specifiers that you used?

When I run the code I get an error "??? Error using ==> textscan Param/value pairs must come in pairs." This seems to come from the inf value which is entered as one of the textscan options. Removing this inf gives me a 1x2 cell... but with empty cells, due to not having header lines :)

I made some small changes to the code you supplied (particularly the string format), and this imports the data correctly:

filename = 'temp.txt';
fid = fopen(filename,'r');
formatspec = ['%q',repmat('%f',1,28)];
A = textscan(fid, formatspec, 'Delimiter',',', 'CollectOutput', true);
fclose(fid);

It is likely to be something specific to the rows where these formats do not parse the data...

Note that using importdata also works for me:

B = importdata(filename, ',');