Code covered by the BSD License  

Highlights from
Read large CSV file in column and save each column in mat file

image thumbnail

Read large CSV file in column and save each column in mat file

by

 

07 May 2012 (Updated )

This is a draft to help you for transferring large CSV file or dat file into mat file

csvscan.m
% large csv file or dat file reader.
% read csv file in column and save each column in mat file.

clear all;
text=[];
textarray=[];
dataarray=[];
data=[];
readflag=1;

%while ~feof(file_id)

for column=1:100   % if the column number is unknown, you can try while function
     
      file_id=fopen('Vw1.8post.csv'); % my file is Vw1.8post.csv, you should use your own filename

      format = [repmat('%*s ',1,(column-1)),'%s %*[^\n]'];

      arr= textscan(file_id,format);  % textscan locate the column which is defined in format.

      if isempty(arr{1})
         readflag=0;
         disp('end?/empty?');
         break;
      end         

      data=str2double(arr{1}(6:end)); % in my file, the numeric data starts from 6th
%*--------------------------------------------
% you can use the other function to locate the first numeric value
%*--------------------------------------------

      dataarray=[dataarray,data];

      text = arr{1}(1:5);

      textarray=[textarray,text];

      flag=mod(column,2);
     
      if ~flag
          filename = [' 1dot8_postnum_' int2str(column/2) '.mat ']; 
          eval(['save',' ', filename,' ', 'dataarray']);
          filename2 = [' 1dot8_posttex_' in t2str(column/2) '.mat ']; 
          eval(['save',' ', filename2,' ', 'textarray']);
          dataarray=[];
          data=[];
          textarray=[];
          text=[];
      end
    
          fclose(file_id);
end


Contact us