txt2mat

fast and versatile ascii data import capable of handling large text files

Description

As txt2mat basically is a wrapper for sscanf, it quickly converts ascii files containing m-by-n numeric data, allowing for header lines. When encountering lines with different numbers of data elements, it will work line-by-line and thus slow down somewhat.  
 
You may let txtmat carry out an automatic data layout analysis on comparatively 'simple' text files (header lines + decimal number data with common delimiters). By this analysis it is able to directly import most .csv-files, for instance.  
 
As txt2mat can perform string replacements before the numeric conversion is done, it can cope with many irregularities within the data. By that it is also capable of detecting and handling commas as decimal characters (common german notation).  
 
You can filter out bad data lines by keywords, provide appropriate format strings (as for sscanf), or split up the import process for huge files if you encounter memory problems (e.g. a typical regular 40MB file should be imported as a whole within less than 10s on most computers).  
 
txt2mat is intended to work on Matlab R13 and newer versions.  
 
Comments and suggestions welcome.  
 
Andres

• Acknowledgements: This submission has inspired the following:; readMM_2D
• MATLAB release: MATLAB 7.7 (R2008b)

[vincenzo ficco]

 

[Florian H]

Excellent! Finally a function that handles textfiles without the hassle of the builtin MATLAB ones.  
 
I've combined it with  
http://www.mathworks.com/matlabcentral/fileexchange/loadFile.do?objectId=15294  
to get a really usefull drag-and-drop import funtion for various textformats.

[Wladimir Alonso]

It worked beautifully when I needed to read solar data tables (http://rredc.nrel.gov/solar/old_data/nsrdb/1991-2005/list_by_state.html) that contained headers, text and numbers (in this case using txt2mat(name_of_the_file,1,43,'ConvString',['%d-%d-%d,%d:%f' repmat(',%f',1,38)]) as kindly suggested by the author of the function)  
thanks Andres!

[K AM]

Fantastic script. Worked right out of the box. I used it bring in oddly shaped data files and it worked like a dream! Thanks Andres!

[wu zhiyong]

Great work!  
Thanks for your suggestion!  
It helps me read the data from text files with ignoring the characters.But I fail to ignore the blank lines.  
Do you have any suggestion?

[Andres T.]

@wu zhiyong  
Please see my posting in the newsgroup.

[rodrigo abarca del rio]

Excellent work. It helped me in reading data from different format, and skipping the headlines ... in just a second, after having tried over more than 4 hours by different methods. we are close to fortran now :-)  
thanks so much a lot for your work.

[djr djr]

 

[Ralf]

 

[Zahra]

A powerful code. I am using it (with some help from the author of the code) to read very complicated data files with headerlines throughout the file as well as data lines with different lengths (line folding).  
 
And it is quite fast too.  
 
 
Thanks Andres for your great work!  
 
Zahra