File Exchange

image thumbnail

Robust Data File Reading utility (RDFREAD)

version 1.0.0.0 (2.96 KB) by Michael Boldin
Imports data from a comma- or tab-delimitted file.

2 Downloads

Updated 01 Nov 2004

No License

RDFREAD handles comma-delimitted and tab-delimitted files and expects, but does not require the variable (column) names to be in the first line. Data may have missing values and character values mixed in the numeric data columns. All non-numerics in data lines are changed to NaN. This function does not properly read character variable columns, but instead turns such data into an entire column of missing values.

If the firstline has no numerics, it is assumed to be a header line with variable names. In this case a structure class is created with the fields name.varnames & name.data. If there is no headerline, a simple data matrix is created.

If there is no output argument (nargout=0), the results are put into base memory using filename as name.

examples
RDFREAD -- no input args, uses GUI file selection (uigetfiles)
RDFREAD(1) -- command line version
RDFREAD('file_name','C:\data_path\') -- works (path is optional)
xdat= RDFREAD('file_name') -- results placed in xdat

Below is an example of a messy data file that this utilty fucntion can read.

SECID,DATE,PRC,X1
10001,19970131,8.625,1
10002,1997215,13,
10003,,-15,4
1099a,o.0,.T,NaN
xd,,.,1.111
999,2,3,6

In this case, the RDFREAD 'data' results are

10001 19970131 8.625 1
10002 1997215 13 NaN
10003 NaN -15 4
NaN NaN NaN NaN
NaN NaN NaN 1.111
999 2 3 6

and filename.varnames=

'SECID' 'DATE' 'PRC' 'X1'

Note that this function is considerably slower than i/o routines
such as LOAD and DLMREAD because it reads and parses one line
at a time, but it works in a much more robust manner as long as you
do not need to read in 'character' columns.

10/25/2004 -- added block pre-allocation step to speed processing of large files

Cite As

Michael Boldin (2020). Robust Data File Reading utility (RDFREAD) (https://www.mathworks.com/matlabcentral/fileexchange/6090-robust-data-file-reading-utility-rdfread), MATLAB Central File Exchange. Retrieved .

Comments and Ratings (5)

great tool.

i just added

fclose(fid)

at the very end of the function. Otherwise it gave an error when calling repeatedly the function ("too many files open")

Andreas Dein

Awesome! Worked very well! And very robust, too!

Tom Stafford

Worked great for me! Neat

Kat F

It makes Matlab newbie's lives (like myself) a lot easier.

Francis Pieraut

It work pretty well except if values are string, they are replace by NaN considering it return an array of double.

Updates

1.0.0.0

Added block pre-allocation step to speed processing of large files. M-file updated.

10/25/2004 -- added block pre-allocation step to speed processing of large files

MATLAB Release Compatibility
Created with R14SP1
Compatible with any release
Platform Compatibility
Windows macOS Linux