How can I read in a fixed width ASCII file which contains missing numeric data in MATLAB?

I have a fixed-width (or fixed-length) ASCII file which contains numeric and text fields. Some of the data in the numeric fields is missing. I am unable to read in this data using the TEXTSCAN function in conjunction with the string and numeric formatting options.

Here is an example of some fixed width data I am trying to read:

Jon123.4Mr
Ann27832Ms
Max     Dr

The last line of data has five spaces between the 'Max' and the 'Dr'. This data is located in a file called test.dat. I attempted to read it in the following way:

fid = fopen('test.dat');
fmt = '%3s %5.2f %2s';
D = textscan(fid,fmt,'whitespace','')
fclose(fid);

When I execute this, the missing field does not appear as an empty array and the last field, 'Dr', is not read in.

Answer 1

[MathWorks Support Team]

As of R2017a, you can use a "FixedWidthImportOptions" object to specify how MATLAB imports fixed-width tabular data from text files.  Please see the following documentation page for more information:

https://www.mathworks.com/help/matlab/ref/matlab.io.text.fixedwidthimportoptions.html

Prior to R2017a, the ability to read in fixed width data with missing numeric values was not available in MATLAB.

To work around this issue, read in the fields as strings of the appropriate widths. Then convert the appropriate fields to numbers using the STR2NUM functions. This can be done is a single line with the CELLFUN function. Setting UniformOutput to false allows for missing values. For example:

fid = fopen('test.dat');     
fmt = '%3s %5s %2s';                     
D = textscan(fid,fmt,'whitespace','');           
D{2} = cellfun(@str2num, D{2}, 'UniformOutput', false)
fclose(fid);

At the end of this script, the variable "D" holds the text and numeric data in a cell array.

[dpb]

I'm not absolutely positive but I'd be very surprised if TMW has all of a sudden done anything this earth-shaking without it being big news in the release notes.

[dpb]

True, but requires human intervention...will the tool generate generic code that will subsequently read any other file of the same format or is it specific to the specific locations missing in the sample file (as I wold expect)?

[per isakson]

At last, see FixedWidthImportOptions, Import options object for fixed-width text files. Introduced in R2017a

[dpb]

Lot of effort to go to mimic Fortran FORMAT statement... :)