- Your file is fixed width, not delimited.
- Every row does not contain delimeters for all 6 columns
importdata produces strange output when reading a textfile with blank cells and text
5 views (last 30 days)
Show older comments
I want to import the attached file which is the result of matlabs function run. As you can see, there are a couple of blank cells and lines of text at the end and beginning of the file. I am only interested in the numerical data, which is a 16x6 matrix for that particular txt file. The empty cells be NaN.
Importdata is able to separate the text from the numerical data, however, thinks that there are only 3 columns because there are three blank cells in the first row.
If I remove the text lines manually and call readtable(), there are, for whatever reasons, the first three lines of the numerical data missing.
How can I get the 16x6 matrix from the text file where blank cells = NaN?
Thank you!
0 Comments
Accepted Answer
Cris LaPierre
on 1 Feb 2023
You will have to take into the consideration the formatting of your file and may have to specify some additiional settings to get what you want.
VariableNames = {'Index', 'exitFlag', 'f(x)', 'iter', 'F-count', 'optimality'};
VariableWidths = [ 8, 10, 13, 11, 9, 14 ];
opts = fixedWidthImportOptions('VariableWidths',VariableWidths,'VariableNames',VariableNames);
opts.DataLines = [4 19];
opts = setvartype(opts,"double");
data = readtable("command_window.txt",opts)
7 Comments
Cris LaPierre
on 2 Feb 2023
I like to inspect text files in a text editor that can show non-visible characters (I use Notepad++)
Double checking here, the column widths are
VariableWidths = [ 8, 10, 13, 10, 10, 14 ];
dpb
on 2 Feb 2023
So do I (inspect w/ "real" programmer's editor), but many newbies aren't that familiar with other toolsets so used the MATLAB representation. I have a shareware/freeware version of the old Brief editor that works with Windows that is my comfortable tool; having used Brief ever since nearly its introduction the key mapping is just too ingrained to get used to something else.
More Answers (1)
dpb
on 1 Feb 2023
>> opt=detectImportOptions('command_window.txt','ExpectedNumVariables',6,'NumHeaderLines',2,"ReadVariableNames",1,'VariableNamingRule','preserve');
>> readtable('command_window.txt',opt)
ans =
18×6 table
Index exitflag f(x) # iter F-count
_____ ________ _________ ___ ____ _________
1 -10 14 NaN NaN NaN
2 3 1.239e-09 39 40 3.484e-06
3 3 3.028e-22 45 46 2.174e-12
4 -10 4 NaN NaN NaN
5 -10 7 NaN NaN NaN
6 3 5.749e-13 17 18 5.438e-08
7 -10 3 NaN NaN NaN
8 -10 2 NaN NaN NaN
9 -10 5 NaN NaN NaN
10 -10 1 NaN NaN NaN
11 -10 2 NaN NaN NaN
12 -10 3 NaN NaN NaN
13 -10 5 NaN NaN NaN
14 -10 1 NaN NaN NaN
15 -10 2 NaN NaN NaN
16 -10 2 NaN NaN NaN
NaN NaN NaN NaN NaN NaN
3 NaN NaN 16 NaN NaN
>>
will then need to clean up the last couple records after the Index sequence stops...
Alternatively, if know the length of the file a priori,
>> tT=readtable('command_window.txt','Range','3:19','ExpectedNumVariables',6,'VariableNamingRule','preserve')
tT =
16×6 table
Index exitflag f(x) # iter F-count
_____ ________ _________ ___ ____ _________
1 -10 14 NaN NaN NaN
2 3 1.239e-09 39 40 3.484e-06
3 3 3.028e-22 45 46 2.174e-12
4 -10 4 NaN NaN NaN
5 -10 7 NaN NaN NaN
6 3 5.749e-13 17 18 5.438e-08
7 -10 3 NaN NaN NaN
8 -10 2 NaN NaN NaN
9 -10 5 NaN NaN NaN
10 -10 1 NaN NaN NaN
11 -10 2 NaN NaN NaN
12 -10 3 NaN NaN NaN
13 -10 5 NaN NaN NaN
14 -10 1 NaN NaN NaN
15 -10 2 NaN NaN NaN
16 -10 2 NaN NaN NaN
>>
truncates on the way in....but that means knowing the length first which may not be generally so.
Avoid importdata -- it's intended to be easy to use, but that comes at a price that while it does something, you don't necessarily know what it's going to do with any given file.
Alternatively to the above, there's always the recourse to revert to parsing generic text files on a line-by-line basis to handle the somewhat unconforming formatting found in machine-generated output files regarding variable names, treatment of missing values, deliminters, etc., etc., etc., .... that a generic routine just can't handle every possible permutation it might find without some klews...
See Also
Categories
Find more on Large Files and Big Data in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!