how to separate a large text file into individual text files based on date, but also export headers?
Show older comments
Hi There,
I have a large text file with some data with the following headings, these run from 1991 to 2019, and i want to extract and generate individual text files , but keep the headings for later analysis. I also want the file name to be saved in a certain format based on the date.
I'm still pretty new to Matlab so any suggestions/ guidence would be great !
Easting Northing Elevation C hainage FC Profile_reg_ID Survey date
258574.81 309931.38 5.033 -65.24 ZZ 20 02/01/1992
Many thanks,
Alex
6 Comments
Mathieu NOE
on 27 Jan 2023
hello
can you share a sample text file ?
tx
Alex
on 27 Jan 2023
Mathieu NOE
on 27 Jan 2023
hello
in the text file provided , the dates appear with a strange format , not what you originaly posted above
did you make some export from excel (xlsx) to txt file ?
Easting Northing Elevation Chainage FC Profile_reg_ID Survey date
259354.73 319311.53 3.841 -58.24 ZZ 31 33588
Alex
on 27 Jan 2023
Alex
on 27 Jan 2023
dpb
on 27 Jan 2023
", I need to seperate them by dates, so each individual date is stored as a seperate text file..."
That's easily-enough done, but I'd ask "Why?" create a zillion different files to have to process with all that extra overhead and code to deal with instead of just processing the data by whatever combination of variables needed? findgroups and groupsummary or rowfun are extremely powerful for such tasks...
Accepted Answer
Star Strider
on 27 Jan 2023
Edited: Star Strider
on 27 Jan 2023
Try something like this —
files = {'https://www.mathworks.com/matlabcentral/answers/uploaded_files/1276100/Gwynedd_1991.txt';'https://www.mathworks.com/matlabcentral/answers/uploaded_files/1276165/Gwynedd_1992.txt'}
for k1 = 1:numel(files)
% fprintf('k1 = %d',k1)
T{k1} = readtable(files{k1}, 'VariableNamingRule','preserve');
if isnumeric(T{k1}.('Survey date'))
Date = datetime(T{k1}.('Survey date'), 'ConvertFrom','excel');
else
Date = T{k1}.('Survey date');
end
[G,Y,M,D] = findgroups(year(Date),month(Date),day(Date));
Gu = unique(G);
for k2 = 1:numel(Gu)
% fprintf('k2 = %d',k2)
SurvID = T{k1}.Profile_reg_ID(G==k2,:);
Dk = Date(G==k2);
[y,m,d] = ymd(Dk(1));
fn{k1,k2} = sprintf('%02d_%4d%02d%02d.txt',SurvID(1,1),y,m,d);
writetable(T{k1}(G==k2,:),fn{k1,k2})
fprintf('Written: %s\n',fn{k1,k2})
end
end
TR1 = readtable(fn{1,1}, 'VariableNamingRule','preserve')
TR6 = readtable(fn{2,6}, 'VariableNamingRule','preserve')
EDIT — (27 Jan 2023 at 16:05)
Adapted original code to accommodate both original files.
.
2 Comments
Alex
on 27 Jan 2023
Star Strider
on 27 Jan 2023
As always, my pleasure!
I edited my original Answer to include and process both the original and added files.
.
More Answers (1)
Mathieu NOE
on 27 Jan 2023
So here we go
A code that is certainly not as refined as StarStrider's ....
filename = 'Gwynedd_1992.txt';
[outdata] = readcell(filename, 'DateTimeType', 'text');
[m,n] = size(outdata);
% extract header line and data
header_line = outdata(1,:);
data = outdata(2:m,:);
dates = string(data(:,n));
datesarray = datetime(dates);
% find unique dates
[dates_unic,ia,ic] = unique(datesarray);
% split and save individual data blocks (one file per date)
for ck = 1:numel(dates_unic)
start = ia(ck);
if ck == numel(dates_unic)
stop = m-1;
else
stop = ia(ck+1)-1;
end
thisdate = strrep(dates(start),'/','-');
data_out = [header_line; data(start:stop,:)];
% write to txt file
filename = strcat("surveyID_",thisdate,".txt");
writecell(data_out,filename,"Delimiter","tab");
end
2 Comments
Alex
on 27 Jan 2023
Mathieu NOE
on 27 Jan 2023
My pleasure !
(difficult to win against Star Strider !! :))
Categories
Find more on Time Series Objects in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!
