MATLAB Answers

Robert
1

Loop files, create a timetable and add file names to variable names

Asked by Robert
on 8 Sep 2017
Latest activity Commented on by Robert
on 9 Sep 2017
Hi,
Wondering if anyone is familiar with how adding file names to the variable names in a loop when creating a timetable?
I have 800 files, each of them have the same number of columns but a different number of rows. Rows are time steps. Columns are variables.
When trying to run the below script I get an error ‘Duplicate variable name’, this happens when building up the final timetable as the variable names are the same. What I need is to add the file name to each column of the timetable, at each loop, plus the variable name Data1, Data2, Data3…. To get something like this…
Timetable…
Time (column 0) %First column in timetable
FileName1_Data1 (Variable Name column 1)
FileName1_Data2 (Variable Name column 2)
FileName1_Data3 (Variable Name column 3)
FileName2_Data1 (Variable Name column 4)
FileName2_Data2 (Variable Name column 5)
FileName2_Data3 (Variable Name column 4)
FileName3_Data1 (Variable Name column 6)
FileName3_Data2 (Variable Name column 7)
FileName3_Data3 (Variable Name column 8)
.
FileName800_Data1 (Variable Name column ...)
FileName800_Data2 (Variable Name column ...)
FileName800_Data3 (Variable Name column ...)
Here is what I have done… …
filePattern = fullfile(myFolder, '*.csv');
theFiles = dir(filePattern); %800 files
FinalM=[]; %Empty matrix
%Create timetable T for synchronization
t1 = datetime(2002,3,10,0,0,0); t2 = datetime(2017,12,31,0,0,0);
Time_All = (t1:t2)';
column=(1:1:5776)';
T = timetable(Time_All,column);
for k = 1 : length(theFiles)
baseFileName = theFiles(k).name;
fullFileName = fullfile(myFolder, baseFileName);
fprintf(1, 'Now reading %s\n', fullFileName);
%Create FileName which I do not know how to use to include in the Variables %(columns) name in the Final timetable
FileName=strcat(theFiles(k).name(1:end-4));
aa=csvread(fullFileName, 2,0);
%Create time from first 6 columns (Year, month, …)
Time=datetime(aa(:,1),aa(:,2),aa(:,3),aa(:,4),aa(:,5),aa(:,6));
%Create Data1, Data2, and Data3
Data1=aa(:,7);
Data2=aa(:,8);
Data3=aa(:,9);
Tab=timetable(Time,Data1, Data2, Data3);
%Obtain some statistics
Dmean=rmmissing(retime(Tab,'Daily',@nanmean));
Drange=rmmissing(retime(Tab,'Daily',@range));
%Synchronize the data based on the largest time series ‘Time_All’ created above
FT=synchronize(Dmean, Drange,Time_All,'firstvalue');
%Create final timetable
FinalM=[FinalM innerjoin(FT,FT)];
end
Any suggestions or comments are welcome.

  0 Comments

Sign in to comment.

2 Answers

Answer by Walter Roberson
on 9 Sep 2017
 Accepted Answer

After you create FT then
FT.Properties.VariableNames = strcat( FileName, '_', FT.Properties.VariableNames );

  1 Comment

Sign in to comment.


Answer by Jose Marques on 9 Sep 2017

Hello Robert.
If I understand correctly, you may try:
FileName=[theFiles(k).name(1) theFiles(k).name(2) theFiles(k).name(3) theFiles(k).name(4)];

  1 Comment

Sign in to comment.