MATLAB Answers

Mat1
0

How can I load multiple dat files of different name pattern and from different directory consecutively to do something

Asked by Mat1
on 16 May 2018 at 18:04
Latest activity Commented on by Ameer Hamza
about 1 hour ago

Hi, I have some dat files of row*Column( 9999*10 )of names:

dir1/a_0.1_1
dir1/b_0.1_2
dir1/c_0.3_1
dir1/c_1.5_1
dir2/a_0.1_1
dir2/a_0.5_1

Now How can I load and recall then sequentially to do something? is there any short cut for loop for this specially when they are in different folder? Thank you

  0 Comments

Sign in to comment.

2 Answers

Answer by Ameer Hamza
on 16 May 2018 at 18:38
 Accepted Answer

You can use this FEX submission. Download it and place it in MATLAB path. To get the list of all .dat files, use it as follow

files = subdir('*.mat');

this will work, if you are present in the top folder of dir1, dir2 etc. This will give you name and path of all the '.dat' files in the subfolder. Then read and process like this

data = cell(1, length(files))
for i=1:length(files)
  filename = files(i).name;
  % load your file here, use load(), tableread() or any appropriate function depending on the type of data in .dat files
  data{i} = readData; % save your read data or do any other processing
end

  7 Comments

Hello Mr. Hamza, it Works !!!!!

files = dir('B:/et/*.dat');

thank you so much. Now it takes just several minutes by your code, what takes several hour for me. God bless you for help me learn.

You are welcome. Yes, I noticed that your code requires huge memory several 10s of GBs. I would have taken a long time. You can further decrease time by making interpolation resolution of 10^8 to smaller values, but this depends on your requirement

In case you haven't noticed, the 2 lines at end should be

ave_time_extr = ave_time/length(files);
ave_radius_extr = ave_radius/length(files);

I forget to change those lines. Therefore right now you are just adding all the columns of X and Y. To get mean value, you need to divide them with length(files).

Sign in to comment.


Answer by Mat1
about 2 hours ago

Yes Mr. Hamza you are exactly right. Now on the first day you advised to download this FEX submission if in case .dat files are in different folder. I have downloaded FEX but do not know how to put that code on the MATLAB path. I have to spent lot of time just to bring the files in the same folder then execute your code. Your code is excellent for this job, but the .dat files that I have being generated in a computer cluster where they all came with the same name 'event_time'. So when I try to put them in one folder to plot in MATLAB I have to change their name every time or they overwrite. So It would be quite robust if I could load the .dat files from different sub-folders keeping the code in the main folder. Thank you very much.

  1 Comment

You need to download the files from FEX and place it in the current folder of MATLAB. The current folder is the folder which is displayed in Current Folder window in MATLAB.

Now, suppose that you have a folder structure like this

B:  <- B drive for you computer
  other folders in the path
    folderTop <--- this folder contains all the .dat files
      folder1
        file1.dat
        file2.dat
        ...
      folder2
        file1.dat
        file2.dat
        ...
      folder3
        file1.dat
        file2.dat
        ...

You can use subdir() as follow

files = subdir('B:/other folders/folderTop/*.dat');

It will give you the similar struct given by dir but containing files in all the subfolders. You can use the remaining code same as before.

Sign in to comment.


Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply Today