is there a better way of accessing the csv files from each folders in the path below?

Hi,

I am using the code below to plot the data but the problem is code is not efficient enough to save me more time. I have 4 paths with csv in them and each csv will go from timestamp 000000.csv to 000100.csv in each folder and I need to plot the data from each csv from each folderwith the same time stamp. I can goahead and do the same way in the code below but is there any better way of analysing the data then what I have below.

files = { 'F:\3-PIV_Experimental_Data\Outlet_110\Data_LaserSheet_D\Data_PIV\Data_Vector\Run 13-56-31.Adaptive PIV.6tvk0mbp\Export.6tvk0mbp.000000.csv'; ...

'F:\3-PIV_Experimental_Data\Outlet_110\Data_LaserSheet_B\Data_PIV\Data_Vector\Run 17-19-03.Adaptive PIV.6ul4ynyv\Export.6ul4ynyv.000000.csv'; ...

'F:\3-PIV_Experimental_Data\Outlet_110\Data_LaserSheet_B\Data_PIV\Data_Vector\Run 17-19-03.Adaptive PIV.6ul4ynyv\Export.6ul4ynyv.000000.csv'; ...

'F:\3-PIV_Experimental_Data\Outlet_110\Data_LaserSheet_A\Data_PIV\Data_Vector\Run 12-45-51.Adaptive PIV.6tskdx6a\Export.6tskdx6a.000000.csv'};

for i = 1:numel(files)

a = readtable(files{i});

X = a{:,5}/1000;

Y = a{:,6}/1000;

U = a{:,9};

V = a{:,10};

quiver(X, Y, U, V, 10);

axis tight

hold on

end

Answer 1

[Walter Roberson]

 prefixes = { 'F:\3-PIV_Experimental_Data\Outlet_110\Data_LaserSheet_D\Data_PIV\Data_Vector\Run 13-56-31.Adaptive PIV.6tvk0mbp\Export.6tvk0mbp.'; ...
          'F:\3-PIV_Experimental_Data\Outlet_110\Data_LaserSheet_B\Data_PIV\Data_Vector\Run 17-19-03.Adaptive PIV.6ul4ynyv\Export.6ul4ynyv.'; ...
          'F:\3-PIV_Experimental_Data\Outlet_110\Data_LaserSheet_B\Data_PIV\Data_Vector\Run 17-19-03.Adaptive PIV.6ul4ynyv\Export.6ul4ynyv.'; ...
          'F:\3-PIV_Experimental_Data\Outlet_110\Data_LaserSheet_A\Data_PIV\Data_Vector\Run 12-45-51.Adaptive PIV.6tskdx6a\Export.6tskdx6a.'};
 ndir = length(prefixes);
 filenames = cell(ndir, 1);
 for K = 1:ndir
    dinfo = dir( [prefixes{K} '*.csv']);
    filenames{K} = fullfile({dinfo.folder}, {dinfo.name});
 end
 
 numfiles = cellfun(@length, filenames);
 assert( ~any(diff(numfiles), 'folders not all same size');
 numfiles = numfiles(1);
 
 for K = 1 : numfiles
    filename_subset = cellfun(@(List) List(K), filenames);
    for d = 1 : ndir
      thisfile = filename_subset{d};
      a = readtable(thisfile);
      some stuff here
    end
 end

The for K loop at the bottom is executed once for each timestamp. Inside that for d loops through pulling out corresponding files from each of the directories. You do whatever is appropriate for each of them "to plot the data from each csv from each folderwith the same time stamp."

[muhammad choudhry]

i think we miss to ) in the line below. I keep putting on different place but still getting error.

assert( ~any(diff(numfiles), 'folders not all same size'); % where to put the missing )

[muhammad choudhry]

I am getting further errors if i comment out the line above.

Errors:

Index exceeds the number of array elements (79).

Error in read_csv>@(List)List(K) (line 67)

filename_subset = cellfun(@(List) List(K), filenames);

Error in read_csv (line 67)

filename_subset = cellfun(@(List) List(K), filenames);

Code I used:

prefixes = { 'F:\3-PIV_Experimental_Data\Outlet_110\Data_LaserSheet_D\Data_PIV\Data_Vector\Run 13-56-31.Adaptive PIV.6tvk0mbp\Export.6tvk0mbp.'; ...
    'F:\3-PIV_Experimental_Data\Outlet_110\Data_LaserSheet_B\Data_PIV\Data_Vector\Run 17-19-03.Adaptive PIV.6ul4ynyv\Export.6ul4ynyv.'; ...
    'F:\3-PIV_Experimental_Data\Outlet_110\Data_LaserSheet_B\Data_PIV\Data_Vector\Run 17-19-03.Adaptive PIV.6ul4ynyv\Export.6ul4ynyv.'; ...
    'F:\3-PIV_Experimental_Data\Outlet_110\Data_LaserSheet_A\Data_PIV\Data_Vector\Run 12-45-51.Adaptive PIV.6tskdx6a\Export.6tskdx6a.'};
ndir = length(prefixes);
filenames = cell(ndir, 1);
for K = 1:ndir
    dinfo = dir( [prefixes{K} '*.csv']);
    filenames{K} = fullfile({dinfo.folder},{dinfo.name});
end

numfiles = cellfun(@length, filenames);
%assert( ~any(diff(numfiles), 'folders not all same size');
numfiles = numfiles(1);

for K = 1 : numfiles
    filename_subset = cellfun(@(List) List(K), filenames);
    for d = 1 : ndir
        thisfile = filename_subset{d};
        a = readtable(thisfile);
        X = a{:,5}/1000;
        Y = a{:,6}/1000;
        U = a{:,9};
        V = a{:,10};
        quiver(X, Y, U, V, 10);
        axis tight 
        hold on
    end
end

Is it possible to plot the graph for each time stamp at a moment I am getting all the data on the same graph and then matlab crashing.

[Walter Roberson]

assert( length(unique(numfiles)) == 1, 'folders not all same size'); 

[Walter Roberson]

Is it possible to plot the graph for each time stamp at a moment I am getting all the data on the same graph and then matlab crashing.

The answer would seem to be NO: you are plotting all the data, and you are getting a crash, which suggests that you are running out of memory trying to do the plots.

You could consider creating a new figure for each time stamp, but that would get you 101 different figures, if I read your Question correctly, and 101 different figures is going to be hard on your system.

Perhaps you could save each one to a file as an image, using exportgraphics() ? Or perhaps you could use VideoWriter to create an animation ?

You have 404 different quiverplots... not clear what you expect the result to look like.

It is not even clear to me what you expect the result to look like to overlay 4 of them at one time.

[muhammad choudhry]

So basically, I performed experiment on a certain device with the technique called 2D PIV and device has certain diameter and whole diameter covered in 4 different experiments, as a result I am getting data transfered in 4 different set with time stamps. I want to put the 4 cases of data with same time stamp on one figure which will cover the whole region of device in 2D then put the line across the figure and plot data over line which will give me the data across the device over the length of line hence I will further then compare the results with the numerical simulations I have. I am attaching you 4 csv files with the data from one time stamp to make more sense. If I get a technique or code to perform on one timestamps then I further do my analysis, at a moment I am stuck. I am plotting x,y (mm) and u,v(m/s) from the data.

Answer 2

[Image Analyst]

How about getting all the filenames and looping over them.

topLevelFolder = 'F:\3-PIV_Experimental_Data\Outlet_110\';
filePattern = fullfile(topLevelFolder, '**/Export*.csv'); % The ** will make it recurse into subfolders.
fileList = dir(filePattern);
for k = 1 : length(fileList)
    thisFolder = fileList(k).folder;
    thisBaseFileName = fileList(k).name;
    fullFileName = fullfile(thisFolder, thisBaseFileName);
    data = readtable(fullFileName);
end

You might have to do some further parsing if you want to process subsets of the entire list in some particular order.