FAQ: How can I process a sequence of files?

How can I process a sequence of files?

[Jan]

This is a tutorial question only! There is no need to answer it and please to not vote for it. This is neither my question nor my answer, but only an example for a nicer, more convenient, more usable FAQ, which is less stuffed with commercials.

See http://www.mathworks.com/matlabcentral/answers/57443-professional-looking-matlab-faq.

[Salah Djerouni]

Hi Jan

can i ask you some question because i have a problem for save result in matlab

my probleme is i run matlab with three loops (for...end ) and i want to save the result every change iterataion loop.

if you possible to help me .

thanks Jan

[Walter Roberson]

Pre-allocate an output array that is at least 3 dimensions, with one dimension being the number of times the first loop will execute, a second dimension being the number of times the second loop will execute, and the third dimension being the number of times the third loop will execute. Then assign to the output according to how many loop iterations you have done.

For example,

Lvals = [-83, 149, -5, 0, -2, 101];
numL = length(Lvals);
output = zeros(7, 3, numL);

for J = 9:17:11
    Jidx = round((J-9)/17) + 1;
    for K = 1000:500:2000
        Kidx = round((K-500)/500);
        for Lidx = 1 : numL
            L = Lvals(Lidx);
            output(Jidx, Kidx, L) = J.^2 + K/7 + sin(L.^3);
        end
    end
end

Answer 1

[Jan]

If the files that you want to process are sequentially numbered, like "file1.txt", "file2.txt", "file3.txt", etc. then you can use SPRINTF or NUM2STR to create the filename and LOAD, IMREAD, FOPEN, etc. to retrieve the data from the file. (Also note the three different ways of building the filename - you can use your favorite way.)

% Read files file1.txt through file20.txt, mat1.mat through mat20.mat
% and image1.jpg through image20.jpg.  Files are in the current directory.
folder = cd;
for k = 1:20
  matFilename  = sprintf('mat%d.mat', k);
  matData      = load(fullfile(cd, matFilename));
  jpgFilename  = sprintf('image%d.jpg', k);
  imageData    = imread(jpgFilename);
  textFilename = sprintf('file%d.txt', k);
  fid          = fopen(fullfile(folder, textFilename), 'rt');
  textData     = fread(fid);
  fclose(fid);
end

In the above code, matData, imageData, and textData will get overwritten each time. You should save them to an array or cell array if you need to use them outside the loop, otherwise use them immediately inside the loop.

If instead you want to process all the files whose name matches a pattern in a directory, you can use DIR to select the files. Note that while this example uses *.jpg as the pattern and IMREAD to read in the data, as with the previous example you could use whatever pattern and file reading function suits your application's needs:

Folder = 'C:\Documents and Settings\yourUserName\My Documents\My Pictures';
if exist(Folder, 'dir') ~= 7  % isfolder(Folder) in modern Matlab versions
  Message = sprintf('Error: The following folder does not exist:\n%s', Folder);
  uiwait(warndlg(Message));
  return;
end
filePattern = fullfile(Folder, '*.jpg');
jpegFiles   = dir(filePattern);
for k = 1:length(jpegFiles)
  baseFileName = jpegFiles(k).name;
  fullFileName = fullfile(Folder, baseFileName);
  fprintf('Now reading %s\n', fullFileName);
  imageArray = imread(fullFileName);
  imshow(imageArray);   % Display image.
  drawnow;              % Force display to update immediately.
end

[EDITED] With modern Matlab versions (I assume since at least R2016b):

You can use '**' for a recursive search, if the files are in multiple subfolders:

Folder   = 'C:\Your\Folder';
FileList = dir(fullfile(Folder, '**', '*.jpg'));
for iFile = 1:numel(FileList)
    thisFolder = FileList(iFile).folder;
    thisFile   = FileList(iFile).name;
    File       = fullfile(thisFolder, thisFile);
   ... Import the File now as you need...
end

Or you can try a "File Exchange Pick of the Week": FileExchange: FileFun.

See also: Matlab docs: Process a sequence of files

This answer is a modified version of:

matlab.wikia.com/wiki/FAQ: How_can_I_process_a_sequence_of_files

[joshua Abam]

Hi

In follow up of your answer is it possible to apply same to iterations from Genetic algorithm, AS I am currently having challenges of saving the iterations from GA to a single file or each iteration to a seperate file

Thanks

Kind regards

Joshua

[Walter Roberson]

Each iteration in a separate file could result in hundreds of thousands of files.

persistent fid
if isempty(fid) || isempty(fopen(fid))
  fid = fopen('output.txt','wt');
end

fprintf(fid, 'appropriate format', appropriate data) ;

[joshua Abam]

Hi Walter

I seem not to have follow your suggestion can you help by adding some more comments to ease my grap or better still use this example to illustrate it and where best can I possible attached that to a GA code.

function f = gaintobj(x)
f = rastriginsfcn([x(1)-6 x(2)-13]);
f = f + rastriginsfcn([x(3)-3*pi x(4)-5*pi]);


lb = [1 1 -30 -30];
ub = [20 20 70 70];
%%
% Set the integer variables and number of variables.
IntCon = [1 2];
nvar = 4;
%%
% Set options to call the custom output function, and to initially have
% little crossover.
options = optimoptions('ga','OutputFcn',@gaoutputfunround,'CrossoverFraction',0.2);
%%
% For reproducibility, set the random number generator.
rng(10)
%%
% Set the objective function and call the solver.
fun = @gaintobj;
[x,fval] = ga(fun,nvar,[],[],[],[],lb,ub,[],IntCon,options) 

Answer 2

[Sean de Wolski]

Examples in the documentation:

http://www.mathworks.com/help/releases/R2013a/matlab/import_export/process-a-sequence-of-files.html

[Jan]

Somebody has accepted my answer. This has not been the intention, but of course the FAQ contain only answers worth to be accepted.

[Image Analyst]

Was not me.

[Sean de Wolski]

Nor me.

Answer 3

[Sean de Wolski]

In MATLAB R2014b and newer, you can use a datastore to read a sequence of files in a load-analyze-discard manner or in one shot with the readall() command.

http://www.mathworks.com/help/matlab/ref/datastore.html

[Ismail Qeshta]

Hi all. Thanks for sharing it. How about if my files are in permutations sequence such as, Data11, Data12, Data 13, etc. Also, what if I would like to plot two different files names within a folder? Regards.