Discover MakerZone

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn more

Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply Today

Thread Subject:
Use save append option in a for loop

Subject: Use save append option in a for loop

From: StarkEffekt Heiles

Date: 18 May, 2010 18:46:19

Message: 1 of 7

Hi MatlabCentral Members!

For one of my research projects I have to deal with big matrices which are generated in a for loop. Since my memory is limited I have to clear my workspace before the next cycle of the loop can start. I asked myself if it is possible to save and append files with the same file name created in the loop? I tried save with the option append but the new data replaces the old file with the same name.

Thanks in advance

StarkEffekt

Subject: Use save append option in a for loop

From: Walter Roberson

Date: 18 May, 2010 19:10:37

Message: 2 of 7

StarkEffekt Heiles wrote:

> For one of my research projects I have to deal with big matrices which
> are generated in a for loop. Since my memory is limited I have to clear
> my workspace before the next cycle of the loop can start. I asked myself
> if it is possible to save and append files with the same file name
> created in the loop? I tried save with the option append but the new
> data replaces the old file with the same name.

The only case that I _know_ of where save -append gives unexpected results is
if you are using binary files (the default) and the new variable you save has
the same name as an existing variable in the file. If you do have this
situation, then the entire .mat file will not be overwritten, but the new
contents of the variable will replace the old contents of the variable.

Subject: Use save append option in a for loop

From: Steven Lord

Date: 18 May, 2010 20:11:26

Message: 3 of 7


"Walter Roberson" <roberson@hushmail.com> wrote in message
news:hsuoof$7c0$2@canopus.cc.umanitoba.ca...
> StarkEffekt Heiles wrote:
>
>> For one of my research projects I have to deal with big matrices which
>> are generated in a for loop. Since my memory is limited I have to clear
>> my workspace before the next cycle of the loop can start. I asked myself
>> if it is possible to save and append files with the same file name
>> created in the loop? I tried save with the option append but the new data
>> replaces the old file with the same name.
>
> The only case that I _know_ of where save -append gives unexpected results
> is if you are using binary files (the default) and the new variable you
> save has the same name as an existing variable in the file. If you do have
> this situation, then the entire .mat file will not be overwritten, but the
> new contents of the variable will replace the old contents of the
> variable.

Walter, the behavior you describe as "unexpected" is actually the documented
behavior. Take a look in the description of "-append" in the Input
Arguments section of the documentation for SAVE:

http://www.mathworks.com/access/helpdesk/help/techdoc/ref/save.html?BB=1

In this case, '-append' means "add new variables to the file" not "add new
content to the variables already in the file". If either of you (or anyone
else) are interested in the latter meaning, I think that would make a
perfectly reasonable enhancement request to file with Technical Support
(along with your use case for how you'd use such a feature.) It would
likely be tricky to determine what it should do when used with the -ascii
flag, so if that scenario is important to you I suggest describing your
expectations in your request.

--
Steve Lord
slord@mathworks.com
comp.soft-sys.matlab (CSSM) FAQ: http://matlabwiki.mathworks.com/MATLAB_FAQ
To contact Technical Support use the Contact Us link on
http://www.mathworks.com

Subject: Use save append option in a for loop

From: Hong Li

Date: 23 Feb, 2011 03:34:06

Message: 4 of 7

Walter Roberson <roberson@hushmail.com> wrote in message <hsuoof$7c0$2@canopus.cc.umanitoba.ca>...
> StarkEffekt Heiles wrote:
>
> > For one of my research projects I have to deal with big matrices which
> > are generated in a for loop. Since my memory is limited I have to clear
> > my workspace before the next cycle of the loop can start. I asked myself
> > if it is possible to save and append files with the same file name
> > created in the loop? I tried save with the option append but the new
> > data replaces the old file with the same name.
>
> The only case that I _know_ of where save -append gives unexpected results is
> if you are using binary files (the default) and the new variable you save has
> the same name as an existing variable in the file. If you do have this
> situation, then the entire .mat file will not be overwritten, but the new
> contents of the variable will replace the old contents of the variable.
I have the same problem. Have you worked out any ideas?
Thank u.

Subject: Use save append option in a for loop

From: Hans Martin

Date: 25 Feb, 2011 21:37:04

Message: 5 of 7

"StarkEffekt Heiles" <Hausmacherworscht@gmx.de> wrote in message <hsun9r$erf$1@fred.mathworks.com>...
> Hi MatlabCentral Members!
>
> For one of my research projects I have to deal with big matrices which are generated in a for loop. Since my memory is limited I have to clear my workspace before the next cycle of the loop can start. I asked myself if it is possible to save and append files with the same file name created in the loop? I tried save with the option append but the new data replaces the old file with the same name.
>
> Thanks in advance
>
> StarkEffekt

Can you be a bit more specific of how you want to append? There cannot be two .mat files or two doubles in a .mat with the same name, so overwriting will happen if you do not change something.

To save new data in a loop you have, as I can see, two options:

1: expand the array of the file in each interation with new data ( x = [x;newdata] )
2: create and save a new file with a new and unique name in each iteration

This can all be done in a loop by calling and naming the variables by feeding the 'eval' function with strings that represent you new/old variables/files.

Remember it was a big revelation for me when I first discovered 'eval' for this usage

Subject: Use save append option in a for loop

From: Olivier

Date: 11 Oct, 2012 22:49:08

Message: 6 of 7

If I have a sequence of yearly time series data sets (data1998.mat data1999.mat ... data2009.mat), which contain the same variable names, and load them sequentially, like this:

files = dir('C:\Timeseries\data*.mat);
for i = 1:length(files)
    eval(['load ' files(i).name, ' time counts']);
end

... how can I concatenate/append the variables 'counts' from each file sequentially (in the for loop above, for instance) in the 'time' dimension, given that certain yearly data sets do not necessarily begin on Jan-1-YYYY, end on Dec-31-YYYY, or are missing altogether?

Subject: Use save append option in a for loop

From: TideMan

Date: 12 Oct, 2012 00:26:01

Message: 7 of 7

On Friday, October 12, 2012 11:49:08 AM UTC+13, Olivier wrote:
> If I have a sequence of yearly time series data sets (data1998.mat data1999.mat ... data2009.mat), which contain the same variable names, and load them sequentially, like this:
>
>
>
> files = dir('C:\Timeseries\data*.mat);
>
> for i = 1:length(files)
>
> eval(['load ' files(i).name, ' time counts']);
>
> end
>
>
>
> ... how can I concatenate/append the variables 'counts' from each file sequentially (in the for loop above, for instance) in the 'time' dimension, given that certain yearly data sets do not necessarily begin on Jan-1-YYYY, end on Dec-31-YYYY, or are missing altogether?

What on earth would possess you to use eval for this?
It's madness.

You already know about structure arrays because you use them in dir, so why don't you use them again:
for ifile=1:length(files)
     s(ifile)=load(files(ifile).name);
end

Tags for this Thread

What are tags?

A tag is like a keyword or category label associated with each thread. Tags make it easier for you to find threads of interest.

Anyone can tag a thread. Tags are public and visible to everyone.

Contact us