Thread Subject: Storing Simulation Results

Hello all

So probably like many of you I run alot of simulations. I
vary a few parameters run them again. Quickly I end up
having dozens of parameters each which can take dozens of
different values.

So here's my question

What is the best way to keep track of all of these
results. So far I have experimented with

1) Just storing the resuts as Mat files in hand made
directories with descriptive names.
pros: simple
cons: it's easy to forget what has already been simulated
      it time intensive to maintain and retrieve

2) Storing all results in a big file, along with the
associated parameters
pros: simple automatic checks to see if it's been run
before are possible.
      retrieval of data can be automated easily.
cons: The file gets big fast,
      when multiple version of the simulator (i.e. updats
as time goes on) exist it's hard to keep track of which
results goes with which version

Are there other better ways to do this?

Is there a way to access the version number (from CVS in
my case) from matlab?

Thoughs or ideas would be appreciated.

Aaron Callard wrote:

> So probably like many of you I run alot of simulations. I
> vary a few parameters run them again. Quickly I end up
> having dozens of parameters each which can take dozens of
> different values.

> So here's my question

> What is the best way to keep track of all of these
> results.

Encode the parameter values into the output file names.

Or keep a list of which file name has been used to store the
results for which configuration. You could probably easily store
the parameters for any one graph together with the file name
in a .mat file and then create a simple function or gui that read
out the .mat file and presented an annotated list of files. The classical
name for this technique is "Database"; you could go as far as SQL
or MS Access if you wanted, but that would likely be overkill.

--
Q = quotation(rand);
if isempty(Q); error('Quotation server filesystem problems')
else sprintf('%s',Q), end

"Aaron Callard" <no.thanks@hotmail.com> wrote in message
<g8mga2$8mn$1@fred.mathworks.com>...
> Hello all
>
> So probably like many of you I run alot of simulations. I
> vary a few parameters run them again. Quickly I end up
> having dozens of parameters each which can take dozens of
> different values.
>
> So here's my question
>
> What is the best way to keep track of all of these
> results. So far I have experimented with
>
> 1) Just storing the resuts as Mat files in hand made
> directories with descriptive names.
> pros: simple
> cons: it's easy to forget what has already been simulated
> it time intensive to maintain and retrieve
>
> 2) Storing all results in a big file, along with the
> associated parameters
> pros: simple automatic checks to see if it's been run
> before are possible.
> retrieval of data can be automated easily.
> cons: The file gets big fast,
> when multiple version of the simulator (i.e. updats
> as time goes on) exist it's hard to keep track of which
> results goes with which version
>
> Are there other better ways to do this?
>
> Is there a way to access the version number (from CVS in
> my case) from matlab?
>
> Thoughs or ideas would be appreciated.

I tend to keep all my different runs as separate projects. I use a project base
directory that is descriptive of the simulations I am running and separate
subdirectories, with descriptive names (including parameter values, for
instance), for each run of the simulation. The data and the plot names are all
generically named (ie 'data.mat') files in the simulation subdirectories.

In addition, I keep track of the directory names, data file names, and anything
else pertinant in a class called, surprisingly, 'Project'. A Project object need
not load the data, it only needs the name of the data file, so the size is kept
minimal. This simplifies things greatly, since I need only call, for example,
project.plot_graph( ) to generate a plot. The only thing I need to remember is
what I named the project.

Hope this helps. For me, automation of all project management tasks was the
key to organizing my data.

Charles