Thread Subject:

creating loops for a large system of variables

consider the following simple example.

a = 1;
b = 3;

c = a+b;

If I wanted to vary a or b, I could simply do this:

a = [1 2];
b = [3 4];

for i = 1:length(a)
for j = 1:length(b)
c = a(i)+b(j);
end
end

but if I had a large system of variables, say 1000, that would be a lot of for loops, and would waste a lot of time going through each of them. Is there a good way to go about doing this?

"Nate Jensen" wrote in message <ina9ds$qcn$1@fred.mathworks.com>...
> consider the following simple example.
>
> a = 1;
> b = 3;
>
> c = a+b;
>
> If I wanted to vary a or b, I could simply do this:
>
> a = [1 2];
> b = [3 4];
>
> for i = 1:length(a)
> for j = 1:length(b)
> c = a(i)+b(j);
> end
> end
>
> but if I had a large system of variables, say 1000, that would be a lot of for loops, and would waste a lot of time going through each of them. Is there a good way to go about doing this?
- - - - - - - - - -
  That would not only be a lot of wasted time on for-loops, it would be an awful lot of additions, never mind the for-loops! If each of a thousand variables were to independently vary over just two numbers, the number of necessary additions would exceed the famous "googol" of Edward Kasner fame. There aren't that many atoms in the known universe.

Roger Stafford

> That would not only be a lot of wasted time on for-loops, it would be an awful lot of additions, never mind the for-loops! If each of a thousand variables were to independently vary over just two numbers, the number of necessary additions would exceed the famous "googol" of Edward Kasner fame. There aren't that many atoms in the known universe.
>
> Roger Stafford

Well, I worked something out, and I guess I'll find out later if it's worth it, but basically this code writes a new code which wraps for loops around a function depending on the length of the variables. If a variable is of length 1, then no for loop is written, but if the variable is of length 2 or greater, then a for loop is wrapped around the function.

In case anyone is interested, here is the code:

clear all
close all

a = 1;
b = [1 2];
c = 1;

s = whos;

fname = 'myfun';

datenow = datestr(now, 'mm-dd-yy HH-MM-SS');

fid = fopen(['temp ' datenow '.m'], 'w');

iteration = cell(length(s),1);
for ii = 1:length(s)
    if s(ii,1).size(2) == 1
        iteration{ii} = ',';
    else
        str = ['for ', s(ii,1).name, 'loop = 1:length(', s(ii,1).name, ')'];
        fprintf(fid,'%s\n',str);
        iteration{ii} = ['(',s(ii,1).name, 'loop),'];
    end
end

str = [fname, '('];
fprintf(fid,'%s',str);
for ii = 1:length(s)
    if ii == length(s)
        str = [s(ii,1).name iteration{ii}(1:end-1)];
        fprintf(fid,'%s',str);
    else
        str = [s(ii,1).name iteration{ii}];
        fprintf(fid,'%s',str);
    end
end
fprintf(fid,'%s\n',')');

for ii = length(s):-1:1
    if s(ii,1).size(2) == 2
        str = 'end';
        fprintf(fid,'%s\n',str);
    end
end

fclose(fid);

whoops, this line (the 7th line from the end):

if s(ii,1).size(2) == 2

should be:

if s(ii,1).size(2) >= 2

"Nate Jensen" wrote in message <inaj1d$j5b$1@fred.mathworks.com>...
>
>
> Well, I worked something out, and I guess I'll find out later if it's worth it, but basically this code writes a new code
==================

Pretty elaborate. You couldhave just use NDGRID, like below. But Roger is right. This will quickly become computationally unrealistic when you increase the number of variables.

vars={[1,2],[3,4],[5,6]};

[combinations{1:3}]=ndgrid(vars{1:3});
combinations=cellfun(@(c) c(:), combinations,'uni',0);
combinations=[combinations{:}]

result=sum(combinations,2)


combinations =

     1 3 5
     2 3 5
     1 4 5
     2 4 5
     1 3 6
     2 3 6
     1 4 6
     2 4 6


result =

     9
    10
    10
    11
    10
    11
    11
    12

> Pretty elaborate. You couldhave just use NDGRID, like below. But Roger is right. This will quickly become computationally unrealistic when you increase the number of variables.
>
> vars={[1,2],[3,4],[5,6]};
>
> [combinations{1:3}]=ndgrid(vars{1:3});
> combinations=cellfun(@(c) c(:), combinations,'uni',0);
> combinations=[combinations{:}]
>
> result=sum(combinations,2)
>
>
> combinations =
>
> 1 3 5
> 2 3 5
> 1 4 5
> 2 4 5
> 1 3 6
> 2 3 6
> 1 4 6
> 2 4 6
>
>
> result =
>
> 9
> 10
> 10
> 11
> 10
> 11
> 11
> 12

And that is why you guys are here. Maybe one day I'll figure out this program. But until then, I'll keep writing retarded code.

"Nate Jensen" <natejensen88@gmail.com> wrote in message
news:inb3v8$4n6$1@fred.mathworks.com...

*snip*

> And that is why you guys are here. Maybe one day I'll figure out this
> program. But until then, I'll keep writing retarded code.

What problem are you trying to solve using this code? Perhaps there's a way
to solve that underlying problem without brute-forcing a problem that is
likely computationally infeasible to force.

--
Steve Lord
slord@mathworks.com
To contact Technical Support use the Contact Us link on
http://www.mathworks.com

> What problem are you trying to solve using this code? Perhaps there's a way
> to solve that underlying problem without brute-forcing a problem that is
> likely computationally infeasible to force.
>
> --
> Steve Lord
> slord@mathworks.com
> To contact Technical Support use the Contact Us link on
> http://www.mathworks.com

What I am really trying to do is examine the effect various inputs have on an output.

I only used 1000 variables earlier as an exaggeration. The actual number of variables that I have is about 25, and between each iteration, I would most likely only examine the effects of one variable at a time, so computation requirements would not be an issue. I was simply trying to make my code somewhat more robust so that I could examine the effects of any of the variables without writing 25 for loops or if statements.

On 4/3/2011 9:22 PM, Nate Jensen wrote:
>> What problem are you trying to solve using this code? Perhaps there's
>> a way to solve that underlying problem without brute-forcing a problem
>> that is likely computationally infeasible to force.
>>
>> --
>> Steve Lord
>> slord@mathworks.com
>> To contact Technical Support use the Contact Us link on
>> http://www.mathworks.com
>
> What I am really trying to do is examine the effect various inputs have
> on an output.
>
> I only used 1000 variables earlier as an exaggeration. The actual number
> of variables that I have is about 25, and between each iteration, I
> would most likely only examine the effects of one variable at a time, so
> computation requirements would not be an issue. I was simply trying to
> make my code somewhat more robust so that I could examine the effects of
> any of the variables without writing 25 for loops or if statements.

Sounds like a potential for more sophisticated methods for selecting
design points...

--

"dpb" <none@non.net> wrote in message news:inbadn$rfb$1@speranza.aioe.org...
> On 4/3/2011 9:22 PM, Nate Jensen wrote:
>>> What problem are you trying to solve using this code? Perhaps there's
>>> a way to solve that underlying problem without brute-forcing a problem
>>> that is likely computationally infeasible to force.
>>>
>>> --
>>> Steve Lord
>>> slord@mathworks.com
>>> To contact Technical Support use the Contact Us link on
>>> http://www.mathworks.com
>>
>> What I am really trying to do is examine the effect various inputs have
>> on an output.
>>
>> I only used 1000 variables earlier as an exaggeration. The actual number
>> of variables that I have is about 25, and between each iteration, I
>> would most likely only examine the effects of one variable at a time, so
>> computation requirements would not be an issue. I was simply trying to
>> make my code somewhat more robust so that I could examine the effects of
>> any of the variables without writing 25 for loops or if statements.
>
> Sounds like a potential for more sophisticated methods for selecting
> design points...

I agree; the design of experiments or multivariate methods functions in
Statistics Toolbox may help the OP determine how their function depends on
the inputs without evaluating their function N^25 times.

http://www.mathworks.com/help/toolbox/stats/bq_w_hm.html#braa3fz-1

--
Steve Lord
slord@mathworks.com
To contact Technical Support use the Contact Us link on
http://www.mathworks.com

> > Sounds like a potential for more sophisticated methods for selecting
> > design points...
>
> I agree; the design of experiments or multivariate methods functions in
> Statistics Toolbox may help the OP determine how their function depends on
> the inputs without evaluating their function N^25 times.
>
> http://www.mathworks.com/help/toolbox/stats/bq_w_hm.html#braa3fz-1
>
> --
> Steve Lord
> slord@mathworks.com
> To contact Technical Support use the Contact Us link on
> http://www.mathworks.com

Thank you for pointing me in that direction. I am not familiar with these methods, but will continue to research further.

On 4/3/2011 9:54 PM, Nate Jensen wrote:
>> > Sounds like a potential for more sophisticated methods for selecting
>> > design points...
>>
>> I agree; the design of experiments or multivariate methods functions
>> in Statistics Toolbox may help the OP determine how their function
>> depends on the inputs without evaluating their function N^25 times.
>>
>> http://www.mathworks.com/help/toolbox/stats/bq_w_hm.html#braa3fz-1
>>
>> --
>> Steve Lord
>> slord@mathworks.com
>> To contact Technical Support use the Contact Us link on
>> http://www.mathworks.com
>
> Thank you for pointing me in that direction. I am not familiar with
> these methods, but will continue to research further.

Or "Latin-Hypercube Sampling" is another buzzword

More actual detail of what is needed could undoubtedly help cull the
general to at least a little more specific.

--

> Or "Latin-Hypercube Sampling" is another buzzword
>
> More actual detail of what is needed could undoubtedly help cull the
> general to at least a little more specific.
>
> --

What I need to do is understand the behavior of my system as different design variables are changed within it.

Although I could run each case independently, it would be to my advantage to compare them to one another in one run. At the moment, optimization or inverse design of the inputs, would be much more difficult, due to their complex, and at times rigid, nature. The main caveat that I have is that, many of the inputs depend upon one another, and may or may not be changed depending upon the design. This all sums up to the ability to change various inputs depending on the design of the system, and compare their outputs.

I am regretful that I am not making myself clear.

I do appreciate the help you have provided.

On 4/3/2011 11:13 PM, Nate Jensen wrote:
>> Or "Latin-Hypercube Sampling" is another buzzword
>>
>> More actual detail of what is needed could undoubtedly help cull the
>> general to at least a little more specific.
>>
>> --
>
> What I need to do is understand the behavior of my system as different
> design variables are changed within it.
>
> Although I could run each case independently, it would be to my
> advantage to compare them to one another in one run. At the moment,
> optimization or inverse design of the inputs, would be much more
> difficult, due to their complex, and at times rigid, nature. The main
> caveat that I have is that, many of the inputs depend upon one another,
> and may or may not be changed depending upon the design. This all sums
> up to the ability to change various inputs depending on the design of
> the system, and compare their outputs.
>
> I am regretful that I am not making myself clear.

No, that's pretty clear. Read up on response surfaces as one possible
tool--if the models aren't terribly nonlinear. Steven L's pointers are
in similar direction.

DOE is the exercise in setting up criteria based on combination of what
is known of the model and the desired estimators to try to help in an
efficient use of the number of points and values for the independent
variables of those points that meet specific design criteria regarding
which effects may be estimated w/o confounding, which effects are
possibly sacrificed or confounded w/ others to reduce the design size,
etc., etc., ...

If this is academic exercise, if you have access to a statistician it
would be _a_good_thing_ (tm)....

--

On Apr 4, 4:36 am, "Steven_Lord" <sl...@mathworks.com> wrote:

> I agree; the design of experiments or multivariate methods functions in
> Statistics Toolbox may help the OP determine how their function depends on
> the inputs without evaluating their function N^25 times.

No, they wouldn't.

While what is supplied with the stats toolbox might help out a bit
with respect to setting up the tests, there is nothing in that
toolbox that helps with the *analysis*. Once the experiment has
been done and the measurements are available, the user of the
stats toolbox is on his own about how to proceed.

Rune

On 4/3/2011 11:34 PM, dpb wrote:
> On 4/3/2011 11:13 PM, Nate Jensen wrote:
>>> Or "Latin-Hypercube Sampling" is another buzzword
>>>
>>> More actual detail of what is needed could undoubtedly help cull the
>>> general to at least a little more specific.
>>>
>>> --
>>
>> What I need to do is understand the behavior of my system as different
>> design variables are changed within it.
>>
>> Although I could run each case independently, it would be to my
>> advantage to compare them to one another in one run. At the moment,
>> optimization or inverse design of the inputs, would be much more
>> difficult, due to their complex, and at times rigid, nature. The main
>> caveat that I have is that, many of the inputs depend upon one another,
>> and may or may not be changed depending upon the design. This all sums
>> up to the ability to change various inputs depending on the design of
>> the system, and compare their outputs.

>>
>> I am regretful that I am not making myself clear.
>
> No, that's pretty clear. Read up on response surfaces as one possible
> tool--if the models aren't terribly nonlinear. Steven L's pointers are
> in similar direction.
>
> DOE is the exercise in setting up criteria based on combination of what
> is known of the model and the desired estimators to try to help in an
> efficient use of the number of points and values for the independent
> variables of those points that meet specific design criteria regarding
> which effects may be estimated w/o confounding, which effects are
> possibly sacrificed or confounded w/ others to reduce the design size,
> etc., etc., ...
>
> If this is academic exercise, if you have access to a statistician it
> would be _a_good_thing_ (tm)....
>
...

And to make Steve's and my point explicit...if you have 25 variables to
study and you only use two levels for each, that's 2^25 total cases in
the simplistic approach or 33554432 runs of the model. If there's any
nonlinearity at all in the model you need a minimum of three levels to
see it or 3^25 ~ 8.5E11. I don't think you'd be able to interpret the
results even if you could run all those cases.

So, you need a smarter approach.

--