Discover MakerZone

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn more

Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply Today

Thread Subject:
Segmented Optimization

Subject: Segmented Optimization

From: Cory

Date: 30 Jul, 2012 02:33:23

Message: 1 of 5

Hi MATLABers,

I am performing a least-squares, non-linear constrained optimization problem with the following structure:

Called the summands (whose sum of squares I want to minimize) DQ_i. One parameter, theta, affects all the DQ_i. For a given theta, the constraints uniquely determine the rest of the parameters. Further, the constraints are segmented so I could solve for different chunks of the parameters separately (given theta). However, any solution would be numeric.

So I could structure the problem in two ways:

1) Frame it as a nested optimization problem. Pick a theta, solve the smaller problems in chunks, calculate the objective function, and repeat.

2) Put all the parameters together and solve it as one big, constrained problem.

Which approach is more advisable? 2) seems cleaner from a MATLAB coding perspective since if the constraint-solving fails even once, it will derail the whole problem. But 1) seems potentially faster as I never have too many variables to solve for at once.

Thoughts?

Cory

Subject: Segmented Optimization

From: Matt J

Date: 30 Jul, 2012 11:10:18

Message: 2 of 5

"Cory" wrote in message <jv4rpj$adm$1@newscl01ah.mathworks.com>...
> Hi MATLABers,
>
> I am performing a least-squares, non-linear constrained optimization problem with the following structure:
>
> Called the summands (whose sum of squares I want to minimize) DQ_i. One parameter, theta, affects all the DQ_i. For a given theta, the constraints uniquely determine the rest of the parameters. Further, the constraints are segmented so I could solve for different chunks of the parameters separately (given theta). However, any solution would be numeric.
>
> So I could structure the problem in two ways:
>
> 1) Frame it as a nested optimization problem. Pick a theta, solve the smaller problems in chunks, calculate the objective function, and repeat.
>
> 2) Put all the parameters together and solve it as one big, constrained problem.
>
> Which approach is more advisable? 2) seems cleaner from a MATLAB coding perspective since if the constraint-solving fails even once, it will derail the whole problem. But 1) seems potentially faster as I never have too many variables to solve for at once.
================


(1) seems the best approach to me. A one-dimensional problem should converge very fast. I don't really understand the motivation you've given for (2). Why would the constraint-solving fail if, as you say, "For a given theta, the constraints uniquely determine the rest of the parameters"? And if it can fail, why would (2) be any more robust to it?

Subject: Segmented Optimization

From: Cory

Date: 30 Jul, 2012 13:48:18

Message: 3 of 5

> (1) seems the best approach to me. A one-dimensional problem should converge very fast. I don't really understand the motivation you've given for (2). Why would the constraint-solving fail if, as you say, "For a given theta, the constraints uniquely determine the rest of the parameters"? And if it can fail, why would (2) be any more robust to it?

The only way to solve for the parameters (given theta) is with a numerical algorithm. So (1) has the undesirable quality of using a numerical solver to solve a problem involving a numerical solver (i.e. 2 levels of numerical solving).

This means that the lower level of the numerical solving has to always succeed. If it succeeds 99% of the time but the code wants to check 100 values of theta, I could be stuck.

(1) also seemed the best to me despite this, but I wanted a second opinion.

Subject: Segmented Optimization

From: Matt J

Date: 30 Jul, 2012 17:28:36

Message: 4 of 5

"Cory" wrote in message <jv63b2$qcp$1@newscl01ah.mathworks.com>...
>

> This means that the lower level of the numerical solving has to always succeed. If it succeeds 99% of the time but the code wants to check 100 values of theta, I could be stuck.
>
> (1) also seemed the best to me despite this, but I wanted a second opinion.

No, that wasn't my opinion. If theta doesn't always determine the other parameters, then (1) won't work. If it does always determine them then the lower level solver should work 100% of the time.

Subject: Segmented Optimization

From: Cory

Date: 30 Jul, 2012 19:35:43

Message: 5 of 5

> No, that wasn't my opinion. If theta doesn't always determine the other parameters, then (1) won't work. If it does always determine them then the lower level solver should work 100% of the time.

I think you've misunderstood a little bit. I've actually run across any number of instances where there is a unique solution to a problem but a numerical solver fails. For instance, it can get stuck at a local optimum. So it will never be 100%, though I suspect for my application it will be close.

Anyway, theta uniquely determines the other parameters (numerically), so I'm going to keep working with approach (1) for now. Thanks for the advice!

Tags for this Thread

No tags are associated with this thread.

What are tags?

A tag is like a keyword or category label associated with each thread. Tags make it easier for you to find threads of interest.

Anyone can tag a thread. Tags are public and visible to everyone.

Contact us