Thread Subject: fmincon runs slow

I am using fmincon to do some topology optimization, so I'm dealing
with a lot of variables.

In case I've been running, each function evaluation takes about 0.5
seconds, but it has to do so many function evaluations that it runs
for 8 to 10 hours and still doesn't converge. (it moves towards a
solution, but doesn't completely get there)

I've been running some smaller problems, trying to find out why it
takes so long and from what I can see, fmincon evaluates the
objective function once for every design variable, calculates the
gradients, takes a step, and then evaluates the objective function
once for every design variable again.

is that really how it should work?

would I benefit from the large-scale option?

Someone might see a problem with my approach that I don't, I will
appreciate any help I get.

Hi Eric,
1) "Lots of variables" - how many?
2) is your function smooth? If not, it is recomended using fminsearch
instead of fmincon. However, fminsearch can't handle any constraints
-lb, ub, linear, nonlin. Do you have any one?
3) Eric Bowman wrote:
> objective function once for every design variable, calculates the
> gradients, takes a step, and then evaluates the objective function
> once for every design variable again

Do you supply gradient or not?
Making this will speedup your calculations very much.
Supplying Hess will yield even more speed, but sometimes it's very
hard to obtain analitically, as for 1st derivatives too.
However, Automatic differentiators exist, you may try MADS from
TOMLAB evaluation ver for example, but i tried & i'm not fond of that
one (my speed greately decriased, however, maybe I did something
wrong).

4) I would recomend to try non-smooth ralg from OpenOpt (free MATLAB
toolbox). However, it was reliesed only some days ago & need many
further improvements. On the other hand, our ralg - fminsearch analog
- can handle analitical (sub)gradients, lb-ub constraints &
(currently via hand-turning) non-lin ineq.
On the other hand, I guess, that if your problem is smooth & you can
provide analitical gradent, fmincon will give you faster
convergence.

Feel free to ask me via e-mail or icq 275976670
WBR, D

Dmitrey wrote:
>
>
> Hi Eric,
> 1) "Lots of variables" - how many?
> 2) is your function smooth? If not, it is recomended using
> fminsearch
> instead of fmincon. However, fminsearch can't handle any
> constraints
> -lb, ub, linear, nonlin. Do you have any one?
> 3) Eric Bowman wrote:
>> objective function once for every design variable, calculates
the
>> gradients, takes a step, and then evaluates the objective
> function
>> once for every design variable again
>
> Do you supply gradient or not?
> Making this will speedup your calculations very much.
> Supplying Hess will yield even more speed, but sometimes it's very
> hard to obtain analitically, as for 1st derivatives too.
> However, Automatic differentiators exist, you may try MADS from
> TOMLAB evaluation ver for example, but i tried & i'm not fond of
> that
> one (my speed greately decriased, however, maybe I did something
> wrong).
>
> 4) I would recomend to try non-smooth ralg from OpenOpt (free
> MATLAB
> toolbox). However, it was reliesed only some days ago & need many
> further improvements. On the other hand, our ralg - fminsearch
> analog
> - can handle analitical (sub)gradients, lb-ub constraints &
> (currently via hand-turning) non-lin ineq.
> On the other hand, I guess, that if your problem is smooth & you
> can
> provide analitical gradent, fmincon will give you faster
> convergence.
>
> Feel free to ask me via e-mail or icq 275976670
> WBR, D

Hello,

You could try SNOPT, NPSOL and KNITRO (3 ALG's internally, see
manual) and CONOPT. They are normally quite a bit faster. Also,
potentially glcCluster/glcDirect (in the Base Module: <http://tomopt.com/tomlab/products/base/solvers/>
 could help to get a better starting point.

Best wishes, Marcus
 <http://tomopt.com/tomlab/>

As I've looked at how my code and emailed Dmitrey Kroshko. (he has
been very helpful) It looks like I have two problems. One problem has
to do with speed and one has to do with convergence.

fmincon is working slowly for me because I did not supply the
gradient. If I can find an efficient way to supply the gradient
myself, it should speed things up a lot.

The convergence problem was because of bad scaling. I've rescaled my
objectives and it converges well now.

 Marcus M. Edvall wrote:

> You could try SNOPT, NPSOL and KNITRO (3 ALG's internally, see
> manual) and CONOPT. They are normally quite a bit faster.

Using one of these algorithms might help too, but I'll probably end
up with the same problems with them unless I have good scaling and
provide a gradient

Hi Eric,

A derivative free method like rbfSolve or EGO might be appliacable
for the initial phase - they do very few objective function calls
since they work on internal approximations.

Can you send a zip with the code for analysis?

All the best, Marcus
 <http://tomopt.com/tomlab/>

 Eric Bowman wrote:
>
>
> As I've looked at how my code and emailed Dmitrey Kroshko. (he has
> been very helpful) It looks like I have two problems. One problem
> has
> to do with speed and one has to do with convergence.
>
> fmincon is working slowly for me because I did not supply the
> gradient. If I can find an efficient way to supply the gradient
> myself, it should speed things up a lot.
>
> The convergence problem was because of bad scaling. I've rescaled
> my
> objectives and it converges well now.
>
> Marcus M. Edvall wrote:
>
>> You could try SNOPT, NPSOL and KNITRO (3 ALG's internally, see
>> manual) and CONOPT. They are normally quite a bit faster.
>
> Using one of these algorithms might help too, but I'll probably end
> up with the same problems with them unless I have good scaling and
> provide a gradient