Thread Subject: fminunc stepsize

Hi,

I am using fminunc to minimize a two dimensional (x and y)
linear function.

My question is what stepize (i.e. unit vector) does fminunc
use when going away from the initial position that I give it?

Thanks in advance

Dan

"Daniel Burke" <daniel.burke@nuigalway.ie> wrote in message
<g69m9t$sif$1@fred.mathworks.com>...
> Hi,
>
> I am using fminunc to minimize a two dimensional (x and y)
> linear function.
>
> My question is what stepize (i.e. unit vector) does fminunc
> use when going away from the initial position that I give it?
>
> Thanks in advance
>
> Dan

You want to minimize a linear function using
an unconstrained optimizer? I'll save you the
effort.

[-inf,-inf]

Or possibly a +inf in there, depending on
some signs. Whats the point? Why do you care
how quickly you get to infinity? It will take a
long time, regardless of the step size.

John

"John D'Errico" <woodchips@rochester.rr.com> wrote in
message <g69msi$3qi$1@fred.mathworks.com>...
> "Daniel Burke" <daniel.burke@nuigalway.ie> wrote in message
> <g69m9t$sif$1@fred.mathworks.com>...
> > Hi,
> >
> > I am using fminunc to minimize a two dimensional (x and y)
> > linear function.
> >
> > My question is what stepize (i.e. unit vector) does fminunc
> > use when going away from the initial position that I
give it?
> >
> > Thanks in advance
> >
> > Dan
>
> You want to minimize a linear function using
> an unconstrained optimizer? I'll save you the
> effort.
>
> [-inf,-inf]
>
> Or possibly a +inf in there, depending on
> some signs. Whats the point? Why do you care
> how quickly you get to infinity? It will take a
> long time, regardless of the step size.
>
> John


The minimisation works fine, it actually only requires
around 20 iterations to find the minimum of the function.

The problem is this function is at a minimum at a certain x
and y value and I want to find them.

"Daniel Burke" <daniel.burke@nuigalway.ie> wrote in message
<g69of9$gkd$1@fred.mathworks.com>...
> "John D'Errico" <woodchips@rochester.rr.com> wrote in
> message <g69msi$3qi$1@fred.mathworks.com>...
> > "Daniel Burke" <daniel.burke@nuigalway.ie> wrote in message
> > <g69m9t$sif$1@fred.mathworks.com>...
> > > Hi,
> > >
> > > I am using fminunc to minimize a two dimensional (x and y)
> > > linear function.
> > >
> > > My question is what stepize (i.e. unit vector) does fminunc
> > > use when going away from the initial position that I
> give it?
> > >
> > > Thanks in advance
> > >
> > > Dan
> >
> > You want to minimize a linear function using
> > an unconstrained optimizer? I'll save you the
> > effort.
> >
> > [-inf,-inf]
> >
> > Or possibly a +inf in there, depending on
> > some signs. Whats the point? Why do you care
> > how quickly you get to infinity? It will take a
> > long time, regardless of the step size.
> >
> > John
>
>
> The minimisation works fine, it actually only requires
> around 20 iterations to find the minimum of the function.
>
> The problem is this function is at a minimum at a certain x
> and y value and I want to find them.

I don't think you understand. A linear function
is unbounded. IT HAS NO MINIMUM.

So is your function not truly linear?

By the way, you cannot set the step size for
fminunc anyway.

John

"John D'Errico" <woodchips@rochester.rr.com> wrote in
message <g69p1v$lee$1@fred.mathworks.com>...
> "Daniel Burke" <daniel.burke@nuigalway.ie> wrote in message
> <g69of9$gkd$1@fred.mathworks.com>...
> > "John D'Errico" <woodchips@rochester.rr.com> wrote in
> > message <g69msi$3qi$1@fred.mathworks.com>...
> > > "Daniel Burke" <daniel.burke@nuigalway.ie> wrote in
message
> > > <g69m9t$sif$1@fred.mathworks.com>...
> > > > Hi,
> > > >
> > > > I am using fminunc to minimize a two dimensional (x
and y)
> > > > linear function.
> > > >
> > > > My question is what stepize (i.e. unit vector) does
fminunc
> > > > use when going away from the initial position that I
> > give it?
> > > >
> > > > Thanks in advance
> > > >
> > > > Dan
> > >
> > > You want to minimize a linear function using
> > > an unconstrained optimizer? I'll save you the
> > > effort.
> > >
> > > [-inf,-inf]
> > >
> > > Or possibly a +inf in there, depending on
> > > some signs. Whats the point? Why do you care
> > > how quickly you get to infinity? It will take a
> > > long time, regardless of the step size.
> > >
> > > John
> >
> >
> > The minimisation works fine, it actually only requires
> > around 20 iterations to find the minimum of the function.
> >
> > The problem is this function is at a minimum at a certain x
> > and y value and I want to find them.
>
> I don't think you understand. A linear function
> is unbounded. IT HAS NO MINIMUM.
>
> So is your function not truly linear?
>
> By the way, you cannot set the step size for
> fminunc anyway.
>
> John

To be honest it's not truly linear, its affine i think.

My function is the logarithm of a Gaussian distribution.

I don't need to set the stepsize I was just wondering is
there a way to know what the step sizes are?

Thanks John for your reply

"Daniel Burke" <daniel.burke@nuigalway.ie> wrote in message
<g69pd7$oj6$1@fred.mathworks.com>...

>
> To be honest it's not truly linear, its affine i think.
>
> My function is the logarithm of a Gaussian distribution.
>

Let me guess:

- What you do is a fit to data (transformed by log from a
Gaussian-like original data) by an affine function.

- This is *not* a minimization of the affine/linear
function, as you wrote at at the first post.

Bruno

"Daniel Burke" <daniel.burke@nuigalway.ie> wrote in message
<g69pd7$oj6$1@fred.mathworks.com>...
 
> To be honest it's not truly linear, its affine i think.

Let me guess that you are not minimizing
an affine function anyway. This would be a
mapping from one domain to another. As
such, it has no "minimum".

 
> My function is the logarithm of a Gaussian distribution.

The log of a gaussian "distribution"? I think
there may still be some confusion about
terminology.

Perhaps this is the negative log of a gaussian
likelihood function, or something like that,
used (a guess from your other comments) to
register two images.

 
> I don't need to set the stepsize I was just wondering is
> there a way to know what the step sizes are?

Do you want to know in advance or set in
advance the step size? No. Fminunc uses a
gradient based tool, that will determine the
step size it will use based on the shape of
your function.

I can't think of a way that fminunc will tell
you the step size (in the context of the large
or medium scale methods) it is currently
using either, at least not without use of the
debugger.

John

"John D'Errico" <woodchips@rochester.rr.com> wrote in
message <g69stp$a8$1@fred.mathworks.com>...
> "Daniel Burke" <daniel.burke@nuigalway.ie> wrote in message
> <g69pd7$oj6$1@fred.mathworks.com>...
>
> > To be honest it's not truly linear, its affine i think.
>
> Let me guess that you are not minimizing
> an affine function anyway. This would be a
> mapping from one domain to another. As
> such, it has no "minimum".
>
>
> > My function is the logarithm of a Gaussian distribution.
>
> The log of a gaussian "distribution"? I think
> there may still be some confusion about
> terminology.
>
> Perhaps this is the negative log of a gaussian
> likelihood function, or something like that,
> used (a guess from your other comments) to
> register two images.
>
>
> > I don't need to set the stepsize I was just wondering is
> > there a way to know what the step sizes are?
>
> Do you want to know in advance or set in
> advance the step size? No. Fminunc uses a
> gradient based tool, that will determine the
> step size it will use based on the shape of
> your function.
>
> I can't think of a way that fminunc will tell
> you the step size (in the context of the large
> or medium scale methods) it is currently
> using either, at least not without use of the
> debugger.
>
> John

You're right John it is the negative log of a gaussian
likelihood function.

> Do you want to know in advance or set in
> advance the step size? No. Fminunc uses a
> gradient based tool, that will determine the
> step size it will use based on the shape of
> your function.

No i don't need to set the step size I was just wondering if
there is a way of reporting upon the steps that fminunc
uses. I had forgotten that it uses a gradient based method

Thanks

"Daniel Burke" <daniel.burke@nuigalway.ie> wrote in message
news:g69uap$ivo$1@fred.mathworks.com...

*snip*

> No i don't need to set the step size I was just wondering if
> there is a way of reporting upon the steps that fminunc
> uses. I had forgotten that it uses a gradient based method

That's a different question than the one you asked first. If you want to
see what FMINUNC is doing as it processes your problem, use an OutputFcn.
Scroll down to the Options section of the reference page for FMINUNC for
more information on the OutputFcn option.

http://www.mathworks.com/access/helpdesk/help/toolbox/optim/ug/fminunc.html

--
Steve Lord
slord@mathworks.com

"Daniel Burke" <daniel.burke@nuigalway.ie> wrote in message
<g69m9t$sif$1@fred.mathworks.com>...
> Hi,
>
> I am using fminunc to minimize a two dimensional (x and y)
> linear function.
>
> My question is what stepize (i.e. unit vector) does fminunc
> use when going away from the initial position that I give it?
>
> Thanks in advance
>
> Dan

You want to minimize a linear function using
an unconstrained optimizer? I'll save you the
effort.

[-inf,-inf]

Or possibly a +inf in there, depending on
some signs. Whats the point? Why do you care
how quickly you get to infinity? It will take a
long time, regardless of the step size.

John

"John D'Errico" <woodchips@rochester.rr.com> wrote in
message <g69msi$3qi$1@fred.mathworks.com>...
> "Daniel Burke" <daniel.burke@nuigalway.ie> wrote in message
> <g69m9t$sif$1@fred.mathworks.com>...
> > Hi,
> >
> > I am using fminunc to minimize a two dimensional (x and y)
> > linear function.
> >
> > My question is what stepize (i.e. unit vector) does fminunc
> > use when going away from the initial position that I
give it?
> >
> > Thanks in advance
> >
> > Dan
>
> You want to minimize a linear function using
> an unconstrained optimizer? I'll save you the
> effort.
>
> [-inf,-inf]
>
> Or possibly a +inf in there, depending on
> some signs. Whats the point? Why do you care
> how quickly you get to infinity? It will take a
> long time, regardless of the step size.
>
> John


The minimisation works fine, it actually only requires
around 20 iterations to find the minimum of the function.

The problem is this function is at a minimum at a certain x
and y value and I want to find them.

"Daniel Burke" <daniel.burke@nuigalway.ie> wrote in message
<g69of9$gkd$1@fred.mathworks.com>...
> "John D'Errico" <woodchips@rochester.rr.com> wrote in
> message <g69msi$3qi$1@fred.mathworks.com>...
> > "Daniel Burke" <daniel.burke@nuigalway.ie> wrote in message
> > <g69m9t$sif$1@fred.mathworks.com>...
> > > Hi,
> > >
> > > I am using fminunc to minimize a two dimensional (x and y)
> > > linear function.
> > >
> > > My question is what stepize (i.e. unit vector) does fminunc
> > > use when going away from the initial position that I
> give it?
> > >
> > > Thanks in advance
> > >
> > > Dan
> >
> > You want to minimize a linear function using
> > an unconstrained optimizer? I'll save you the
> > effort.
> >
> > [-inf,-inf]
> >
> > Or possibly a +inf in there, depending on
> > some signs. Whats the point? Why do you care
> > how quickly you get to infinity? It will take a
> > long time, regardless of the step size.
> >
> > John
>
>
> The minimisation works fine, it actually only requires
> around 20 iterations to find the minimum of the function.
>
> The problem is this function is at a minimum at a certain x
> and y value and I want to find them.

I don't think you understand. A linear function
is unbounded. IT HAS NO MINIMUM.

So is your function not truly linear?

By the way, you cannot set the step size for
fminunc anyway.

John

"John D'Errico" <woodchips@rochester.rr.com> wrote in
message <g69p1v$lee$1@fred.mathworks.com>...
> "Daniel Burke" <daniel.burke@nuigalway.ie> wrote in message
> <g69of9$gkd$1@fred.mathworks.com>...
> > "John D'Errico" <woodchips@rochester.rr.com> wrote in
> > message <g69msi$3qi$1@fred.mathworks.com>...
> > > "Daniel Burke" <daniel.burke@nuigalway.ie> wrote in
message
> > > <g69m9t$sif$1@fred.mathworks.com>...
> > > > Hi,
> > > >
> > > > I am using fminunc to minimize a two dimensional (x
and y)
> > > > linear function.
> > > >
> > > > My question is what stepize (i.e. unit vector) does
fminunc
> > > > use when going away from the initial position that I
> > give it?
> > > >
> > > > Thanks in advance
> > > >
> > > > Dan
> > >
> > > You want to minimize a linear function using
> > > an unconstrained optimizer? I'll save you the
> > > effort.
> > >
> > > [-inf,-inf]
> > >
> > > Or possibly a +inf in there, depending on
> > > some signs. Whats the point? Why do you care
> > > how quickly you get to infinity? It will take a
> > > long time, regardless of the step size.
> > >
> > > John
> >
> >
> > The minimisation works fine, it actually only requires
> > around 20 iterations to find the minimum of the function.
> >
> > The problem is this function is at a minimum at a certain x
> > and y value and I want to find them.
>
> I don't think you understand. A linear function
> is unbounded. IT HAS NO MINIMUM.
>
> So is your function not truly linear?
>
> By the way, you cannot set the step size for
> fminunc anyway.
>
> John

To be honest it's not truly linear, its affine i think.

My function is the logarithm of a Gaussian distribution.

I don't need to set the stepsize I was just wondering is
there a way to know what the step sizes are?

Thanks John for your reply

"Daniel Burke" <daniel.burke@nuigalway.ie> wrote in message
<g69pd7$oj6$1@fred.mathworks.com>...

>
> To be honest it's not truly linear, its affine i think.
>
> My function is the logarithm of a Gaussian distribution.
>

Let me guess:

- What you do is a fit to data (transformed by log from a
Gaussian-like original data) by an affine function.

- This is *not* a minimization of the affine/linear
function, as you wrote at at the first post.

Bruno

"Daniel Burke" <daniel.burke@nuigalway.ie> wrote in message
<g69pd7$oj6$1@fred.mathworks.com>...
 
> To be honest it's not truly linear, its affine i think.

Let me guess that you are not minimizing
an affine function anyway. This would be a
mapping from one domain to another. As
such, it has no "minimum".

 
> My function is the logarithm of a Gaussian distribution.

The log of a gaussian "distribution"? I think
there may still be some confusion about
terminology.

Perhaps this is the negative log of a gaussian
likelihood function, or something like that,
used (a guess from your other comments) to
register two images.

 
> I don't need to set the stepsize I was just wondering is
> there a way to know what the step sizes are?

Do you want to know in advance or set in
advance the step size? No. Fminunc uses a
gradient based tool, that will determine the
step size it will use based on the shape of
your function.

I can't think of a way that fminunc will tell
you the step size (in the context of the large
or medium scale methods) it is currently
using either, at least not without use of the
debugger.

John

"John D'Errico" <woodchips@rochester.rr.com> wrote in
message <g69stp$a8$1@fred.mathworks.com>...
> "Daniel Burke" <daniel.burke@nuigalway.ie> wrote in message
> <g69pd7$oj6$1@fred.mathworks.com>...
>
> > To be honest it's not truly linear, its affine i think.
>
> Let me guess that you are not minimizing
> an affine function anyway. This would be a
> mapping from one domain to another. As
> such, it has no "minimum".
>
>
> > My function is the logarithm of a Gaussian distribution.
>
> The log of a gaussian "distribution"? I think
> there may still be some confusion about
> terminology.
>
> Perhaps this is the negative log of a gaussian
> likelihood function, or something like that,
> used (a guess from your other comments) to
> register two images.
>
>
> > I don't need to set the stepsize I was just wondering is
> > there a way to know what the step sizes are?
>
> Do you want to know in advance or set in
> advance the step size? No. Fminunc uses a
> gradient based tool, that will determine the
> step size it will use based on the shape of
> your function.
>
> I can't think of a way that fminunc will tell
> you the step size (in the context of the large
> or medium scale methods) it is currently
> using either, at least not without use of the
> debugger.
>
> John

You're right John it is the negative log of a gaussian
likelihood function.

> Do you want to know in advance or set in
> advance the step size? No. Fminunc uses a
> gradient based tool, that will determine the
> step size it will use based on the shape of
> your function.

No i don't need to set the step size I was just wondering if
there is a way of reporting upon the steps that fminunc
uses. I had forgotten that it uses a gradient based method

Thanks

"Daniel Burke" <daniel.burke@nuigalway.ie> wrote in message
news:g69uap$ivo$1@fred.mathworks.com...

*snip*

> No i don't need to set the step size I was just wondering if
> there is a way of reporting upon the steps that fminunc
> uses. I had forgotten that it uses a gradient based method

That's a different question than the one you asked first. If you want to
see what FMINUNC is doing as it processes your problem, use an OutputFcn.
Scroll down to the Options section of the reference page for FMINUNC for
more information on the OutputFcn option.

http://www.mathworks.com/access/helpdesk/help/toolbox/optim/ug/fminunc.html

--
Steve Lord
slord@mathworks.com