Path: news.mathworks.com!newsfeed-00.mathworks.com!newscon02.news.prodigy.net!prodigy.net!news.glorb.com!news.aset.psu.edu!news.cse.psu.edu!elk.ncren.net!newsflash.concordia.ca!canopus.cc.umanitoba.ca!not-for-mail
From: roberson@ibd.nrc-cnrc.gc.ca (Walter Roberson)
Newsgroups: comp.soft-sys.matlab
Subject: Re: Find Minimum
Date: Mon, 11 Feb 2008 21:36:55 +0000 (UTC)
Organization: National Research Council Canada - Conseil national de rechereches Canada
Lines: 53
Message-ID: <foqf5n$kpq$1@canopus.cc.umanitoba.ca>
References: <ac460e51-a1a5-4e93-9031-220be5a23e5a@f10g2000hsf.googlegroups.com> <Xns9A419D4C96F57scottseidmanmindspri@130.133.1.4> <foqcnc$i3l$1@canopus.cc.umanitoba.ca> <Xns9A41A493EF475scottseidmanmindspri@130.133.1.4>
NNTP-Posting-Host: origin.ibd.nrc.ca
X-Trace: canopus.cc.umanitoba.ca 1202765815 21306 192.70.172.160 (11 Feb 2008 21:36:55 GMT)
X-Complaints-To: abuse@cc.umanitoba.ca
NNTP-Posting-Date: Mon, 11 Feb 2008 21:36:55 +0000 (UTC)
Originator: roberson@ibd.nrc-cnrc.gc.ca (Walter Roberson)
Xref: news.mathworks.com comp.soft-sys.matlab:450670


In article <Xns9A41A493EF475scottseidmanmindspri@130.133.1.4>,
Scott Seidman  <namdiesttocs@mindspring.com> wrote:
>roberson@ibd.nrc-cnrc.gc.ca (Walter Roberson) wrote in news:foqcnc$i3l$1
>@canopus.cc.umanitoba.ca:

>> Even if you had an algorithm that (for example) after five
>> function evaluations was able to predict a parameter combination that
>> was unsurpassed in another billion explorations of the parameter
>> space, then because of the black-box nature of the function,

>This is getting a little circular.  To know any function as well as you 
>seem to assert you need to know it to do an optimization, it would seem 
>that you need to know the function value at every possible value of 
>parameter space.

Take a finite subset as an example: let
F(-2^52:2^52-1, 0:10:200, .01:.01:1) = rand(2^53,21,100);
where rand is the standard Matlab pseudo-random 'twister' function
with state initialized to any one integer you choose.

In classical mathematics, this defines a "function". A function
is a mapping between the domain and the range. Nothing more.
There need not be any "formula" for a function, just a list of tuples.
A nice formula can make describing the function tremendously more
compact, but the domain -> range value list is the essence of
functions, not the formulae.

Now, how are you going to find the minimum value of F over this
finite subset without evaluating F at every location?

There are an infinite number of formulae for which any particular
set of F values as described above would be the best approximation.
Some of them are continuously differentiable, some differentiable
only a few steps, some of them nowhere differentiable. And
2^53 is pretty small as the integers go.


>Obviously, this must be a misunderstanding on my part, but it does be the 
>question of hat, exactly, are your requirements for a solvable 
>optimization?

I would have to think about that more.

I was going to put in a few words about "frequency"
(if the frequency of the change is too high, any gradient deduced
around neighbouring points would not be usable to predict new points)
but then I recalled that the frequency of anything approaching
a flat plateau involves an infinite series of non-zero frequencies,
so "frequency" is probably not exactly the right idea.
-- 
  "Every intellectual product must be judged from the point of view
  of the age and the people in which it was produced."
                                              -- Walter Horatio Pater