1-D minimization using golden section search

`[a,gX,perf,retcode,delta,tol] = srchgol(net,X,Pd,Tl,Ai,Q,TS,dX,gX,perf,dperf,delta,tol,ch_perf)`

`srchgol`

is a linear search routine. It searches
in a given direction to locate the minimum of the performance function
in that direction. It uses a technique called the golden section search.

`[a,gX,perf,retcode,delta,tol] = srchgol(net,X,Pd,Tl,Ai,Q,TS,dX,gX,perf,dperf,delta,tol,ch_perf)`

takes
these inputs,

`net` | Neural network |

`X` | Vector containing current values of weights and biases |

`Pd` | Delayed input vectors |

`Tl` | Layer target vectors |

`Ai` | Initial input delay conditions |

`Q` | Batch size |

`TS` | Time steps |

`dX` | Search direction vector |

`gX` | Gradient vector |

`perf` | Performance value at current |

`dperf` | Slope of performance value at current |

`delta` | Initial step size |

`tol` | Tolerance on search |

`ch_perf` | Change in performance on previous step |

and returns

`a` | Step size that minimizes performance |

`gX` | Gradient at new minimum point |

`perf` | Performance value at new minimum point |

`retcode` | Return code that has three elements. The first two elements correspond to the number of function evaluations in the two stages of the search. The third element is a return code. These have different meanings for different search algorithms. Some might not be used in this function. |

`0` Normal | |

`1` Minimum
step taken | |

`2` Maximum
step taken | |

`3` Beta
condition not met | |

`delta` | New initial step size, based on the current step size |

`tol` | New tolerance on search |

Parameters used for the golden section algorithm are

`alpha` | Scale factor that determines sufficient reduction in |

`bmax` | Largest step size |

`scale_tol` | Parameter that relates the tolerance |

The defaults for these parameters are set in the training function
that calls them. See `traincgf`

, `traincgb`

, `traincgp`

, `trainbfg`

, and `trainoss`

.

Dimensions for these variables are

`Pd` |
| Each element |

`Tl` |
| Each element |

`Ai` |
| Each element |

where

`Ni ` | `=` | `net.numInputs` |

`Nl ` | `=` | `net.numLayers` |

`LD ` | `=` | `net.numLayerDelays` |

`Ri ` | `=` | `net.inputs{i}.size` |

`Si` | `=` | `net.layers{i}.size` |

`Vi` | `=` | `net.targets{i}.size` |

`Dij` | `=` | `Ri * length(net.inputWeights{i,j}.delays)` |

Here is a problem consisting of inputs `p`

and
targets `t`

to be solved with a network.

p = [0 1 2 3 4 5]; t = [0 0 0 1 1 1];

A two-layer feed-forward network is created. The network's
input ranges from `[0 to 10]`

. The first layer has
two `tansig`

neurons, and the second layer has one `logsig`

neuron.
The `traincgf`

network training function and the `srchgol`

search
function are to be used.

net = newff([0 5],[2 1],{'tansig','logsig'},'traincgf'); a = sim(net,p)

net.trainParam.searchFcn = 'srchgol'; net.trainParam.epochs = 50; net.trainParam.show = 10; net.trainParam.goal = 0.1; net = train(net,p,t); a = sim(net,p)

You can create a standard network that uses `srchgol`

with `newff`

, `newcf`

,
or `newelm`

.

To prepare a custom network to be trained with `traincgf`

,
using the line search function `srchgol`

,

Set

`net.trainFcn`

to`'traincgf'`

. This sets`net.trainParam`

to`traincgf`

's default parameters.Set

`net.trainParam.searchFcn`

to`'srchgol'`

.

The `srchgol`

function can be used with any
of the following training functions: `traincgf`

, `traincgb`

, `traincgp`

, `trainbfg`

, `trainoss`

.

The golden section search `srchgol`

is a linear search that does not require the calculation of
the slope. This routine begins by locating an interval in which the
minimum of the performance function occurs. This is accomplished by
evaluating the performance at a sequence of points, starting at a
distance of `delta`

and doubling in distance each
step, along the search direction. When the performance increases between
two successive iterations, a minimum has been bracketed. The next
step is to reduce the size of the interval containing the minimum.
Two new points are located within the initial interval. The values
of the performance at these two points determine a section of the
interval that can be discarded, and a new interior point is placed
within the new interval. This procedure is continued until the interval
of uncertainty is reduced to a width of `tol`

, which
is equal to `delta`

/`scale_tol`

.

See [HDB96],
starting on page 12-16, for a complete description of the golden section
search. Try the *Neural Network Design* demonstration `nnd12sd1`

[HDB96] for
an illustration of the performance of the golden section search in
combination with a conjugate gradient algorithm.

Scales, L.E., *Introduction to Non-Linear Optimization*,
New York, Springer-Verlag, 1985

Was this topic helpful?