## Documentation Center |

1-D minimization using backtracking

`[a,gX,perf,retcode,delta,tol] = srchbac(net,X,Pd,Tl,Ai,Q,TS,dX,gX,perf,dperf,delta,TOL,ch_perf)`

`srchbac` is a linear search routine. It searches
in a given direction to locate the minimum of the performance function
in that direction. It uses a technique called backtracking.

`[a,gX,perf,retcode,delta,tol] = srchbac(net,X,Pd,Tl,Ai,Q,TS,dX,gX,perf,dperf,delta,TOL,ch_perf)` takes
these inputs,

net | Neural network |

X | Vector containing current values of weights and biases |

Pd | Delayed input vectors |

Tl | Layer target vectors |

Ai | Initial input delay conditions |

Q | Batch size |

TS | Time steps |

dX | Search direction vector |

gX | Gradient vector |

perf | Performance value at current |

dperf | Slope of performance value at current |

delta | Initial step size |

tol | Tolerance on search |

ch_perf | Change in performance on previous step |

and returns

a | Step size that minimizes performance |

gX | Gradient at new minimum point |

perf | Performance value at new minimum point |

retcode | Return code that has three elements. The first two elements correspond to the number of function evaluations in the two stages of the search. The third element is a return code. These have different meanings for different search algorithms. Some might not be used in this function. |

0 Normal | |

1 Minimum
step taken | |

2 Maximum
step taken | |

3 Beta
condition not met | |

delta | New initial step size, based on the current step size |

tol | New tolerance on search |

Parameters used for the backstepping algorithm are

alpha | Scale factor that determines sufficient reduction in |

beta | Scale factor that determines sufficiently large step size |

low_lim | Lower limit on change in step size |

up_lim | Upper limit on change in step size |

maxstep | Maximum step length |

minstep | Minimum step length |

scale_tol | Parameter that relates the tolerance |

The defaults for these parameters are set in the training function
that calls them. See `traincgf`, `traincgb`, `traincgp`, `trainbfg`, and `trainoss`.

Dimensions for these variables are

Pd |
| Each element |

Tl |
| Each element |

V |
| Each element |

where

Ni | = | net.numInputs |

Nl | = | net.numLayers |

LD | = | net.numLayerDelays |

Ri | = | net.inputs{i}.size |

Si | = | net.layers{i}.size |

Vi | = | net.targets{i}.size |

Dij | = | Ri * length(net.inputWeights{i,j}.delays) |

Here is a problem consisting of inputs `p` and
targets `t` to be solved with a network.

p = [0 1 2 3 4 5]; t = [0 0 0 1 1 1];

A two-layer feed-forward network is created. The network's
input ranges from `[0 to 10]`. The first layer has
two `tansig` neurons, and the second layer has one `logsig` neuron.
The `traincgf` network training function and the `srchbac` search
function are to be used.

net = newff([0 5],[2 1],{'tansig','logsig'},'traincgf'); a = sim(net,p)

net.trainParam.searchFcn = 'srchbac'; net.trainParam.epochs = 50; net.trainParam.show = 10; net.trainParam.goal = 0.1; net = train(net,p,t); a = sim(net,p)

You can create a standard network that uses `srchbac` with `newff`, `newcf`,
or `newelm`.

To prepare a custom network to be trained with `traincgf`,
using the line search function `srchbac`,

The `srchbac` function can be used with any
of the following training functions: `traincgf`, `traincgb`, `traincgp`, `trainbfg`, `trainoss`.

The backtracking search routine `srchbac` is
best suited to use with the quasi-Newton optimization algorithms. It begins with a
step multiplier of 1 and then backtracks until an acceptable reduction
in the performance is obtained. On the first step it uses the value
of performance at the current point and a step multiplier of 1. It
also uses the value of the derivative of performance at the current
point to obtain a quadratic approximation to the performance function
along the search direction. The minimum of the quadratic approximation
becomes a tentative optimum point (under certain conditions) and the
performance at this point is tested. If the performance is not sufficiently
reduced, a cubic interpolation is obtained and the minimum of the
cubic interpolation becomes the new tentative optimum point. This
process is continued until a sufficient reduction in the performance
is obtained.

The backtracking algorithm is described in Dennis and Schnabel. It is used as the default line search for the quasi-Newton algorithms, although it might not be the best technique for all problems.

Dennis, J.E., and R.B. Schnabel, *Numerical Methods
for Unconstrained Optimization and Nonlinear Equations*,
Englewood Cliffs, NJ, Prentice-Hall, 1983

Was this topic helpful?