Neural Network Toolbox™

Calculate Jacobian performance vector

Syntax

• [je,jj,normje] = calcjejj(net,Pd,BZ,IWZ,LWZ,N,Ac,El,Q,TS,MR)
  

Description

This function calculates two values (related to the Jacobian of a network) required to calculate the network's Hessian, in a memory-efficient way.

Two values needed to calculate the Hessian of a network are J*E (Jacobian times errors) and J'J (Jacobian squared). However the Jacobian J can take up a lot of memory. This function calculates J*E and J'J by dividing training vectors into groups, calculating partial Jacobians Ji and its associated values Ji*Ei and Ji'Ji, then summing the partial values into the full J*E and J'J values.

This allows the J*E and J'J values to be calculated with a series of smaller Ji matrices instead of a larger J matrix.

[je,jj,normgX] = calcjejj(net,PD,BZ,IWZ,LWZ,N,Ac,El,Q,TS,MR) takes

net	Neural network
PD	Delayed inputs
BZ	Concurrent biases
IWZ	Weighted inputs
LWZ	Weighted layer outputs
N	Net inputs
Ac	Combined layer outputs
El	Layer errors
Q	Concurrent size
TS	Time steps
MR	Memory reduction factor

and returns

je	Jacobian times errors
jj	Jacobian transposed times the Jacobian.normgX
normgX	Norm of gradient

Examples

Here is a linear network with a single input element ranging from 0 to 1, two neurons, and a tap delay on the input with taps at 0, 2, and 4 time steps. The network is also given a recurrent connection from layer 1 to itself with tap delays of [1 2].

• net = newlin([0 1],2);
  net.layerConnect(1,1) = 1;
  net.layerWeights{1,1}.delays = [1 2];
  

Here is a single (Q = 1) input sequence P with five time steps (TS = 5), and the four initial input delay conditions Pi, combined inputs Pc, and delayed inputs Pd.

• P = {0 0.1 0.3 0.6 0.4};
  Pi = {0.2 0.3 0.4 0.1};
  Pc = [Pi P];
  Pd = calcpd(net,5,1,Pc);
  

Here the two initial layer delay conditions for each of the two neurons and the layer targets for the two neurons over five time steps are defined.

• Ai = {[0.5; 0.1] [0.6; 0.5]};
  Tl = {[0.1;0.2] [0.3;0.1], [0.5;0.6] [0.8;0.9], [0.5;0.1]};
  

Here the network's weight and bias values are extracted, and the network's performance and other signals are calculated.

• [perf,El,Ac,N,BZ,IWZ,LWZ] = calcperf(net,X,Pd,Tl,Ai,1,5);
  

Finally you can use calcjejj to calculate the Jacobian times error, Jacobian squared, and the norm of the Jacobian times error, using a memory reduction of 2.

• [je,jj,normje] = calcjejj(net,Pd,BZ,IWZ,LWZ,N,Ac,El,1,5,2);
  

The results should be the same whatever the memory reduction used. Here a memory reduction of 3 is used.

• [je,jj,normje] = calcjejj(net,Pd,BZ,IWZ,LWZ,N,Ac,El,1,5,3);
  

See Also

calcjx, calcjejj

Provide feedback about this page

calcgx

calcjx

© 1984-2008- The MathWorks, Inc.    -   Site Help   -   Patents   -   Trademarks   -   Privacy Policy   -   Preventing Piracy   -   RSS