LSTM

LSTM module including forward propagation and backward propagation algorithm

This code implements forward propagation and backward propagation of Long-Short Term Memory recurrent neural network. Please note this code is a part of a library so please see below for how to use. This file (LSTM.m) extends BaseLayer class (classdef LSTM < BaseLayer) but it can be deleted.
<initialization>
obj.vis = vis; % number of dimension of input vector
obj.hid = hid; % number of dimension of output vector
obj.T = T; % Length of time step for BPTT
obj.batchSize = batchSize; % size of mini-batch
obj.prms = cell(obj.prmNum, 1); % container for parameters
obj.states = cell(obj.stateNum,1); % container for state of gates, memory cell, hidden units
obj.delta = zeros(vis, batchSize, T); % delta to be passed to a lower layer

initPrms(obj);
initStates(obj);
obj.gprms = obj.prms;

<forward propagation>
LSTM.affineTrans(input); % input should be a matrix size of vis * batchSize * T
output = LSTM.nonlinearTrans();

<backward propagation>
dgate = LSTM.bpropGate(delta); % this delta is from a upper layer
newDelta = LSTM.bpropDelta(dgate);