a problem with neural network training
1 view (last 30 days)
Show older comments
Hi everybody I have read in some papers that in order to avoid your neural network getting stuck in local minima during resampling methods, a network is trained on the entire data set to obtain a model ) with weights W0, then, these weights are used as the starting point for vtraining the other samples. I have applied this method on a dataset but, while trying to train the second sample the training is not being completed. I have changed some of the neural network parameters like min_grad periodically however, it did not solve this problem. Is it due to the fact that our initial guess is so close to the minimum? Is the problem related to my code? or there is other problem in this field. The code is as follows: [x,t]=house_dataset; Inputs=cell(1,2); Targets=cell(1,2); for i=1:2 Ind=randi(size(x,2),size(x,2),1); Inputs{i} = x(:, Ind); Targets{i}=t(Ind); end NN=cell(1,2); net=feedforwardnet; rng(0); NN{1}=train(net,Inputs{1},Targets{1}); IW = NN{1}.IW; LW = NN{1}.LW; B = net.b; NN{2}.IW = IW; NN{2}.LW = LW; NN{2}.b = B; net.initFcn=''; NN{2}=train(net,Inputs{2},Targets{2});
Your help is greatly appreciated.
0 Comments
Accepted Answer
Greg Heath
on 5 Dec 2013
There are MANY thing wrong with your code.
1. Your premise of using a full data design to initialize divided data designs doesn't make sense.
2. Your data selection using randi randomly selects data with replacement. Therefore ,e.g., you may only have 325 unique examples out of 506. The rest are duplicates and 181 are not used at all!
3. The default data division applied to the non-unique data selection not only results in training, validation and test subsets with duplicated data, some data points belong to more than one of the 3 subsets.
4. You train and use a net, NN{1} without checking to see if the design is any good.
5. I see no reason to put networks inside cells.
Thank you for formally accepting my answer
Greg
P.S. Search using
greg fitnet Ntrials % regression/curvefitting
greg patternnet Ntrials % classification/pattern-recognition
for some of my sample code.
2 Comments
Greg Heath
on 6 Dec 2013
1. Thanks for the references. However, I don't have access to them. Moreover, since the dates are ancient, I'll stand by my skepticism.
2. Sampling with replacement does not mean that you should not try to use all of the data in each replication. The last subset should contain all of the data not chosen by the other subset(s). However, you can get away with it if you have a huge data set AND a huge number of replications. Otherwise, you should try to use all of the data in each replication.
4. I don't object to storing nets in a cell. However if you use the command whos, you will see that your second net is not recognized as a net.
More Answers (0)
See Also
Categories
Find more on Sequence and Numeric Feature Data Workflows in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!