Code covered by the BSD License  

Highlights from
CNN - Convolutional neural network class

4.92308
4.9 | 13 ratings Rate this file 592 Downloads (last 30 days) File Size: 613 KB File ID: #24291 Version: 1.29
image thumbnail

CNN - Convolutional neural network class

by

Mihail Sirotenko (view profile)

  • 1 file
  • 592 downloads
  • 4.92308

 

28 May 2009 (Updated )

This project provides matlab class for implementation of convolutional neural networks.

| Watch this File

File Information
Description

NOTE: Next versions of this library is hosted at https://bitbucket.org/intelligenceagent/cudacnn-public/wiki/Home
The reason is that now it's rather C++/CUDA library with Matlab front-end than a Matlab library.

This project provides matlab class for implementation of convolutional neural networks. This networks was developed by Yann LeCun and have sucessfully used in many practical applications, such as handwritten digits recognition, face detection, robot navigation and others (see references for more info). Because of some architectural features of convolutional networks, such as weight sharing it is imposible to implement it using Matlab Neural Network Toolbox without it's source modifications. That's why this class works almost independently from NN toolbox (coming soon full independence).

This release includes sample of handwritten digits recognition using CNN. If you just want to try it run cnet_tool. You'll see a simple GUI. It loads pretrained convolutional neural net from cnet.mat and recognizes image of digit either painted in painting area or downloaded from MNIST database.

Further information about the library can be found here:
https://sites.google.com/site/mihailsirotenko/projects/convolutional-neural-network-class
and here:
https://sites.google.com/site/mihailsirotenko/projects/cuda-cnn

Acknowledgements

This file inspired My Cnn.

Required Products MATLAB
MATLAB release MATLAB 7.4 (R2007a)
Tags for This File   Please login to tag files.
Please login to add a comment or rating.
Comments and Ratings (39)
31 Jul 2014 NetDragon wanglong

Hi,
I want to change the last layer(output layer) to use linear regression, how to do this?

07 May 2014 Heider

Heider (view profile)

Hello,
I need the structure of the CNN class, i.e. it's methods and functions, and it's parameters. I need information like the components: numSLayers, numInputs, Perf, num_dec which are found with cnet object in the form of cnet.numSLayers, cnet.numInputs, cnet.Pref ....etc

Comment only
27 Jan 2014 Endel

Endel (view profile)

I have a problem with understanding the actual architecture of this network. Usually, LeCun et al have used different weights for the connections from different feature maps of a previous layer (something that looks like 3D kernel). Therefore, the number of weights of a convolution layer (assuming full map of connections) is kernelHeight*kernelWidth*numFeatMapsLayer(k)*numFeatMapsLayer(k-1). I am not sure, but it seems that there are kernelHeight*kernelWidth*numFeatMapsLayer(k) different weights used in this program. Does it means that the connections from different feature maps of a previous layer to a particular feature map of the next layer have the same weights? Or maybe, I misunderstand something?

Comment only
11 Jul 2013 Manav

Manav (view profile)

I want to use this toolbox on my own sound data set. Each spectogram equivalent to 1 image has dimensions 13x500, where as each image in MNIST has dimensions 28x28. I want to change the input width and height to 13 and 500 respectively. How do I go about doing so?

Comment only
24 May 2013 pan junfu

I apllied it to traffic sign recognition,it classified all classes to a same class,have you had any experience with this - if so what parameters might you suggest I change?How can I train the cnn with the features extracted from the images instead of the images themselves?
Where should I change?

29 Apr 2013 tc

tc (view profile)

Hello, I have run train_cnn on Octave, however, I got a error message "max_recursion_depth exceeded". Is anybody know/experience same problem? How to solve it? Thanks.

Comment only
12 Feb 2013 Johnathan  
07 Feb 2013 Mihail Sirotenko

Mihail Sirotenko (view profile)

  • 1 file
  • 592 downloads
  • 4.92308

In this pure matlab version there's no max pooling. But you can find it in C++/CUDA version, which is on bitbucket repository.

Comment only
06 Feb 2013 Johnathan

I've been searching around for a bit but I can't find the max pooling layer code. Can someone point me in the right direction?

Comment only
29 Oct 2012 Elham

Elham (view profile)

Hi, I am running the demo train-cnn which is provided to train the network from Dr. Lecun's paper. I do full training for over 20 epochs. The best MCR I get is .12 which corresponds to 12% error. How can I improve this? (The goal is to get .8% error)
Thanks

Comment only
23 Oct 2012 Mihail Sirotenko

Mihail Sirotenko (view profile)

  • 1 file
  • 592 downloads
  • 4.92308

2Eliud Gonzalez:
1. the output of CNN is saved to workspace variable sinet. You can use matlab function "save" to save it on disk.
2. Hessian computation controlled on line 147:
sinet.HcalcMode = 0;
Default value 0 means running estimate of hessian. In this case you'll not notice it in gui. If you change it to 1, gui progress bar will show the hessian recomputation.

Comment only
22 Oct 2012 Eliud Gonzalez

Thanks, when training the CNN, how do i save it, or it saves itself? and why Hessian calculation is always 0?

Comment only
16 Oct 2012 Mihail Sirotenko

Mihail Sirotenko (view profile)

  • 1 file
  • 592 downloads
  • 4.92308

2 Eliud Gonzalez: usually this means that you don't have MNIST dataset. It is not distributed with the library, but you can download in from http://yann.lecun.com/exdb/mnist/
Please dowload and put the files into the MNIST folder.

Comment only
16 Oct 2012 Eliud Gonzalez

I get this errors when running train_cnn.m
??? Error using ==> fread
Invalid file identifier. Use fopen to generate a valid file identifier.

Error in ==> readMNIST at 25
magicNum = fread(fid,1,'int32'); %Magic number

Error in ==> train_cnn at 30
[I,labels,I_test,labels_test] = readMNIST(1000);

>>

Comment only
16 Oct 2012 Eliud Gonzalez

What can i do if i open train_cnn.m and click run, and nothing happens, do i need to install CUDA, i'm sorry i'm new in this world :(

Comment only
15 Oct 2012 Mihail Sirotenko

Mihail Sirotenko (view profile)

  • 1 file
  • 592 downloads
  • 4.92308

2 yin: yes, it can. Actually you can feed in any data not only images.
2 Xiaolei Hu: As far as I know there's no support of Conv Nets in latest Matlab NN Toolbox.

Comment only
12 Oct 2012 Xiaolei Hu

Does anybody know whether the latest Matlab has changed its NN toolbox so CNN can be implemented?

Comment only
27 Aug 2012 yin

yin (view profile)

can the cnn train with the features extracted from the images instead of the images themselves?

16 Mar 2012 Mihail Sirotenko

Mihail Sirotenko (view profile)

  • 1 file
  • 592 downloads
  • 4.92308

Unfortunately I don't have Matlab 2011b. It seems that they changed something in NN toolbox. I'm going to remove the dependencies from NN toolbox in next release, which I'm trying to finalize now.

Comment only
02 Mar 2012 Atanasko Atanasov

I try to train the network but I get error:
Undefined function or variable 'boiler_transfer'.
Error in tansig_mod (line 64)

I am using MATLAB R2011b. Any help?

Comment only
06 Feb 2012 Mihail Sirotenko

Mihail Sirotenko (view profile)

  • 1 file
  • 592 downloads
  • 4.92308

Full training cycle needs about 10 epochs on full training set with progressively decreasing theta. As far as I understand you did 1 epoch.

Comment only
28 Jan 2012 Bo Hu

Bo Hu (view profile)

This is a great job. However I have a question. I test this program using the MNIST handwritten digit database. The mcr rate is very high (about 15%) even I train the cnn using 10000 input. If I tried to train the cnn using 60000 input, then the program would took fairly long time, about several hours to finish. Anyway, the mcr is always about 15%. I wonder what parameter we should change to make the prediction accuracy higher. Thanks a lot.

12 Jan 2012 Mihail Sirotenko

Mihail Sirotenko (view profile)

  • 1 file
  • 592 downloads
  • 4.92308

Regarding several last questions. I'm going to put new release of the lib which will contain example with faces and would be more convenient to use with various data. It already works in general. But I need a week or two more to finalize it since I do it in my free time.

Comment only
11 Jan 2012 chipo josé

hi
please can any one help as to adapt this code to face recognition
thanks

Comment only
14 Nov 2011 Zhipan Ren  
13 Sep 2011 Aron Sceidt

seeing it now... looks great :) is it possible to load a jpg or png in some way? what would be required?

03 Aug 2011 Mihail Sirotenko

Mihail Sirotenko (view profile)

  • 1 file
  • 592 downloads
  • 4.92308

2Gaurav. It means that file train_cnn.m have comments for almost every line of code, so you can find parameter you want to change and actually change it before start training. Basically train_cnn.m contains settings for network architecture (number of layers, number of neurons etc)

Comment only
02 Aug 2011 Gaurav

Gaurav (view profile)

in the read me file to train the network "
If you interested in training you should open train_cnn.m, set all parameters following to comments and start learning by runing it."
i cant understand the "set all parameters following to comments and start learning by runing it." part

please help

Comment only
10 Jun 2011 Jose M. Alvarez

Is it possible to train using color images? (from the example it says it is not). However, the version from Nikolay Chumerin can be trained using color patches.

11 Apr 2011 Silvio Filipe  
19 Jan 2011 Sebastien PARIS

Sebastien PARIS (view profile)

 
15 Oct 2010 Mihail Sirotenko

Mihail Sirotenko (view profile)

  • 1 file
  • 592 downloads
  • 4.92308

Thanks!
Actually I had a plans to adapt this to face recognition, but unfortunately there're still only plans, because now I have absolutely no time to develop it.
So I'd advice to look closely on preprocessing algorithms and try to make architecture similar to face detection papers advices.

Comment only
15 Oct 2010 Mark

Mark (view profile)

Excellent work!

I've attempted to get this working on face recognition but without success. The MCR remains very high. Have you had any experience with this - if so what parameters might you suggest I change?

29 Aug 2010 pupu QQ

it's ok ,but only numbers.high quality .

Comment only
25 Jun 2010 Jonathan Masci  
23 Nov 2009 Mihail Sirotenko

Mihail Sirotenko (view profile)

  • 1 file
  • 592 downloads
  • 4.92308

Thanks a lot for the valuable comment. First bug is a result of merging different versions of CNN used in my experements. About second bug, the problem is in the error calculation. In the new version I'll fix these bugs.

Comment only
22 Nov 2009 Grigory

22.10.2009 - in current Ver_0.72
- in file train_cnn.m Conv. Layer 4 should have 16 kernels not 6 (sinet.CLayer{4}.numKernels = 6 ) or sinet.CLayer{4}.ConMap shouldn't be 6x16 in size.
- net training result doesn't return "minimal error" net state. I.e. I can see lover error during training than final is. However I apply CNN to different task with my own image set.

Comment only
30 Sep 2009 Nikolay Chumerin

Well done! Good job. I think this is the first publicly available implementation of CNN training in Matlab. The source code is written in a pretty good style with extensive comments, which are really useful for such complex classes. This submission is an asset for computer vision Matlab community.

28 May 2009 Mihail Sirotenko

Mihail Sirotenko (view profile)

  • 1 file
  • 592 downloads
  • 4.92308

New version coming soon. It will include:
1. Trained network for experements.
2. Simple GUI, visualising digits recognition.
3. Improoved performance.
4. Independence from NN Toolbox.

Comment only
Updates
09 Feb 2010 1.5

1. Support of CUDA-accelerated simulation and training of convolutional neural networks;
2. GUI added, providing RMSE, MCR plots;
3. Ability to choose from 3 training modes added.
See changelog.txt for more.

12 Feb 2010 1.6

Fixed bug with extra output error in non-cuda training example

11 Apr 2011 1.8

Compatibility with Matlab 2010 issue fixed (Thanks to Silvio Filipe)

26 Sep 2012 1.25

Matlab 2012a compatibility issues resolved. Dependency from NN toolbox removed.

15 Oct 2012 1.26

Description changed: added the notice about future versions of library.

24 Oct 2012 1.29

readMNIST.m script improved to explicitly check if MNIST library is exist and located in MNIST folder.

Contact us