Code covered by the BSD License  

Highlights from
CNN - Convolutional neural network class

4.92308

4.9 | 13 ratings Rate this file 394 Downloads (last 30 days) File Size: 613 KB File ID: #24291
image thumbnail

CNN - Convolutional neural network class

by

 

28 May 2009 (Updated )

This project provides matlab class for implementation of convolutional neural networks.

| Watch this File

File Information
Description

NOTE: Next versions of this library is hosted at https://bitbucket.org/intelligenceagent/cudacnn-public/wiki/Home
The reason is that now it's rather C++/CUDA library with Matlab front-end than a Matlab library.

This project provides matlab class for implementation of convolutional neural networks. This networks was developed by Yann LeCun and have sucessfully used in many practical applications, such as handwritten digits recognition, face detection, robot navigation and others (see references for more info). Because of some architectural features of convolutional networks, such as weight sharing it is imposible to implement it using Matlab Neural Network Toolbox without it's source modifications. That's why this class works almost independently from NN toolbox (coming soon full independence).

This release includes sample of handwritten digits recognition using CNN. If you just want to try it run cnet_tool. You'll see a simple GUI. It loads pretrained convolutional neural net from cnet.mat and recognizes image of digit either painted in painting area or downloaded from MNIST database.

Further information about the library can be found here:
https://sites.google.com/site/mihailsirotenko/projects/convolutional-neural-network-class
and here:
https://sites.google.com/site/mihailsirotenko/projects/cuda-cnn

Acknowledgements

This file inspired My Cnn.

Required Products MATLAB
MATLAB release MATLAB 7.4 (R2007a)
Tags for This File   Please login to tag files.
Please login to add a comment or rating.
Comments and Ratings (39)
31 Jul 2014 NetDragon wanglong

Hi,
I want to change the last layer(output layer) to use linear regression, how to do this?

07 May 2014 Heider

Hello,
I need the structure of the CNN class, i.e. it's methods and functions, and it's parameters. I need information like the components: numSLayers, numInputs, Perf, num_dec which are found with cnet object in the form of cnet.numSLayers, cnet.numInputs, cnet.Pref ....etc

27 Jan 2014 Endel

I have a problem with understanding the actual architecture of this network. Usually, LeCun et al have used different weights for the connections from different feature maps of a previous layer (something that looks like 3D kernel). Therefore, the number of weights of a convolution layer (assuming full map of connections) is kernelHeight*kernelWidth*numFeatMapsLayer(k)*numFeatMapsLayer(k-1). I am not sure, but it seems that there are kernelHeight*kernelWidth*numFeatMapsLayer(k) different weights used in this program. Does it means that the connections from different feature maps of a previous layer to a particular feature map of the next layer have the same weights? Or maybe, I misunderstand something?

11 Jul 2013 Manav

I want to use this toolbox on my own sound data set. Each spectogram equivalent to 1 image has dimensions 13x500, where as each image in MNIST has dimensions 28x28. I want to change the input width and height to 13 and 500 respectively. How do I go about doing so?

24 May 2013 pan junfu

I apllied it to traffic sign recognition,it classified all classes to a same class,have you had any experience with this - if so what parameters might you suggest I change?How can I train the cnn with the features extracted from the images instead of the images themselves?
Where should I change?

29 Apr 2013 tc

Hello, I have run train_cnn on Octave, however, I got a error message "max_recursion_depth exceeded". Is anybody know/experience same problem? How to solve it? Thanks.

12 Feb 2013 Johnathan  
07 Feb 2013 Mihail Sirotenko

In this pure matlab version there's no max pooling. But you can find it in C++/CUDA version, which is on bitbucket repository.

06 Feb 2013 Johnathan

I've been searching around for a bit but I can't find the max pooling layer code. Can someone point me in the right direction?

29 Oct 2012 Elham

Hi, I am running the demo train-cnn which is provided to train the network from Dr. Lecun's paper. I do full training for over 20 epochs. The best MCR I get is .12 which corresponds to 12% error. How can I improve this? (The goal is to get .8% error)
Thanks

23 Oct 2012 Mihail Sirotenko

2Eliud Gonzalez:
1. the output of CNN is saved to workspace variable sinet. You can use matlab function "save" to save it on disk.
2. Hessian computation controlled on line 147:
sinet.HcalcMode = 0;
Default value 0 means running estimate of hessian. In this case you'll not notice it in gui. If you change it to 1, gui progress bar will show the hessian recomputation.

22 Oct 2012 Eliud Gonzalez

Thanks, when training the CNN, how do i save it, or it saves itself? and why Hessian calculation is always 0?

16 Oct 2012 Mihail Sirotenko

2 Eliud Gonzalez: usually this means that you don't have MNIST dataset. It is not distributed with the library, but you can download in from http://yann.lecun.com/exdb/mnist/
Please dowload and put the files into the MNIST folder.

16 Oct 2012 Eliud Gonzalez

I get this errors when running train_cnn.m
??? Error using ==> fread
Invalid file identifier. Use fopen to generate a valid file identifier.

Error in ==> readMNIST at 25
magicNum = fread(fid,1,'int32'); %Magic number

Error in ==> train_cnn at 30
[I,labels,I_test,labels_test] = readMNIST(1000);

>>

16 Oct 2012 Eliud Gonzalez

What can i do if i open train_cnn.m and click run, and nothing happens, do i need to install CUDA, i'm sorry i'm new in this world :(

15 Oct 2012 Mihail Sirotenko

2 yin: yes, it can. Actually you can feed in any data not only images.
2 Xiaolei Hu: As far as I know there's no support of Conv Nets in latest Matlab NN Toolbox.

12 Oct 2012 Xiaolei Hu

Does anybody know whether the latest Matlab has changed its NN toolbox so CNN can be implemented?

27 Aug 2012 yin

can the cnn train with the features extracted from the images instead of the images themselves?

16 Mar 2012 Mihail Sirotenko

Unfortunately I don't have Matlab 2011b. It seems that they changed something in NN toolbox. I'm going to remove the dependencies from NN toolbox in next release, which I'm trying to finalize now.

02 Mar 2012 Atanasko Atanasov

I try to train the network but I get error:
Undefined function or variable 'boiler_transfer'.
Error in tansig_mod (line 64)

I am using MATLAB R2011b. Any help?

06 Feb 2012 Mihail Sirotenko

Full training cycle needs about 10 epochs on full training set with progressively decreasing theta. As far as I understand you did 1 epoch.

28 Jan 2012 Bo Hu

This is a great job. However I have a question. I test this program using the MNIST handwritten digit database. The mcr rate is very high (about 15%) even I train the cnn using 10000 input. If I tried to train the cnn using 60000 input, then the program would took fairly long time, about several hours to finish. Anyway, the mcr is always about 15%. I wonder what parameter we should change to make the prediction accuracy higher. Thanks a lot.

12 Jan 2012 Mihail Sirotenko

Regarding several last questions. I'm going to put new release of the lib which will contain example with faces and would be more convenient to use with various data. It already works in general. But I need a week or two more to finalize it since I do it in my free time.

11 Jan 2012 chipo josé

hi
please can any one help as to adapt this code to face recognition
thanks

14 Nov 2011 Zhipan Ren  
13 Sep 2011 Aron Sceidt

seeing it now... looks great :) is it possible to load a jpg or png in some way? what would be required?

03 Aug 2011 Mihail Sirotenko

2Gaurav. It means that file train_cnn.m have comments for almost every line of code, so you can find parameter you want to change and actually change it before start training. Basically train_cnn.m contains settings for network architecture (number of layers, number of neurons etc)

02 Aug 2011 Gaurav

in the read me file to train the network "
If you interested in training you should open train_cnn.m, set all parameters following to comments and start learning by runing it."
i cant understand the "set all parameters following to comments and start learning by runing it." part

please help

10 Jun 2011 Jose M. Alvarez

Is it possible to train using color images? (from the example it says it is not). However, the version from Nikolay Chumerin can be trained using color patches.

11 Apr 2011 Silvio Filipe  
19 Jan 2011 Sebastien PARIS  
15 Oct 2010 Mihail Sirotenko

Thanks!
Actually I had a plans to adapt this to face recognition, but unfortunately there're still only plans, because now I have absolutely no time to develop it.
So I'd advice to look closely on preprocessing algorithms and try to make architecture similar to face detection papers advices.

15 Oct 2010 Mark

Excellent work!

I've attempted to get this working on face recognition but without success. The MCR remains very high. Have you had any experience with this - if so what parameters might you suggest I change?

29 Aug 2010 pupu QQ

it's ok ,but only numbers.high quality .

25 Jun 2010 Jonathan Masci  
23 Nov 2009 Mihail Sirotenko

Thanks a lot for the valuable comment. First bug is a result of merging different versions of CNN used in my experements. About second bug, the problem is in the error calculation. In the new version I'll fix these bugs.

22 Nov 2009 Grigory

22.10.2009 - in current Ver_0.72
- in file train_cnn.m Conv. Layer 4 should have 16 kernels not 6 (sinet.CLayer{4}.numKernels = 6 ) or sinet.CLayer{4}.ConMap shouldn't be 6x16 in size.
- net training result doesn't return "minimal error" net state. I.e. I can see lover error during training than final is. However I apply CNN to different task with my own image set.

30 Sep 2009 Nikolay Chumerin

Well done! Good job. I think this is the first publicly available implementation of CNN training in Matlab. The source code is written in a pretty good style with extensive comments, which are really useful for such complex classes. This submission is an asset for computer vision Matlab community.

28 May 2009 Mihail Sirotenko

New version coming soon. It will include:
1. Trained network for experements.
2. Simple GUI, visualising digits recognition.
3. Improoved performance.
4. Independence from NN Toolbox.

Updates
09 Feb 2010

1. Support of CUDA-accelerated simulation and training of convolutional neural networks;
2. GUI added, providing RMSE, MCR plots;
3. Ability to choose from 3 training modes added.
See changelog.txt for more.

12 Feb 2010

Fixed bug with extra output error in non-cuda training example

11 Apr 2011

Compatibility with Matlab 2010 issue fixed (Thanks to Silvio Filipe)

26 Sep 2012

Matlab 2012a compatibility issues resolved. Dependency from NN toolbox removed.

15 Oct 2012

Description changed: added the notice about future versions of library.

24 Oct 2012

readMNIST.m script improved to explicitly check if MNIST library is exist and located in MNIST folder.

Contact us