CNN - Convolutional neural network class

This project provides matlab class for implementation of convolutional neural networks.

Description

This project provides matlab class for implementation of convolutional neural networks. This networks was developed by Yann LeCun and have sucessfully used in many practical applications, such as handwritten digits recognition, face detection, robot navigation and others (see references for more info). Because of some architectural features of convolutional networks, such as weight sharing it is imposible to implement it using Matlab Neural Network Toolbox without it's source modifications. That's why this class works almost independently from NN toolbox (coming soon full independence).

This release includes sample of handwritten digits recognition using CNN. If you just want to try it run cnet_tool. You'll see a simple GUI. It loads pretrained convolutional neural net from cnet.mat and recognizes image of digit either painted in painting area or downloaded from MNIST database.

The significant improovement in this version is support of nVidia CUDA technology, which speeds up the training up to 44 times. You'll need a CUDA-capable graphic card and CUDA SDK (especially cudart.dll and cublas.dll). Currently only stochastic gradient is supported by CUDA-training, but Hessian approximation is going to be soon also.
IMPORTANT:since matlabcentral is not allows to include mex-files into submission, you need to download cudacnn.mex (mex32 or mex64) from https://sites.google.com/site/mihailsirotenko/projects/convolutional-neural-network-class for full functionality.
Though without cudacnn.mex this software is also functional in usual way.
To run CUDA-based training use cutrain_cnn.m
Note that ther're some problems with CUDA v3.
The source of cudacnn.mex is not included by now, but I plan to do It in future.
See readme.txt for more details.
Changes in version 0.81:
- Compatibility with Matlab 2010 issue fixed (Thanks to Silvio Filipe)

• Acknowledgements: This submission has inspired the following:; myCNN
• Required Products: Neural Network Toolbox
• MATLAB release: MATLAB 7.4 (R2007a)
• Other requirements: You'll need a CUDA driver, toolkit and to download mex-files from external link (see description) in order to enable CUDA features.

[Mihail Sirotenko]

New version coming soon. It will include:
1. Trained network for experements.
2. Simple GUI, visualising digits recognition.
3. Improoved performance.
4. Independence from NN Toolbox.

[Nikolay Chumerin]

Well done! Good job. I think this is the first publicly available implementation of CNN training in Matlab. The source code is written in a pretty good style with extensive comments, which are really useful for such complex classes. This submission is an asset for computer vision Matlab community.

[Grigory]

22.10.2009 - in current Ver_0.72
- in file train_cnn.m Conv. Layer 4 should have 16 kernels not 6 (sinet.CLayer{4}.numKernels = 6 ) or sinet.CLayer{4}.ConMap shouldn't be 6x16 in size.
- net training result doesn't return "minimal error" net state. I.e. I can see lover error during training than final is. However I apply CNN to different task with my own image set.

[Mihail Sirotenko]

Thanks a lot for the valuable comment. First bug is a result of merging different versions of CNN used in my experements. About second bug, the problem is in the error calculation. In the new version I'll fix these bugs.

[Jonathan Masci]

 

[pupu QQ]

it's ok ,but only numbers.high quality .

[Mark]

Excellent work!

I've attempted to get this working on face recognition but without success. The MCR remains very high. Have you had any experience with this - if so what parameters might you suggest I change?

[Mihail Sirotenko]

Thanks!
Actually I had a plans to adapt this to face recognition, but unfortunately there're still only plans, because now I have absolutely no time to develop it.
So I'd advice to look closely on preprocessing algorithms and try to make architecture similar to face detection papers advices.

[Sebastien Paris]

 

[Silvio Filipe]

 

[Jose M. Alvarez]

Is it possible to train using color images? (from the example it says it is not). However, the version from Nikolay Chumerin can be trained using color patches.

[Gaurav]

in the read me file to train the network "
If you interested in training you should open train_cnn.m, set all parameters following to comments and start learning by runing it."
i cant understand the "set all parameters following to comments and start learning by runing it." part

please help

[Mihail Sirotenko]

2Gaurav. It means that file train_cnn.m have comments for almost every line of code, so you can find parameter you want to change and actually change it before start training. Basically train_cnn.m contains settings for network architecture (number of layers, number of neurons etc)

[Aron Sceidt]

seeing it now... looks great :) is it possible to load a jpg or png in some way? what would be required?

[Zhipan Ren]

 

[chipo josé]

hi
please can any one help as to adapt this code to face recognition
thanks

[Mihail Sirotenko]

Regarding several last questions. I'm going to put new release of the lib which will contain example with faces and would be more convenient to use with various data. It already works in general. But I need a week or two more to finalize it since I do it in my free time.

[Bo Hu]

This is a great job. However I have a question. I test this program using the MNIST handwritten digit database. The mcr rate is very high (about 15%) even I train the cnn using 10000 input. If I tried to train the cnn using 60000 input, then the program would took fairly long time, about several hours to finish. Anyway, the mcr is always about 15%. I wonder what parameter we should change to make the prediction accuracy higher. Thanks a lot.

[Mihail Sirotenko]

Full training cycle needs about 10 epochs on full training set with progressively decreasing theta. As far as I understand you did 1 epoch.