File Exchange

image thumbnail

CNN - Convolutional neural network class

version 1.29 (613 KB) by

This project provides matlab class for implementation of convolutional neural networks.

4.95455
23 Ratings

361 Downloads

Updated

View License

NOTE: Next versions of this library is hosted at https://bitbucket.org/intelligenceagent/cudacnn-public/wiki/Home
The reason is that now it's rather C++/CUDA library with Matlab front-end than a Matlab library.

This project provides matlab class for implementation of convolutional neural networks. This networks was developed by Yann LeCun and have sucessfully used in many practical applications, such as handwritten digits recognition, face detection, robot navigation and others (see references for more info). Because of some architectural features of convolutional networks, such as weight sharing it is imposible to implement it using Matlab Neural Network Toolbox without it's source modifications. That's why this class works almost independently from NN toolbox (coming soon full independence).

This release includes sample of handwritten digits recognition using CNN. If you just want to try it run cnet_tool. You'll see a simple GUI. It loads pretrained convolutional neural net from cnet.mat and recognizes image of digit either painted in painting area or downloaded from MNIST database.

Further information about the library can be found here:
https://sites.google.com/site/mihailsirotenko/projects/convolutional-neural-network-class
and here:
https://sites.google.com/site/mihailsirotenko/projects/cuda-cnn

Comments and Ratings (55)

Sheon yan

can i use it for pattern recognition ? and how to run it please

asiawu

asiawu (view profile)

Thanks.

deep blue

xs chen

shabaz khan

may I get the code for finding false positive and false negative?

IDA JENSONA

How can I implement solar radiation forecasting using CNN. Can anyone help me?

I wish to run cnn in matlab 2016a for training 40 images using trainnetwork from help documentation. But I get error
convnet = trainNetwork(trainDigitData,layers,options); % running the CNN model
Caused by:
    The CUDA driver could not be loaded. The library name used was 'nvcuda.dll'. The error was:
    The specified module could not be found.
I don't want to use cuda now. What to do?

how can i use this for speech classification?

Yan Li

Yan Li (view profile)

Yan Li

Yan Li (view profile)

Yifei Wu

wzy w

wzy w (view profile)

I apllied it to traffic sign recognition,it classified all classes to a same class,have you had any experience with this - if so what parameters might you suggest I change?How can

Shemmy

Shemmy (view profile)

Hi,
I want to change the last layer(output layer) to use linear regression, how to do this?

Heider

Heider (view profile)

Hello,
I need the structure of the CNN class, i.e. it's methods and functions, and it's parameters. I need information like the components: numSLayers, numInputs, Perf, num_dec which are found with cnet object in the form of cnet.numSLayers, cnet.numInputs, cnet.Pref ....etc

Endel

Endel (view profile)

I have a problem with understanding the actual architecture of this network. Usually, LeCun et al have used different weights for the connections from different feature maps of a previous layer (something that looks like 3D kernel). Therefore, the number of weights of a convolution layer (assuming full map of connections) is kernelHeight*kernelWidth*numFeatMapsLayer(k)*numFeatMapsLayer(k-1). I am not sure, but it seems that there are kernelHeight*kernelWidth*numFeatMapsLayer(k) different weights used in this program. Does it means that the connections from different feature maps of a previous layer to a particular feature map of the next layer have the same weights? Or maybe, I misunderstand something?

Manav

Manav (view profile)

I want to use this toolbox on my own sound data set. Each spectogram equivalent to 1 image has dimensions 13x500, where as each image in MNIST has dimensions 28x28. I want to change the input width and height to 13 and 500 respectively. How do I go about doing so?

pan junfu

I apllied it to traffic sign recognition,it classified all classes to a same class,have you had any experience with this - if so what parameters might you suggest I change?How can I train the cnn with the features extracted from the images instead of the images themselves?
Where should I change?

tc

tc (view profile)

Hello, I have run train_cnn on Octave, however, I got a error message "max_recursion_depth exceeded". Is anybody know/experience same problem? How to solve it? Thanks.

Johnathan

Mihail Sirotenko

Mihail Sirotenko (view profile)

  • 1 file
  • 361 downloads
  • 4.95455

In this pure matlab version there's no max pooling. But you can find it in C++/CUDA version, which is on bitbucket repository.

Johnathan

I've been searching around for a bit but I can't find the max pooling layer code. Can someone point me in the right direction?

Elham

Elham (view profile)

Hi, I am running the demo train-cnn which is provided to train the network from Dr. Lecun's paper. I do full training for over 20 epochs. The best MCR I get is .12 which corresponds to 12% error. How can I improve this? (The goal is to get .8% error)
Thanks

Mihail Sirotenko

Mihail Sirotenko (view profile)

  • 1 file
  • 361 downloads
  • 4.95455

2Eliud Gonzalez:
1. the output of CNN is saved to workspace variable sinet. You can use matlab function "save" to save it on disk.
2. Hessian computation controlled on line 147:
sinet.HcalcMode = 0;
Default value 0 means running estimate of hessian. In this case you'll not notice it in gui. If you change it to 1, gui progress bar will show the hessian recomputation.

Thanks, when training the CNN, how do i save it, or it saves itself? and why Hessian calculation is always 0?

Mihail Sirotenko

Mihail Sirotenko (view profile)

  • 1 file
  • 361 downloads
  • 4.95455

2 Eliud Gonzalez: usually this means that you don't have MNIST dataset. It is not distributed with the library, but you can download in from http://yann.lecun.com/exdb/mnist/
Please dowload and put the files into the MNIST folder.

I get this errors when running train_cnn.m
??? Error using ==> fread
Invalid file identifier. Use fopen to generate a valid file identifier.

Error in ==> readMNIST at 25
magicNum = fread(fid,1,'int32'); %Magic number

Error in ==> train_cnn at 30
[I,labels,I_test,labels_test] = readMNIST(1000);
 
>>

What can i do if i open train_cnn.m and click run, and nothing happens, do i need to install CUDA, i'm sorry i'm new in this world :(

Mihail Sirotenko

Mihail Sirotenko (view profile)

  • 1 file
  • 361 downloads
  • 4.95455

2 yin: yes, it can. Actually you can feed in any data not only images.
2 Xiaolei Hu: As far as I know there's no support of Conv Nets in latest Matlab NN Toolbox.

Xiaolei Hu

Does anybody know whether the latest Matlab has changed its NN toolbox so CNN can be implemented?

yin

yin (view profile)

can the cnn train with the features extracted from the images instead of the images themselves?

Mihail Sirotenko

Mihail Sirotenko (view profile)

  • 1 file
  • 361 downloads
  • 4.95455

Unfortunately I don't have Matlab 2011b. It seems that they changed something in NN toolbox. I'm going to remove the dependencies from NN toolbox in next release, which I'm trying to finalize now.

I try to train the network but I get error:
Undefined function or variable 'boiler_transfer'.
Error in tansig_mod (line 64)

I am using MATLAB R2011b. Any help?

Mihail Sirotenko

Mihail Sirotenko (view profile)

  • 1 file
  • 361 downloads
  • 4.95455

Full training cycle needs about 10 epochs on full training set with progressively decreasing theta. As far as I understand you did 1 epoch.

Bo Hu

Bo Hu (view profile)

This is a great job. However I have a question. I test this program using the MNIST handwritten digit database. The mcr rate is very high (about 15%) even I train the cnn using 10000 input. If I tried to train the cnn using 60000 input, then the program would took fairly long time, about several hours to finish. Anyway, the mcr is always about 15%. I wonder what parameter we should change to make the prediction accuracy higher. Thanks a lot.

Mihail Sirotenko

Mihail Sirotenko (view profile)

  • 1 file
  • 361 downloads
  • 4.95455

Regarding several last questions. I'm going to put new release of the lib which will contain example with faces and would be more convenient to use with various data. It already works in general. But I need a week or two more to finalize it since I do it in my free time.

chipo josé

hi
please can any one help as to adapt this code to face recognition
thanks

Zhipan Ren

Aron Sceidt

seeing it now... looks great :) is it possible to load a jpg or png in some way? what would be required?

Mihail Sirotenko

Mihail Sirotenko (view profile)

  • 1 file
  • 361 downloads
  • 4.95455

2Gaurav. It means that file train_cnn.m have comments for almost every line of code, so you can find parameter you want to change and actually change it before start training. Basically train_cnn.m contains settings for network architecture (number of layers, number of neurons etc)

Gaurav

Gaurav (view profile)

in the read me file to train the network "
If you interested in training you should open train_cnn.m, set all parameters following to comments and start learning by runing it."
i cant understand the "set all parameters following to comments and start learning by runing it." part

please help

Is it possible to train using color images? (from the example it says it is not). However, the version from Nikolay Chumerin can be trained using color patches.

Silvio Filipe

Sebastien PARIS

Mihail Sirotenko

Mihail Sirotenko (view profile)

  • 1 file
  • 361 downloads
  • 4.95455

Thanks!
Actually I had a plans to adapt this to face recognition, but unfortunately there're still only plans, because now I have absolutely no time to develop it.
So I'd advice to look closely on preprocessing algorithms and try to make architecture similar to face detection papers advices.

Mark

Mark (view profile)

Excellent work!

I've attempted to get this working on face recognition but without success. The MCR remains very high. Have you had any experience with this - if so what parameters might you suggest I change?

pupu QQ

it's ok ,but only numbers.high quality .

Mihail Sirotenko

Mihail Sirotenko (view profile)

  • 1 file
  • 361 downloads
  • 4.95455

Thanks a lot for the valuable comment. First bug is a result of merging different versions of CNN used in my experements. About second bug, the problem is in the error calculation. In the new version I'll fix these bugs.

Grigory

22.10.2009 - in current Ver_0.72
- in file train_cnn.m Conv. Layer 4 should have 16 kernels not 6 (sinet.CLayer{4}.numKernels = 6 ) or sinet.CLayer{4}.ConMap shouldn't be 6x16 in size.
- net training result doesn't return "minimal error" net state. I.e. I can see lover error during training than final is. However I apply CNN to different task with my own image set.

Nikolay Chumerin

Well done! Good job. I think this is the first publicly available implementation of CNN training in Matlab. The source code is written in a pretty good style with extensive comments, which are really useful for such complex classes. This submission is an asset for computer vision Matlab community.

Mihail Sirotenko

Mihail Sirotenko (view profile)

  • 1 file
  • 361 downloads
  • 4.95455

New version coming soon. It will include:
1. Trained network for experements.
2. Simple GUI, visualising digits recognition.
3. Improoved performance.
4. Independence from NN Toolbox.

Updates

1.29

readMNIST.m script improved to explicitly check if MNIST library is exist and located in MNIST folder.

1.26

Description changed: added the notice about future versions of library.

1.25

Matlab 2012a compatibility issues resolved. Dependency from NN toolbox removed.

1.8

Compatibility with Matlab 2010 issue fixed (Thanks to Silvio Filipe)

1.6

Fixed bug with extra output error in non-cuda training example

1.5

1. Support of CUDA-accelerated simulation and training of convolutional neural networks;
2. GUI added, providing RMSE, MCR plots;
3. Ability to choose from 3 training modes added.
See changelog.txt for more.

MATLAB Release
MATLAB 7.4 (R2007a)
Acknowledgements

Inspired: myCNN

Download apps, toolboxes, and other File Exchange content using Add-On Explorer in MATLAB.

» Watch video