- Home
- Search
- Contributors
- Recent Activity
- Flagged Content
- Log in to ask and answer questions.
- Ask a Question
- About MATLAB Answers

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn moreOpportunities for recent engineering grads.

Apply Today**New to MATLAB?**

Asked by Soumyadip
on 24 Mar 2013

Hi,

I have trying to extract the numbers from a calculator screen image. I am using my laptop webcam to detect the image and simultaneously display the result after image processing. Can anybody please help how to proceed??

Answer by Image Analyst
on 24 Mar 2013

Accepted answer

Very very easy. You don't even need to do sophisticated OCR. You merely need to sample the image at known locations for known numbers. Trivial for a 7 segment display. Just read the image at 7 locations. Let's say the locations are

1 2 3 4 5 6 7

Now a "1" would have 2 and 5 lit, and all the others unlit.

A "2" would have 1, 3, 4, 5, & 7 lit and the others unlit.

You can make up a 128 element look up table and use that to read off the number, given an input pattern. it's the same concept but a little more complicated if you have something with more resolution than a 7 segment display.

Show 20 older comments

Soumyadip
on 25 Mar 2013

Link

**Direct link to this comment:**

http://www.mathworks.com/matlabcentral/answers/68387#comment_138718

its fine but my calculator has more resolution than a 7 segment display. Can you suggest how to seperate this individual dots and identify the necessary pattern from the look up table?? Actually I am a total rookie in matlab. It would help if you provide some resources regarding this problem.

Image Analyst
on 25 Mar 2013

Link

**Direct link to this comment:**

http://www.mathworks.com/matlabcentral/answers/68387#comment_138743

It's the same concept. You may just need to have 7 "zones" where you use mean2() to get the mean value of the image in that location. It will either be bright or dark depending on if the number is "set" there. Where did you upload your image?

Soumyadip
on 26 Mar 2013

Link

**Direct link to this comment:**

http://www.mathworks.com/matlabcentral/answers/68387#comment_139029

Actually, my laptop has a VGA webcam where the quality of the pic is not good. Also, the display of the calculator is not at all like 7-segment. If you can please provide me your email id, I can mail you the images. Kindly help.

Image Analyst
on 26 Mar 2013

Link

**Direct link to this comment:**

http://www.mathworks.com/matlabcentral/answers/68387#comment_139031

Try tinypic.com or snag.gy/. You may have to process the image a bit so that you can tell when a segment is lit or not. After that, the concept is the same.

Soumyadip
on 26 Mar 2013

Link

**Direct link to this comment:**

http://www.mathworks.com/matlabcentral/answers/68387#comment_139034

http://i50.tinypic.com/dpvms8.jpg

This is the image. How to proceed?

Image Analyst
on 26 Mar 2013

Link

**Direct link to this comment:**

http://www.mathworks.com/matlabcentral/answers/68387#comment_139049

First of all, improve your image by using more light. Next, make sure your calculator is in the same location every time. Then just use the same concepts I spelled out. There are certain locations that will be blue and certain locations that will be gray. These locations will be different for every number. I don't think you should have to check more than about 7-10 regions to uniquely identify every number. Is there something in the algorithm that is still confusing? Or do you understand the concept and algorithm, but you just don't know how to put it into MATLAB code? Here's a hint. To get the red, green, and blue value you can use mean2:

meanRed = mean2(rgbImage(row1:row2, col1:col2, 1)); meanGreen = mean2(rgbImage(row1:row2, col1:col2, 2)); meanBlue = mean2(rgbImage(row1:row2, col1:col2, 3)); tolerance = 10; % or whatever. zoneIsBlue(zoneNumber) = meanBlue > (meanRed + tolerance) & meanBlue > (meanGreen + tolerance)

Of course the rows and columns will have to refer to different locations for each number because they are not all in the same location. You could pre-identfy those regions with ginput(), imrect(), or rbbox(), which is easiest if your calculator is in the same location, as I recommended. If the calculator moves around and you, unfortunately, have no way at all to prevent that, then you can identify the blue region and try to find out how many numbers are showing by looking at the width of the blue region, then get your zones knowing that. You can look in my File Exchange for several color segmentation methods.

Soumyadip
on 26 Mar 2013

Link

**Direct link to this comment:**

http://www.mathworks.com/matlabcentral/answers/68387#comment_139071

I tried getting the blue part of the RGB image above but the numbers are not clearly displaying there..Do you mean that the numbers will be the blue part where as the background will be the grey part? Also, can I improve the image so that processing becomes better?

Image Analyst
on 26 Mar 2013

Link

**Direct link to this comment:**

http://www.mathworks.com/matlabcentral/answers/68387#comment_139073

More light and a longer exposure time. And save it as PNG, not jpg. And of course a better camera and lens. Here's some code to get you started:

clc; % Clear the command window. close all; % Close all figures (except those of imtool.) imtool close all; % Close all imtool figures if you have the Image Processing Toolbox. clear; % Erase all existing variables. Or clearvars if you want. workspace; % Make sure the workspace panel is showing. format long g; format compact; fontSize = 20; % Read in a standard MATLAB color demo image. folder = 'C:\Users\Soumyadip\Documents\Temporary'; baseFileName = 'dpvms8.jpg'; % Get the full filename, with path prepended. fullFileName = fullfile(folder, baseFileName); if ~exist(fullFileName, 'file') % Didn't find it there. Check the search path for it. fullFileName = baseFileName; % No path this time. if ~exist(fullFileName, 'file') % Still didn't find it. Alert user. errorMessage = sprintf('Error: %s does not exist.', fullFileName); uiwait(warndlg(errorMessage)); return; end end rgbImage = imread(fullFileName); % Get the dimensions of the image. numberOfColorBands should be = 3. [rows columns numberOfColorBands] = size(rgbImage); % Display the original color image. subplot(2, 2, 1); imshow(rgbImage); axis on; title('Original Color Image', 'FontSize', fontSize); % Enlarge figure to full screen. set(gcf, 'units','normalized','outerposition',[0 0 1 1]);

% Crop the image. rgbImage = imcrop(rgbImage, [275, 320, 250 80]); % Display the image. subplot(2, 2, 2); imshow(rgbImage); axis on; title('Cropped Color Image', 'FontSize', fontSize);

% imwrite(rgbImage, 'calculator.png');

% Extract the individual red, green, and blue color channels. redChannel = rgbImage(:, :, 1); greenChannel = rgbImage(:, :, 2); blueChannel = rgbImage(:, :, 3);

tolerance = 7; % or whatever. bluePixels = blueChannel > (redChannel + tolerance) & blueChannel > (greenChannel + tolerance); % Display the image. subplot(2, 2, 3); imshow(bluePixels); axis on; title('Blue Pixels', 'FontSize', fontSize);

% meanRed = mean2(redChannel(row1:row2, col1:col2)); % meanGreen = mean2(greenChannel(row1:row2, col1:col2)); % meanBlue = mean2(blueChannel(row1:row2, col1:col2));

Soumyadip
on 26 Mar 2013

Link

**Direct link to this comment:**

http://www.mathworks.com/matlabcentral/answers/68387#comment_139116

Thanks immensely for the code. However, I think it would not be possible to keep the calculator always at EXACTLY the same position though I will try my level best. Then I count the number of digits based on width of blue pixels. But how to segement them ? Can you please explain what we are doing by the meanRed, etc variables?

Image Analyst
on 26 Mar 2013

Link

**Direct link to this comment:**

http://www.mathworks.com/matlabcentral/answers/68387#comment_139137

Well I just took you up as far as getting an binary image that says whether the pixels are blue or not. You still have to examine that image in each location of each digit to determine what the digit is. That's what that code is for. You need to determine what all the row and columns are for the 7 locations and the N digits.

Soumyadip
on 29 Mar 2013

Link

**Direct link to this comment:**

http://www.mathworks.com/matlabcentral/answers/68387#comment_139747

Hi...I thought of doing it in a different way using pattern recognition. But first I intend to convert the image into a binary one where only the digits will be black and the rest 0. How to do this? Any image enhancement because normal thresholding is not working?? The link to the cropped image - http://i49.tinypic.com/33y25u0.png

Image Analyst
on 31 Mar 2013

Link

**Direct link to this comment:**

http://www.mathworks.com/matlabcentral/answers/68387#comment_140065

Here, I did it using color classification plus some standard image processing techniques to clean the image of spurious blobs so that only the numbers remain. I looked at the 3D color gamut and noticed that in LAB color space, the numbers had negative b values (bluish) while the background had positive b values (greenish/yellowish). Although it does an excellent job on the one image you provided, you'd still be better off getting a better camera to get less noise and using more light, which will also give less noise.

clc; % Clear the command window. close all; % Close all figures (except those of imtool.) imtool close all; % Close all imtool figures if you have the Image Processing Toolbox. clear; % Erase all existing variables. Or clearvars if you want. workspace; % Make sure the workspace panel is showing. format long g; format compact; fontSize = 20;

% Read in a standard MATLAB color demo image. folder = 'C:\Users\soumyadip\Documents\Temporary'; baseFileName = 'calculator.png'; % Get the full filename, with path prepended. fullFileName = fullfile(folder, baseFileName); if ~exist(fullFileName, 'file') % Didn't find it there. Check the search path for it. fullFileName = baseFileName; % No path this time. if ~exist(fullFileName, 'file') % Still didn't find it. Alert user. errorMessage = sprintf('Error: %s does not exist.', fullFileName); uiwait(warndlg(errorMessage)); return; end end rgbImage = imread(fullFileName); % Get the dimensions of the image. numberOfColorBands should be = 3. [rows, columns, numberOfColorBands] = size(rgbImage); % Display the original color image. subplot(2, 2, 1); imshow(rgbImage); title('Original Color Image', 'FontSize', fontSize); % Enlarge figure to full screen. set(gcf, 'units','normalized','outerposition',[0 0 1 1]);

% Convert to lab color space. cform = makecform('srgb2lab'); lab = applycform(im2double(rgbImage),cform); LImage = lab(:,:, 1); aImage = lab(:,:, 2); bImage = lab(:,:, 3); % Display them subplot(2,2,2); imshow(LImage, []); title('L Image', 'FontSize', fontSize); subplot(2,2,3); imshow(aImage, []); title('A Image', 'FontSize', fontSize); subplot(2,2,4); imshow(bImage); title('B Image', 'FontSize', fontSize);

figure; % Open a new figure. subplot(2, 2, 1); imshow(bImage, []); title('B Image', 'FontSize', fontSize); % Enlarge figure to full screen. set(gcf, 'units','normalized','outerposition',[0 0 1 1]);

% Binarize to get negative b (bluish pixels). originalBinaryImage = bImage < 0; subplot(2, 2, 2); imshow(originalBinaryImage, []); title('Binary Image', 'FontSize', fontSize);

% Clean up the image. binaryImage = bwareaopen(originalBinaryImage, 20); % Dilate to merge regions together. binaryImage = imdilate(binaryImage, true(9)); % Fill holes to make it more solid; binaryImage = imfill(binaryImage, 'holes'); subplot(2, 2, 3); imshow(binaryImage, []); title('Cleaned Binary Image', 'FontSize', fontSize);

% Find the rows verticalProfile = sum(binaryImage, 2); subplot(2, 2, 4); plot(verticalProfile, 'b-'); grid on; title('Vertical Profile', 'FontSize', fontSize); % Find the top and bottom rows - where the jumps are biggest. [~, topRow] = max(diff(verticalProfile)) [~, bottomRow] = max(-diff(verticalProfile)) % Crop image - erase above top row and below bottom row binaryImage(1:topRow,:) = false; binaryImage(bottomRow:end,:) = false;

% Find the columns horizontalProfile = sum(binaryImage, 1); figure subplot(2, 2, 1); plot(horizontalProfile, 'b-'); grid on; title('Horizontal Profile', 'FontSize', fontSize); % Enlarge figure to full screen. set(gcf, 'units','normalized','outerposition',[0 0 1 1]); % Find the top and bottom rows - where the jumps are biggest. [~, leftCol] = max(diff(horizontalProfile)) [~, rightCol] = max(-diff(horizontalProfile)) % Crop image - erase above top row and below bottom row binaryImage(:, 1:leftCol) = false; binaryImage(:, rightCol:end) = false; subplot(2, 2, 2); imshow(binaryImage, []); title('Cleaned Binary Image', 'FontSize', fontSize);

% Extract the largest blob % Find the areas. labeledImage = bwlabel(binaryImage); blobMeasurements = regionprops(labeledImage, 'area'); allAreas = [blobMeasurements.Area]; % Sort in descending order [sortedAreas, sortIndexes] = sort(allAreas, 'descend'); % Extract the blob with the largest area. maskImage = ismember(labeledImage, sortIndexes(1)) > 0; % Mask this against the first binary image. originalBinaryImage(~maskImage) = false; subplot(2, 2, 3); imshow(originalBinaryImage, []); title('Cleaned Binary Image', 'FontSize', fontSize);

Soumyadip
on 31 Mar 2013

Link

**Direct link to this comment:**

http://www.mathworks.com/matlabcentral/answers/68387#comment_140087

Thanks sir immensely for the code. The conversion to the Lab color space proved really useful. Although I have a few doubts:

1) While finding the columns where the jumps are biggest, the rightmost col did not give the desired value because there is a sharp decline in the horizontal profile at X = 129, so many of the digits got clipped. How to correct this?

2) If I wish to extract the digits directly from the bImage by cleaning up some noise, is it feasible?

Soumyadip
on 31 Mar 2013

Link

**Direct link to this comment:**

http://www.mathworks.com/matlabcentral/answers/68387#comment_140088

I solved the first problem sir.. I increased the structuring element in the imdilate function from true(9) to true(11). It works now.

Soumyadip
on 31 Mar 2013

Link

**Direct link to this comment:**

http://www.mathworks.com/matlabcentral/answers/68387#comment_140107

Now I have a problem. I have to isolate only the calculator screen (as I attached in the link above) from the main image automatically every time. I tried that using bwboundaries but it is only thresholding those regions. How to extract a particular region??

Main image - <http://i46.tinypic.com/28cd4dl.jpg>

Image Analyst
on 31 Mar 2013

Link

**Direct link to this comment:**

http://www.mathworks.com/matlabcentral/answers/68387#comment_140122

The link does not work. Plus I don't know what you mean about bwboundaries() and thresholding. You may have to tweak the algorithm as you encounter new and different images. In fact there are some images where it will fail completely, like where there are no digits at all or where the noise level is too great, etc.

Soumyadip
on 31 Mar 2013

Link

**Direct link to this comment:**

http://www.mathworks.com/matlabcentral/answers/68387#comment_140129

Sorry the pic is

Please suggest how to extract the screen only.

Image Analyst
on 31 Mar 2013

Link

**Direct link to this comment:**

http://www.mathworks.com/matlabcentral/answers/68387#comment_140148

I'd just threshold to find the LAB value of the screen.

Soumyadip
on 31 Mar 2013

Link

**Direct link to this comment:**

http://www.mathworks.com/matlabcentral/answers/68387#comment_140155

I didnt quite understand. How to find LAB value of the screen?

Image Analyst
on 31 Mar 2013

Link

**Direct link to this comment:**

http://www.mathworks.com/matlabcentral/answers/68387#comment_140157

Recall my code above:

% Convert to lab color space. cform = makecform('srgb2lab'); lab = applycform(im2double(rgbImage),cform); LImage = lab(:,:, 1); aImage = lab(:,:, 2); bImage = lab(:,:, 3);

Just threshold it properly. Pick a row and columns - say put "axis on" or something and look at the lab images to see about what lab value the screen has and threshold to select that.

Soumyadip
on 31 Mar 2013

Link

**Direct link to this comment:**

http://www.mathworks.com/matlabcentral/answers/68387#comment_140174

I tried thresholding the b image but the screen is not getting isolated at all. Actually the screen is not having any distinct property and many corners are also getting thresholded. Can you please provide me a little more code?

Image Analyst
on 31 Mar 2013

Link

**Direct link to this comment:**

http://www.mathworks.com/matlabcentral/answers/68387#comment_140188

Sorry but I think I've provided you enough to go on. If I do this, then there will be another problem with another weird image, and I can't go on to do the whole project with you. And I don't have time to delve deeply into any one particular project. I think you have the general concepts enough for you to continue on your own and take pride in ownership of your code. If lab thresholding won't work, try thresholding on RGB and look for large black squares. Of course this won't work the minute you start to look at **black** calculators **instead of white** calculators - see what I mean?

Soumyadip
on 31 Mar 2013

Link

**Direct link to this comment:**

http://www.mathworks.com/matlabcentral/answers/68387#comment_140202

okay...guess i am on my own..thanks

- © 1994-2015 The MathWorks, Inc.
- Site Help
- Patents
- Trademarks
- Privacy Policy
- Preventing Piracy
- Terms of Use
- RSS
- Google+

- Featured MathWorks.com Topics:
- New Products
- Support
- Documentation
- Training
- Webinars
- Newsletters
- MATLAB Trials
- Careers

## 0 Comments