MATLAB Answers^{™}

- 13 questions
- 0 answers
- 0 accepted answers
- Reputation: 1

Accepted Answer by Image Analyst

- 0 questions
- 16,955 answers
- 5,479 accepted answers
- Reputation: 27,534

Hi,

I have trying to extract the numbers from a calculator screen image. I am using my laptop webcam to detect the image and simultaneously display the result after image processing. Can anybody please help how to proceed??

- 0 questions
- 16,955 answers
- 5,479 accepted answers
- Reputation: 27,534

**Direct link to this answer:**

http://www.mathworks.com/matlabcentral/answers/68387#answer_79720

Answer by Image Analyst
on 24 Mar 2013

- 0 questions
- 16,955 answers
- 5,479 accepted answers
- Reputation: 27,534

Accepted answer

Very very easy. You don't even need to do sophisticated OCR. You merely need to sample the image at known locations for known numbers. Trivial for a 7 segment display. Just read the image at 7 locations. Let's say the locations are

1 2 3 4 5 6 7

Now a "1" would have 2 and 5 lit, and all the others unlit.

A "2" would have 1, 3, 4, 5, & 7 lit and the others unlit.

You can make up a 128 element look up table and use that to read off the number, given an input pattern. it's the same concept but a little more complicated if you have something with more resolution than a 7 segment display.

Show 20 older comments

Link

**Direct link to this comment:**

http://www.mathworks.com/matlabcentral/answers/68387#comment_138718

Link

**Direct link to this comment:**

http://www.mathworks.com/matlabcentral/answers/68387#comment_138743

Link

**Direct link to this comment:**

http://www.mathworks.com/matlabcentral/answers/68387#comment_139029

Link

**Direct link to this comment:**

http://www.mathworks.com/matlabcentral/answers/68387#comment_139031

Link

**Direct link to this comment:**

http://www.mathworks.com/matlabcentral/answers/68387#comment_139034

http://i50.tinypic.com/dpvms8.jpg

This is the image. How to proceed?

Link

**Direct link to this comment:**

http://www.mathworks.com/matlabcentral/answers/68387#comment_139049

First of all, improve your image by using more light. Next, make sure your calculator is in the same location every time. Then just use the same concepts I spelled out. There are certain locations that will be blue and certain locations that will be gray. These locations will be different for every number. I don't think you should have to check more than about 7-10 regions to uniquely identify every number. Is there something in the algorithm that is still confusing? Or do you understand the concept and algorithm, but you just don't know how to put it into MATLAB code? Here's a hint. To get the red, green, and blue value you can use mean2:

meanRed = mean2(rgbImage(row1:row2, col1:col2, 1)); meanGreen = mean2(rgbImage(row1:row2, col1:col2, 2)); meanBlue = mean2(rgbImage(row1:row2, col1:col2, 3)); tolerance = 10; % or whatever. zoneIsBlue(zoneNumber) = meanBlue > (meanRed + tolerance) & meanBlue > (meanGreen + tolerance)

Of course the rows and columns will have to refer to different locations for each number because they are not all in the same location. You could pre-identfy those regions with ginput(), imrect(), or rbbox(), which is easiest if your calculator is in the same location, as I recommended. If the calculator moves around and you, unfortunately, have no way at all to prevent that, then you can identify the blue region and try to find out how many numbers are showing by looking at the width of the blue region, then get your zones knowing that. You can look in my File Exchange for several color segmentation methods.

Link

**Direct link to this comment:**

http://www.mathworks.com/matlabcentral/answers/68387#comment_139071

Link

**Direct link to this comment:**

http://www.mathworks.com/matlabcentral/answers/68387#comment_139073

More light and a longer exposure time. And save it as PNG, not jpg. And of course a better camera and lens. Here's some code to get you started:

clc; % Clear the command window. close all; % Close all figures (except those of imtool.) imtool close all; % Close all imtool figures if you have the Image Processing Toolbox. clear; % Erase all existing variables. Or clearvars if you want. workspace; % Make sure the workspace panel is showing. format long g; format compact; fontSize = 20; % Read in a standard MATLAB color demo image. folder = 'C:\Users\Soumyadip\Documents\Temporary'; baseFileName = 'dpvms8.jpg'; % Get the full filename, with path prepended. fullFileName = fullfile(folder, baseFileName); if ~exist(fullFileName, 'file') % Didn't find it there. Check the search path for it. fullFileName = baseFileName; % No path this time. if ~exist(fullFileName, 'file') % Still didn't find it. Alert user. errorMessage = sprintf('Error: %s does not exist.', fullFileName); uiwait(warndlg(errorMessage)); return; end end rgbImage = imread(fullFileName); % Get the dimensions of the image. numberOfColorBands should be = 3. [rows columns numberOfColorBands] = size(rgbImage); % Display the original color image. subplot(2, 2, 1); imshow(rgbImage); axis on; title('Original Color Image', 'FontSize', fontSize); % Enlarge figure to full screen. set(gcf, 'units','normalized','outerposition',[0 0 1 1]);

% Crop the image. rgbImage = imcrop(rgbImage, [275, 320, 250 80]); % Display the image. subplot(2, 2, 2); imshow(rgbImage); axis on; title('Cropped Color Image', 'FontSize', fontSize);

% imwrite(rgbImage, 'calculator.png');

% Extract the individual red, green, and blue color channels. redChannel = rgbImage(:, :, 1); greenChannel = rgbImage(:, :, 2); blueChannel = rgbImage(:, :, 3);

tolerance = 7; % or whatever. bluePixels = blueChannel > (redChannel + tolerance) & blueChannel > (greenChannel + tolerance); % Display the image. subplot(2, 2, 3); imshow(bluePixels); axis on; title('Blue Pixels', 'FontSize', fontSize);

% meanRed = mean2(redChannel(row1:row2, col1:col2)); % meanGreen = mean2(greenChannel(row1:row2, col1:col2)); % meanBlue = mean2(blueChannel(row1:row2, col1:col2));

Link

**Direct link to this comment:**

http://www.mathworks.com/matlabcentral/answers/68387#comment_139116

Link

**Direct link to this comment:**

http://www.mathworks.com/matlabcentral/answers/68387#comment_139137

Link

**Direct link to this comment:**

http://www.mathworks.com/matlabcentral/answers/68387#comment_139747

Link

**Direct link to this comment:**

http://www.mathworks.com/matlabcentral/answers/68387#comment_140065

Here, I did it using color classification plus some standard image processing techniques to clean the image of spurious blobs so that only the numbers remain. I looked at the 3D color gamut and noticed that in LAB color space, the numbers had negative b values (bluish) while the background had positive b values (greenish/yellowish). Although it does an excellent job on the one image you provided, you'd still be better off getting a better camera to get less noise and using more light, which will also give less noise.

clc; % Clear the command window. close all; % Close all figures (except those of imtool.) imtool close all; % Close all imtool figures if you have the Image Processing Toolbox. clear; % Erase all existing variables. Or clearvars if you want. workspace; % Make sure the workspace panel is showing. format long g; format compact; fontSize = 20;

% Read in a standard MATLAB color demo image. folder = 'C:\Users\soumyadip\Documents\Temporary'; baseFileName = 'calculator.png'; % Get the full filename, with path prepended. fullFileName = fullfile(folder, baseFileName); if ~exist(fullFileName, 'file') % Didn't find it there. Check the search path for it. fullFileName = baseFileName; % No path this time. if ~exist(fullFileName, 'file') % Still didn't find it. Alert user. errorMessage = sprintf('Error: %s does not exist.', fullFileName); uiwait(warndlg(errorMessage)); return; end end rgbImage = imread(fullFileName); % Get the dimensions of the image. numberOfColorBands should be = 3. [rows, columns, numberOfColorBands] = size(rgbImage); % Display the original color image. subplot(2, 2, 1); imshow(rgbImage); title('Original Color Image', 'FontSize', fontSize); % Enlarge figure to full screen. set(gcf, 'units','normalized','outerposition',[0 0 1 1]);

% Convert to lab color space. cform = makecform('srgb2lab'); lab = applycform(im2double(rgbImage),cform); LImage = lab(:,:, 1); aImage = lab(:,:, 2); bImage = lab(:,:, 3); % Display them subplot(2,2,2); imshow(LImage, []); title('L Image', 'FontSize', fontSize); subplot(2,2,3); imshow(aImage, []); title('A Image', 'FontSize', fontSize); subplot(2,2,4); imshow(bImage); title('B Image', 'FontSize', fontSize);

figure; % Open a new figure. subplot(2, 2, 1); imshow(bImage, []); title('B Image', 'FontSize', fontSize); % Enlarge figure to full screen. set(gcf, 'units','normalized','outerposition',[0 0 1 1]);

% Binarize to get negative b (bluish pixels). originalBinaryImage = bImage < 0; subplot(2, 2, 2); imshow(originalBinaryImage, []); title('Binary Image', 'FontSize', fontSize);

% Clean up the image. binaryImage = bwareaopen(originalBinaryImage, 20); % Dilate to merge regions together. binaryImage = imdilate(binaryImage, true(9)); % Fill holes to make it more solid; binaryImage = imfill(binaryImage, 'holes'); subplot(2, 2, 3); imshow(binaryImage, []); title('Cleaned Binary Image', 'FontSize', fontSize);

% Find the rows verticalProfile = sum(binaryImage, 2); subplot(2, 2, 4); plot(verticalProfile, 'b-'); grid on; title('Vertical Profile', 'FontSize', fontSize); % Find the top and bottom rows - where the jumps are biggest. [~, topRow] = max(diff(verticalProfile)) [~, bottomRow] = max(-diff(verticalProfile)) % Crop image - erase above top row and below bottom row binaryImage(1:topRow,:) = false; binaryImage(bottomRow:end,:) = false;

% Find the columns horizontalProfile = sum(binaryImage, 1); figure subplot(2, 2, 1); plot(horizontalProfile, 'b-'); grid on; title('Horizontal Profile', 'FontSize', fontSize); % Enlarge figure to full screen. set(gcf, 'units','normalized','outerposition',[0 0 1 1]); % Find the top and bottom rows - where the jumps are biggest. [~, leftCol] = max(diff(horizontalProfile)) [~, rightCol] = max(-diff(horizontalProfile)) % Crop image - erase above top row and below bottom row binaryImage(:, 1:leftCol) = false; binaryImage(:, rightCol:end) = false; subplot(2, 2, 2); imshow(binaryImage, []); title('Cleaned Binary Image', 'FontSize', fontSize);

% Extract the largest blob % Find the areas. labeledImage = bwlabel(binaryImage); blobMeasurements = regionprops(labeledImage, 'area'); allAreas = [blobMeasurements.Area]; % Sort in descending order [sortedAreas, sortIndexes] = sort(allAreas, 'descend'); % Extract the blob with the largest area. maskImage = ismember(labeledImage, sortIndexes(1)) > 0; % Mask this against the first binary image. originalBinaryImage(~maskImage) = false; subplot(2, 2, 3); imshow(originalBinaryImage, []); title('Cleaned Binary Image', 'FontSize', fontSize);

Link

**Direct link to this comment:**

http://www.mathworks.com/matlabcentral/answers/68387#comment_140087

Thanks sir immensely for the code. The conversion to the Lab color space proved really useful. Although I have a few doubts:

1) While finding the columns where the jumps are biggest, the rightmost col did not give the desired value because there is a sharp decline in the horizontal profile at X = 129, so many of the digits got clipped. How to correct this?

2) If I wish to extract the digits directly from the bImage by cleaning up some noise, is it feasible?

Link

**Direct link to this comment:**

http://www.mathworks.com/matlabcentral/answers/68387#comment_140088

Link

**Direct link to this comment:**

http://www.mathworks.com/matlabcentral/answers/68387#comment_140107

Now I have a problem. I have to isolate only the calculator screen (as I attached in the link above) from the main image automatically every time. I tried that using bwboundaries but it is only thresholding those regions. How to extract a particular region??

Main image - <http://i46.tinypic.com/28cd4dl.jpg>

Link

**Direct link to this comment:**

http://www.mathworks.com/matlabcentral/answers/68387#comment_140122

Link

**Direct link to this comment:**

http://www.mathworks.com/matlabcentral/answers/68387#comment_140129

Sorry the pic is

Please suggest how to extract the screen only.

Link

**Direct link to this comment:**

http://www.mathworks.com/matlabcentral/answers/68387#comment_140148

I'd just threshold to find the LAB value of the screen.

Link

**Direct link to this comment:**

http://www.mathworks.com/matlabcentral/answers/68387#comment_140155

I didnt quite understand. How to find LAB value of the screen?

Link

**Direct link to this comment:**

http://www.mathworks.com/matlabcentral/answers/68387#comment_140157

Recall my code above:

% Convert to lab color space. cform = makecform('srgb2lab'); lab = applycform(im2double(rgbImage),cform); LImage = lab(:,:, 1); aImage = lab(:,:, 2); bImage = lab(:,:, 3);

Just threshold it properly. Pick a row and columns - say put "axis on" or something and look at the lab images to see about what lab value the screen has and threshold to select that.

Link

**Direct link to this comment:**

http://www.mathworks.com/matlabcentral/answers/68387#comment_140174

Link

**Direct link to this comment:**

http://www.mathworks.com/matlabcentral/answers/68387#comment_140188

**black** calculators **instead of white** calculators - see what I mean?

Link

**Direct link to this comment:**

http://www.mathworks.com/matlabcentral/answers/68387#comment_140202

okay...guess i am on my own..thanks

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi test

Learn moreOpportunities for recent engineering grads.

Apply Today
## 0 Comments