Image Analysis Help for Eye blink detection

I am working on a eye blink detection project. By this moment I have been successfully extracted the images of pupil of eye from the real time video.

But I am stuck at the process to differentiate the open eye pupil to the closed eye lid state.

1. Open eye pupil

2. closed eye lid

Could you please suggest me any image processing function or algorithm to differentiate between the two states.

(Attached images are the original images on which image processing is to be done.)

[Vikas]

actually i want to get a value which can be used to decide the state of eyes.

i am trying to do it by

newIm2= im2bw(eyeImage,0.4);
B = bwboundaries(newIm2,8,'noholes');
L= length(B);

after this comparing "L"... as it would be different for eye open and closed state.

but the above algorithm is not working. please help

[ramkumar r]

Hello VIkas,

Can you share the code of eye detection and pupil ext6raction plz.

Answer 1

[Dishant Arora]

You can have a plot of average horizontal intensities.In case eye is closed the distance between two deep valleys would increase as eyelashes would move far from the brows:

I1 = imread('openEyeImage');
I1 = rgb2gray(I1)
plot(1:size(I1,1) , mean(I1,2));
hold on

 I2 = imread('closeEyeImage');
I2 = rgb2gray(I2)
plot(1:size(I2,1) , mean(I2,2) , 'r');

Check out the difference between two plots.

Or you can go from blob detection, find circular objects. Call regionprops check out for ecentricity. This would be useful: Blobs Detection

[Vikas]

I actually want to get a value for different states, so that i can proceed further to detect the drowsy state of driver if it detects continuously 4 times closed eyelids.

could you tell me how to get a specific value for the two states? As in the plot method I am unable to compare the values.

[Vikas]

i am trying to do it by

newIm2= im2bw(eyeImage,0.4);
B = bwboundaries(newIm2,8,'noholes');
L= length(B);

after this comparing "L"... as it would be different for eye open and closed state.

but the above algorithm is not working. please help

[Dishant Arora]

As you can see from the plot distance between first two deep valleys(local minima) would be more for a closed image. You can set a threshold distance to check out weather image is closed. You can locate minima using findpeaks

 horizontalAverages = mean(I1 , 2);
plot(1:size(I1,1) , horizontalAverages)
[Minimas locs] = findpeaks(-horizontalAverages) 
hold on
plot(locs , -1*Minimas , 'r*')
if (locs(2)-locs(1))< threshold  %Threshold distance 
    % Eye is open
else
    % Eye is closed.

And I dont think bwboudaries will do,as L will number of objects and that might vary due to imperfections in imaging.

[Vikas]

i am getting following output

1)Eye open state

----------plot for various open eye images

2)Eye Closed state

----------plot for various closed eye images

[Dishant Arora]

You can see the pattern mimimas are located far when eye is closed. You can choose a threshold based on these patterns.

[Izzati]

Greetings, Mr. Dishant. May I know what is the name of this method? I would like to read a few papers regarding this method since I'm doing a research on eye blink detection. Thank you.

[Image Analyst]

I think it's usually called "drowsiness detection". See my Answer for a link to published papers on it.

Answer 2

[Image Analyst]

Perhaps you'd be interested in how others do it: http://iris.usc.edu/Vision-Notes/bibliography/motion-f725.html#Driver%20Monitoring,%20Eyes,%20Gaze

[Vikas]

can you suggest any matlab image processing algorithm or function to be applied on the actual images which are generated/extracted through the program.

1. Open eye pupil

2. closed eye lid

[Image Analyst]

I'm sure several of the papers suggest algorithms. I'm sure what I would suggest for your one subject (such as looking at the mean intensity or the average radial profile) might work okay for that one pair of images but mgiht fail in other cases, like for very very dark skinned individuals. That's why I suggested published algorithms where presumably they've considered those things and made their algorithm very robust.

[Vikas]

I'm unable to view the IEEE or Science direct papers as i have not subscribed for their membership.

It would be great if you can suggest for above images as it would give me an approach on how to deal with it, would be able to try that on different subjects.

[Image Analyst]

If you're at a University, the papers will be available for free most likely. If you work for a company they will spend less money to order papers than paying you to flounder for months trying different, unproven approaches.

I did give two approaches (radial profile or mean). Try to program those up.

Answer 3

[kishan nagesh]

please can anyone send me the code for detecting the pupil of the eye from segmented image of the face . I am currently working on a project for detecting driver fatigue i am not able to get an idea as to how to get to know state of eye ie open or closed after edge detection of face . Please help me out people .

Answer 4

[rohith h.c]

iam doing the project on visual pattern recognition based on color,morphological statistical and textural features i have the algorithm and i need the matlab code since iam new to matlab. please mail the code to [e-mail] 1.take the rgb input image of a fruit 2.convert it to gray scale 3.remove all noises 4.if background is light,make it dark 5.perform background subtraction and image segmentation using otsu's method.

[Image Analyst]

This has little or nothing to do with eye blink detection. Why don't you start your own thread? Or search for tag:fruit since lots of people here want to do fruit analysis and there are lots of threads on it.

Answer 5

[rohith h.c]

i have a piece of code can anyone explain me the code?

clc clear all close all sand = imread('fruit_256.BMP'); [x,y]=size(sand); I3 = medfilt2(sand,[3 3]); I3 = imadjust(I3); bw1 = im2bw(I3, 0.9); bw = bwareaopen(bw1, 10); bw0= 1 - bw; I2 = bwfill(bw0,'holes'); se=strel('square',3); SI1=imerode(I2,se); se1=strel('square',10); I1=imdilate(SI1,se1); [labeled,numObjects] = bwlabel(I1,4);

RGB_label=label2rgb(labeled,@spring,'c','shuffle'); chrdata = regionprops(labeled,'all') allchrs = [chrdata.Area]; num = size(allchrs); % BW1=edge(labeled,'sobel', 0.11); yuan = [chrdata.Eccentricity]; [line,row]=size(yuan); long = [chrdata.MajorAxisLength]; width = [chrdata.MinorAxisLength]; nbins = 20;

figure(1); imshow(sand);title('Original') figure(2); imshow(I3);title('medfilt') figure(3); imshow(bw);title('0-1') figure(4); imshow(I2);title('bwfill') figure(5); imshow(I1);title('imdilate') figure(6); imshow(RGB_label);title('RGB') figure(7),hist(allchrs,nbins); title(num(2)) figure(8),hist(yuan,nbins); title(num(2))

[Dishant Arora]

Create a new thread, it has nothing to do with this.

Answer 6

[rohith h.c]

pls can u explain each line of the below code with comments?

clc clear all close all sand = imread('fruit_256.BMP'); [x,y]=size(sand); I3 = medfilt2(sand,[3 3]); I3 = imadjust(I3); bw1 = im2bw(I3, 0.9); bw = bwareaopen(bw1, 10); bw0= 1 - bw; I2 = bwfill(bw0,'holes'); se=strel('square',3); SI1=imerode(I2,se); se1=strel('square',10); I1=imdilate(SI1,se1); [labeled,numObjects] = bwlabel(I1,4);

RGB_label=label2rgb(labeled,@spring,'c','shuffle'); chrdata = regionprops(labeled,'all') allchrs = [chrdata.Area]; num = size(allchrs); % BW1=edge(labeled,'sobel', 0.11); yuan = [chrdata.Eccentricity]; [line,row]=size(yuan); long = [chrdata.MajorAxisLength]; width = [chrdata.MinorAxisLength]; nbins = 20;

figure(1); imshow(sand);title('Original') figure(2); imshow(I3);title('medfilt') figure(3); imshow(bw);title('0-1') figure(4); imshow(I2);title('bwfill') figure(5); imshow(I1);title('imdilate') figure(6); imshow(RGB_label);title('RGB') figure(7),hist(allchrs,nbins); title(num(2)) figure(8),hist(yuan,nbins); title(num(2))

[Image Analyst]

No. Read the responses to the prior two times you asked the same thing.

Answer 7

[Waheed Ullah]

After reading an image (jpeg, png, etc.), then I want to detect eyes and find the distance between human eyes. So if anyone have such a code then please inbox me (<mailto:[e-mail] [e-mail]>) or give here. Thanks.

[Shahrokh Firouzi]

There is a thing which call creativity. I think instead of asking codes from others, use the sample codes above and find your way. For sure there are difficulties to find out the others' coding but in that moment you figure out the way of writing codes, you can do whatever that you want.