Thread Subject:

Problem in detecting two colors simultaenously

Hi there,
I have to make a robot which can follow colored balls mainly red and blue. I have designed a code for tracking the red ball but if another ball of blue color comes, then what else should I add in the code, so that it tracks that ball as well . I am using an overhead camera.
Please reply ASAP.
Thank you !

On 22/12/10 5:18 AM, Animesh Pandey wrote:

> I have to make a robot which can follow colored balls mainly red and
> blue. I have designed a code for tracking the red ball but if another
> ball of blue color comes, then what else should I add in the code, so
> that it tracks that ball as well . I am using an overhead camera.

We can't know what you need to _add_ to the code as you have not shown
us what you have now.

%This is the code !!!!

clear;
clc
vid=videoinput('winvideo',5,'YUY2_160x120');
set(vid,'FramesPerTrigger',1);
preview(vid);
pause(10);
yuy_image = getsnapshot(vid);
rgb_image = ycbcr2rgb(yuy_image);
[a b c]= size(rgb_image);
y=a;
x=b;

x1=x/2-120;
x2=x/2+120;
y1=y/2-30;
y2=y/2+30;

while (1)

yuy_image = getsnapshot(vid);
rgb_image = ycbcr2rgb(yuy_image);
flushdata(vid);
rbar=0;
cbar=0;
e=0;
fR=rgb_image(:,:,1);
fG=rgb_image(:,:,2);
fB=rgb_image(:,:,3);

I=((fR>=200) & (fG<=100) & (fB<=100));

se=strel('rectangle',[3 4]);
%se=strel('disk',10);
B=imopen(I,se);
%figure,imshow(B);
final=imclose(B,se);
pause(0.01);
imshow(final);
[L,n]=bwlabel(final);
%imshow(rgb_image); %////THIS IS TO BE USED ONLY WHILE TESTING
%hold on % ////THIS IS TO BE USED ONLY WHILE TESTING

for k=1:n
[r,c]=find(L==1);
rbar=mean(r);
cbar=mean(c);
%plot(cbar,rbar,'Marker','*','MarkerEdgeColor','B' ,'MarkerSize',20) %////THIS IS TO BE USED ONLY WHILE TESTING...
e=( ( (cbar>=x1)*2*2*2 ) + ( ( cbar<=x2 )*2*2 ) + ( ( rbar>=y1 )*2 ) + ( rbar<=y2 ) ); % Converting to decimal number

end

switch (e)
case 5
disp('Move left');
case 6
disp('Move left');
case 7
disp('Move left');
case 9
disp('Move right');
case 10
disp('Move right');
case 11
disp('Move right');
case 13
disp('Move forward');
case 14
disp('Move back');
otherwise
disp('Stop Moving');
end

end

%The switch case part will be added with serial communication later.
%This code is exclusively for RED color and I wish to make it rum for BLUE color also.

You need to add additional detections for other colors, so add more of
these lines:

redBinaryImage = ((fR>=200) & (fG<=100) & (fB<=100));

blueBinaryImage = ((fR<=100) & (fG<=100) & (fB>=200));

Put in one such line for every color you want to detect. Of course
you will have to tweak the values for each color.

Alternatively you could try it in HSV color space, like in my demo:
http://www.mathworks.com/matlabcentral/fileexchange/authors/31862

ImageAnalyst <imageanalyst@mailinator.com> wrote in message <c53be32e-bc1e-4002-9947-9e9d616f7aa8@l8g2000yqh.googlegroups.com>...
> You need to add additional detections for other colors, so add more of
> these lines:
>
> redBinaryImage = ((fR>=200) & (fG<=100) & (fB<=100));
>
> blueBinaryImage = ((fR<=100) & (fG<=100) & (fB>=200));
>
> Put in one such line for every color you want to detect. Of course
> you will have to tweak the values for each color.
>
> Alternatively you could try it in HSV color space, like in my demo:
> http://www.mathworks.com/matlabcentral/fileexchange/authors/31862
>

Sir..thanks a lot....
But I was wondering that how will I judge that the color is blue or red.
This robot has to collect red and blue balls in separate boxes.

I will explain the situation.......
Here, we have a robot, that has to collect red and blue balls and put them into box of their respective colors. This bot has to avoid black balls.
How will I make these three decisions at the same time.....

Please help !

It seems so obvious that I'm not sure what you're asking. You have
two images
 redBinaryImage = ((fR>=200) & (fG<=100) & (fB<=100));
 blueBinaryImage = ((fR<=100) & (fG<=100) & (fB>=200));
Then you label each one, as you've already done. If n>=1 then there
is an object of that color present. But you *already* know this
because you *already* have code for doing that. And you don't make
these three decisions at the same time (unless you have the parallel
computing toolbox) - you have two or three images and you examine each
one in turn by itself one at a time. Of course it will be so fast
that it may appear to be happening all "at the same time."

So, since you already know all that, what additional information do
you want to know? Maybe you want to do noise filtering to get rid of
objects smaller than a certain size, or take just the single largest
object - I don't know since you haven't said. Take a look at my
BlobsDemo for examples on how to filter the objects based on size or
other measurements.

ImageAnalyst <imageanalyst@mailinator.com> wrote in message <52f4b287-0b2e-409c-b453-0af5f6863daa@l32g2000yqc.googlegroups.com>...
> It seems so obvious that I'm not sure what you're asking. You have
> two images
> redBinaryImage = ((fR>=200) & (fG<=100) & (fB<=100));
> blueBinaryImage = ((fR<=100) & (fG<=100) & (fB>=200));
> Then you label each one, as you've already done. If n>=1 then there
> is an object of that color present. But you *already* know this
> because you *already* have code for doing that. And you don't make
> these three decisions at the same time (unless you have the parallel
> computing toolbox) - you have two or three images and you examine each
> one in turn by itself one at a time. Of course it will be so fast
> that it may appear to be happening all "at the same time."
>
> So, since you already know all that, what additional information do
> you want to know? Maybe you want to do noise filtering to get rid of
> objects smaller than a certain size, or take just the single largest
> object - I don't know since you haven't said. Take a look at my
> BlobsDemo for examples on how to filter the objects based on size or
> other measurements.

In addition to removing small stuff, if your objects are whole, (such as Steve's m&m example on his blog right now) you may want to do an imfill(im,'holes')...

To detect darker shades of color, you can do something along these lines (example is blue):
blueMask = rgb(:,:,3)>(rgb(:,:,1)+20)&rgb(:,:,3)>(rgb(:,:,2)+20);
%blue channel greater than red and green by 20

ImageAnalyst <imageanalyst@mailinator.com> wrote in message <52f4b287-0b2e-409c-b453-0af5f6863daa@l32g2000yqc.googlegroups.com>...
> It seems so obvious that I'm not sure what you're asking. You have
> two images
> redBinaryImage = ((fR>=200) & (fG<=100) & (fB<=100));
> blueBinaryImage = ((fR<=100) & (fG<=100) & (fB>=200));
> Then you label each one, as you've already done. If n>=1 then there
> is an object of that color present. But you *already* know this
> because you *already* have code for doing that. And you don't make
> these three decisions at the same time (unless you have the parallel
> computing toolbox) - you have two or three images and you examine each
> one in turn by itself one at a time. Of course it will be so fast
> that it may appear to be happening all "at the same time."
>
> So, since you already know all that, what additional information do
> you want to know? Maybe you want to do noise filtering to get rid of
> objects smaller than a certain size, or take just the single largest
> object - I don't know since you haven't said. Take a look at my
> BlobsDemo for examples on how to filter the objects based on size or
> other measurements.

Sir, this is the changed code.
--------------------------------------------------------------------------------------------------------
clear;
clc
vid=videoinput('winvideo',5,'YUY2_160x120');
set(vid,'FramesPerTrigger',1);
preview(vid);
pause(10);
yuy_image = getsnapshot(vid);
rgb_image = ycbcr2rgb(yuy_image);
[a b c]= size(rgb_image);
y=a;
x=b;

x1=x/2-120;
x2=x/2+120;
y1=y/2-30;
y2=y/2+30;

while (1)

yuy_image = getsnapshot(vid);
rgb_image = ycbcr2rgb(yuy_image);
flushdata(vid);
rbar=0;
cbar=0;
rbar1=0;
cbar1=0;
rbar2=0;
cbar2=0;
e=0;
fR=rgb_image(:,:,1);
fG=rgb_image(:,:,2);
fB=rgb_image(:,:,3);

I = (((fR>=200) & (fG<=100) & (fB<=100)) | ((fR>=200) & (fG>=200) & (fB<=100)) | ((fR<=100) & (fG<=120) & (fB>=200)));

se=strel('rectangle',[3 4]);
%se=strel('disk',10);
B=imopen(I,se);
%figure,imshow(B);
final=imclose(B,se);
pause(0.01);
imshow(final);
[L,n]=bwlabel(final);
%imshow(rgb_image); %////THIS IS TO BE USED ONLY WHILE TESTING
%hold on % ////THIS IS TO BE USED ONLY WHILE TESTING

%for k=1:n
[r,c]=find(L==1);
[r1,c1]=find(L==2);
[r2,c2]=find(L==3);

rbar=mean(r); %Here the centers of the detected images will found.
cbar=mean(c);
rbar1=mean(r1);
cbar1=mean(c1);
rbar2=mean(r2);
cbar2=mean(c2);

%plot(cbar,rbar,cbar1,rbar1,cbar2,rbar2,'Marker','*','MarkerEdgeColor','B' ,'MarkerSize',20) %////THIS IS TO BE USED ONLY WHILE TESTING

e=( ( (cbar>=x1)*2*2*2 ) + ( ( cbar<=x2 )*2*2 ) + ( ( rbar>=y1 )*2 ) + ( rbar<=y2 ) ); % Converting to decimal number

%end

switch (e)
case 5
disp('Move left');
case 6
disp('Move left');
case 7
disp('Move left');
case 9
disp('Move right');
case 10
disp('Move right');
case 11
disp('Move right');
case 13
disp('Move forward');
case 14
disp('Move back');
otherwise
disp('Stop Moving');
end

end
---------------------------------------------------------------------------------------------------------

In this code the variable 'I' stores the binary images for Yellow, Blue and Red. The last part (switch-case) isn't complete yet.

Sir, I wanted to know if there is any other better way of finding center of the White figures in the binary images.

It's the exact same concept. Just make up a binary image for white,
like
whiteBinary = (fR>=200) & (fG>=200) & (fB>=200)

Tweak numbers as needed.

ImageAnalyst <imageanalyst@mailinator.com> wrote in message <cd1893d3-bc55-4c28-a7b9-ee4be6a0978a@c2g2000yqc.googlegroups.com>...
> It's the exact same concept. Just make up a binary image for white,
> like
> whiteBinary = (fR>=200) & (fG>=200) & (fB>=200)
>
> Tweak numbers as needed.

Right sir........
The robot which I am making is supposed to collect balls of colors blues or red. How will I judge that which colored ball the robot is holding ? Because the binary image that I have captured contains three colors namely yellow, red and blue.
I wish to do something like ...

if (collected ball = red)
   execute Algorithm 1
else if ( collected ball = blue)
   execute Algorithm 2
else execute Algorithm 3

The third color will be 'Black', the robot has to evade that colored ball.
What should I do ?

You've defined your color ranges. Then you label and regionprops the
binary images so now you have centroids of every colored blob. Then
you need to know where in the field of view your robot hand is. The
ball that the hand is holding will be the one with the centroid that
is closest to the hand.

ImageAnalyst <imageanalyst@mailinator.com> wrote in message <6fed125b-5f6b-4177-b8c1-21eee41ed591@i18g2000yqn.googlegroups.com>...
> You've defined your color ranges. Then you label and regionprops the
> binary images so now you have centroids of every colored blob. Then
> you need to know where in the field of view your robot hand is. The
> ball that the hand is holding will be the one with the centroid that
> is closest to the hand.

Sir, basically I am using an overhead camera. There the robot is of yellow color.
If I take a hypothetical situation like there is a ball rolling down the ramp, and behind it there is another ball rolling down. The centroid of the first ball is closer so it will move towards that ball.
How am I going to judge the color of the ball while it is rolling down the ramp ?
The reason is that the statement :
I = (((fR>=200) & (fG<=100) & (fB<=100)) | ((fR>=200) & (fG>=200) & (fB<=100)) | ((fR<=100) & (fG<=120) & (fB>=200)));
gives the binary image detecting the Yellow, red and blue parts of the image. But still the robot won't know which colored ball it is near to !!! I mean how will the bot make sure that whether red or blue ball is rolling down ??? 'I' is not helping in doing so !!!!

That was your code. I have no idea why you did it. I assumed you
wanted an image where you had all the colored balls in it for some
reason, as opposed to images with just one color in them which I
originally gave you. If you want individual colors, go back to what I
said and get just the images for individual colors. Then use bwlabel
and regionprops to find the ball locations.

Post your image somewhere.

ImageAnalyst <imageanalyst@mailinator.com> wrote in message <4507634c-2a8c-42af-972e-2e537c38703b@o14g2000yqe.googlegroups.com>...
> That was your code. I have no idea why you did it. I assumed you
> wanted an image where you had all the colored balls in it for some
> reason, as opposed to images with just one color in them which I
> originally gave you. If you want individual colors, go back to what I
> said and get just the images for individual colors. Then use bwlabel
> and regionprops to find the ball locations.
>
> Post your image somewhere.

Sir, I searched through the internet and checked out a few tutorial videos of regionprops.
I have used it in my code because you had said to use it and I found it useful.

How do I post any image here ?

If you don't mind can I mail the code and the image to you Email ID mentioned above i.e 'imageanalyst@mailinator.com' ?

Animesh Pandey wrote:
...

> How do I post any image here ?
...

Can't; c.s-s.m is a usenet newsgroup (and not a binary group at that).

Use one of the many free hosting sites instead.

--

On Dec 25, 10:10 am, "Animesh Pandey" <apanimesh...@gmail.com> wrote:
> How do I post any image here ?
>
> If you don't mind can I mail the code and the image to you Email ID mentioned above i.e 'imageanal...@mailinator.com' ?
-------------------------------------------------------------------

Try this:
http://ww.lmgtfy.com/?q=free+image+hosting+websites

And I don't take email there. That's just a spamcatcher web site
where I can't even reply from. I never even look at it. I reply
here . . . where everyone can benefit.

ImageAnalyst <imageanalyst@mailinator.com> wrote in message <502c4a76-6967-4533-9953-f6a21b4c44cb@n10g2000yqd.googlegroups.com>...
> On Dec 25, 10:10 am, "Animesh Pandey" <apanimesh...@gmail.com> wrote:
> > How do I post any image here ?
> >
> > If you don't mind can I mail the code and the image to you Email ID mentioned above i.e 'imageanal...@mailinator.com' ?
> -------------------------------------------------------------------
>
> Try this:
> http://ww.lmgtfy.com/?q=free+image+hosting+websites
>
> And I don't take email there. That's just a spamcatcher web site
> where I can't even reply from. I never even look at it. I reply
> here . . . where everyone can benefit.

Right sir......I have uploaded my images on:
http://usera.imagecave.com/apanimesh061

I have changed the code......

clear;
clc
vid=videoinput('winvideo',5,'YUY2_160x120');
set(vid,'FramesPerTrigger',1);
preview(vid);
pause(10);
yuy_image = getsnapshot(vid);
rgb_image = ycbcr2rgb(yuy_image);
[a b c]= size(rgb_image);
y=a;
x=b;

x1=x/2-120;
x2=x/2+120;
y1=y/2-30;
y2=y/2+30;

while (1)

yuy_image = getsnapshot(vid);
rgb_image = ycbcr2rgb(yuy_image);
flushdata(vid);
e=0;
fR=rgb_image(:,:,1);
fG=rgb_image(:,:,2);
fB=rgb_image(:,:,3);

I = (((fR>=200) & (fG<=100) & (fB<=100)) | ((fR>=200) & (fG>=200) & (fB<=100)) | ((fR<=100) & (fG<=120) & (fB>=200)));


se=strel('rectangle',[3 4]);
B=imopen(I,se);
final=imclose(B,se);
pause(0.01);
imshow(final);
[L,n]=bwlabel(final);
prop=regionprops(L);

hold on % ////THIS IS TO BE USED ONLY WHILE TESTING

for k=1:length(prop)
rectangle('Position', prop(n).BoundingBox, 'EdgeColor', 'g', 'LineWidth', 2)
x=prop(n).Centroid(1);
y=prop(n).Centroid(2);
plot(x,y,'*')
end

hold off;

switch (e)
case 5
disp('Move left');
case 6
disp('Move left');
case 7
disp('Move left');
case 9
disp('Move right');
case 10
disp('Move right');
case 11
disp('Move right');
case 13
disp('Move forward');
case 14
disp('Move back');
otherwise
disp('Stop Moving');
end

end

I have got three images:
Objects.jpg-
This is the rgb image of the balls and the robot.
Yellow : robot
Red/Blue : Balls

BW.jpg-
This is the binary image of Objects.jpg i.e result of
I = (((fR>=200) & (fG<=100) & (fB<=100)) | ((fR>=200) & (fG>=200) & (fB<=100)) | ((fR<=100) & (fG<=120) & (fB>=200)));

Plot.jpg-
This is the result of the 'for' loop.

The 'for' loop is not giving required result. I was trying to put the props in a rectangle and try to find their center. But it only shows rectangle around the red figure and also I am not able to show the centers of the figures.

The centers are important for the movement of the robot.

And in the image 'Objects', the blue ball is near to the robot. What changes should I make in the code so that the robot knows that it should know that blue color is near to it ?

You're going to have to either

1. run regionprops on every color channel and then look at the
centroids, or

2 you're going to have to also ask regionprops for the centroid and
get the color from the 3 color channels (if you want to sample the
color just at one pixel point and want to run regionprops on the
combined binary image only) to determine which color channel that
object primarily lives in. Also ask for the centroids of course.

Sir,
I would like to know how does this works,
e=(((cbar>=x1)*2*2*2) + ((cbar<=x2)*2*2) + ((rbar>=y1)*2) + (rbar<=y2)) % Converting to decimal number
how it convert to decimal?I'm planning to do up and down besides left and right and i'm goona ignored foward and backward.any suggestion sir?

On Mar 11, 9:33 pm, "donny jason" <pooler_pool...@yahoo.com> wrote:
> Sir,
> I would like to know how does this works,
> e=(((cbar>=x1)*2*2*2) + ((cbar<=x2)*2*2) + ((rbar>=y1)*2) + (rbar<=y2)) % Converting to decimal number
> how it convert to decimal?I'm planning to do up and down besides left and right and i'm goona ignored foward and backward.any suggestion sir?

----------------------------------------------------------------------------------
donny jason:
I have no idea what you're trying to do, other than it seems to be
building up a decimal number from binary bits. I don't know what x1,
x2, y1, y2, cbar, and rbar are. In fact I don't even know if you
replied to the correct message since it doesn't seem related to this
thread at all. Why don't you start your own thread and give much much
more context about what you're asking?
ImageAnalyst

ImageAnalyst <imageanalyst@mailinator.com> wrote in message <eefcc42a-dc71-40f1-8921-d34d7fd3c25a@x13g2000vbe.googlegroups.com>...
> On Mar 11, 9:33 pm, "donny jason" <pooler_pool...@yahoo.com> wrote:
> > Sir,
> > I would like to know how does this works,
> > e=(((cbar>=x1)*2*2*2) + ((cbar<=x2)*2*2) + ((rbar>=y1)*2) + (rbar<=y2)) % Converting to decimal number
> > how it convert to decimal?I'm planning to do up and down besides left and right and i'm goona ignored foward and backward.any suggestion sir?
>
> ----------------------------------------------------------------------------------
> donny jason:
> I have no idea what you're trying to do, other than it seems to be
> building up a decimal number from binary bits. I don't know what x1,
> x2, y1, y2, cbar, and rbar are. In fact I don't even know if you
> replied to the correct message since it doesn't seem related to this
> thread at all. Why don't you start your own thread and give much much
> more context about what you're asking?
> ImageAnalyst

ok sir..question is basically referring to the program in this thread. I don't understand how it can convert to decimal by doing the equation that i have just stated.

ImageAnalyst <imageanalyst@mailinator.com> wrote in message <eefcc42a-dc71-40f1-8921-d34d7fd3c25a@x13g2000vbe.googlegroups.com>...
> On Mar 11, 9:33 pm, "donny jason" <pooler_pool...@yahoo.com> wrote:
> > Sir,
> > I would like to know how does this works,
> > e=(((cbar>=x1)*2*2*2) + ((cbar<=x2)*2*2) + ((rbar>=y1)*2) + (rbar<=y2)) % Converting to decimal number
> > how it convert to decimal?I'm planning to do up and down besides left and right and i'm goona ignored foward and backward.any suggestion sir?
>
> ----------------------------------------------------------------------------------
> donny jason:
> I have no idea what you're trying to do, other than it seems to be
> building up a decimal number from binary bits. I don't know what x1,
> x2, y1, y2, cbar, and rbar are. In fact I don't even know if you
> replied to the correct message since it doesn't seem related to this
> thread at all. Why don't you start your own thread and give much much
> more context about what you're asking?
> ImageAnalyst

ok sir..question is basically referring to the program in this thread. I don't understand how it can convert to decimal by doing the equation that i have just stated.

On Mar 11, 11:34 pm, "donny jason" <pooler_pool...@yahoo.com> wrote:
> ok sir..question is basically referring to the program in this thread. I don't understand how it can convert to decimal by doing the equation that i have just stated.
-----------------------------------------------------------------
Sorry my answer still stands. And I don't see how your code has
anything to do with Animesh's colored ball tracking image processing
problem. I don't even know what the "it" you're referring to is. I
guess I'll let someone else help you. Good luck anyway.