Hey everyone, I'm doing a virtual mouse project by controlling the mouse pointer on the computer screen using a webcam to track movement of my hand n some mouse clicking event. I have to learn matlab from scratch n I've tried doing as possible as I can but still can't produce the right codes or thru simulink. Can anyone really help me on this bcuz I'm kinda desperate right now n I'm really don't know what I'm suppose to do next. If anyone can give me the codes or simulink models, that would be really helpful, thank u..
No products are associated with this question.
Controlling the mouse pointer is actually a bit difficult, as you need to use the Java Robot class (it's not a Matlab function per se). I'm not even sure how you would access this via Simulink.
MATLAB can move the mouse around:
x = 500; y = 600;
But, I think you do need the Java Robot to make a click.
hye all, ive found this one m file code from the matlab central file exchange, this code is to track red object in real time image acquisition, my question is, how do u connect the tracking bounding box to the mouse input from the java robot library?? the following are the codes
% Capture the video frames using the videoinput function % You have to replace the resolution & your installed adaptor name.
vid = videoinput('winvideo',1,'RGB24_320x240');
% Set the properties of the video object
set(vid, 'FramesPerTrigger', Inf); set(vid, 'ReturnedColorspace', 'rgb') vid.FrameGrabInterval = 5;
%start the video aquisition here
% Set a loop that stop after 100 frames of aquisition
% Get the snapshot of the current frame data = getsnapshot(vid);
% Now to track red objects in real time % we have to subtract the red component % from the grayscale image to extract the red components in the image. diff_im = imsubtract(data(:,:,1), rgb2gray(data)); %Use a median filter to filter out noise diff_im = medfilt2(diff_im, [3 3]); % Convert the resulting grayscale image into a binary image. diff_im = im2bw(diff_im,0.18);
% Remove all those pixels less than 300px diff_im = bwareaopen(diff_im,300);
% Label all the connected components in the image. bw = bwlabel(diff_im, 8);
% Here we do the image blob analysis. % We get a set of properties for each labeled region. stats = regionprops(bw, 'BoundingBox', 'Centroid');
% Display the image imshow(data)
%This is a loop to bound the red objects in a rectangular box. for object = 1:length(stats) bb = stats(object).BoundingBox; bc = stats(object).Centroid; rectangle('Position',bb,'EdgeColor','r','LineWidth',2) plot(bc(1),bc(2), '-m+') a=text(bc(1)+15,bc(2), strcat('X: ', num2str(round(bc(1))), ' Y: ', num2str(round(bc(2))))); set(a, 'FontName', 'Arial', 'FontWeight', 'bold', 'FontSize', 12, 'Color', 'yellow'); end
hold off end
put the value of red object's centroid in the [x y] mat. and set(0,'PointerLocation',[x y]); and the pointer moves according to centroid may be it work good luck!
hye all, i would like to say thank u so much for all your help n suggestions, finally, my code works by using the red tracking code n connect it to the java robot, thanx again all, if i encounter further problems, ill post it here, :D
Hye all, from the code above, I've successfully execute it with the java robot command, I hv a question, can anyone explain to me how can a mouse clicking event being execute when the program detects 2 tracking red object, the code can detect 2 red object, my plan is the first red object is to move the mouse pointer, but when it detects the second red object, a mouse clicking event is execure
@syahrul fikri : i used your code and its working... the mouse pointer moves and also clicks, but the problems i faced are..... 1) i dont know how the clicking operation is being executed.... does it executes when pointer is stable on a folder for some specified time or when another color is detected by the system?...... 2) if i move my hand to right , the pointer moves left and vice versa..... how to reverse that....... 3)if the camera resolution is less than the resolution of the screen, whole screen is not covered with this system.... how to eliminate this?..... 4)if i want to detect two different colors and execute clicking operations by manipulating the distance between the colors, how can i do that..?? please help....
Hey i tried to check this interesting thing: i used java robot.was able to move pointer and click as well. But there is this thing : when i right clicked a block in simulink(ofcourse by java robot), menu drop down appeared as normal; in the next lines of codes, i meant to position cursor on one of the menu item so i tried to reposition and use right click to select the menu item. But by the time these lines were to execute, the menu was not visible and hence the menu item cannot be selected
i think the menu items disappears when next lines of codes starts execution because the script window activates.
Do you know how to avoid this disappearance of the menu.