Virtual Touch Screen

Webcam based virtual touchscreen/keyboard

Virtual TouchScreen

TouchScreen has revolutionized the mobile & tablet market.
With touch-screen user can just touch then screen and buttons are pressed. There are no physical buttons. As per the requirement various hotspots/buttons for different size can be place on touch-screen.

Next big change in technology can be wireless interface to mobile or computer.
Virtual Touchscreen uses camera for implementing buttons. Hence buttons can be anywhere within range of camera view. Button can be on table, on blackboard, on walls, on roof, on hand, on refrigerator everywhere.

While working on our final year project Robot Dog using webcam, we were able to follow a colored ball.
Suddenly an idea strike in my mind that we can divide then image acquired from webcam into 2*2 matrix. Each matrix represents a virtual button.
In order to point a particular button we used a red LED in mouse.
We can used any colored Led for the same or even we can design our own ring or handheld device for pressing virtual button.
Basic principle behind our concept is detecting the redObject in images i.e. fining its X-Y coordinates and labeling them.
We call know image consist of 3layers of red, green and blue color.
Here we consider than Red object to be recognized have color with green and blue components less than 50 and red component greater than 150 on image with 255levels. By manipulating the threshold levels, other color objects can also be determined.
First of all we Remove all those pixels less than 300px.Then we label all the connected components in the image. After that we do the image blob analysis. We get X and Y coordinated of labeled components.
We place a marker over the object and display its X-Y coordinates.

We divide then image into 4parts i.e. 2*2matrix
Depending upon the X-Y coordinated of detected redObject we change then brightness of that particular area indicating button is pressed.

Further enhancement to this project can be adding various sophisticated techniques so that false detection can be avoided, multiple detection , depth detection, motion and gesture recognition is possible.
A sample matlab code for basic virtual touchscreen is provided.