There are other integrations of Kinect and robots, but this is done exclusively in Matlab for the new LEGO EV3 robot. It was developed in Matlab 2014a and requires the Image Acquisition Toolbox. It might run on older Matlab versions but probably Kinect support cannot be installed on Matlab versions older than 2013a. For controlling the Kinect, one uses the Kinect for Windows Sensor Support for Matlab Image Acquisition Toolbox, as explained at:
In addition, prior to plug in the Kinect, one may also need to install the Kinect for Windows Runtime v1.6 (~114 MB), at no cost from: http://www.microsoft.com/en-ca/download/confirmation.aspx?id=34811.
For controlling the EV3 robot one uses the QUT toolkit from: https://wiki.qut.edu.au/display/cyphy/QUT+EV3+MATLAB+toolkit.
This allows communication over WiFi, Bluetooth (BT), or USB, but here it’s implemented only the BT. The driving motors IDs are B and C, while the arm motor is A.
Alternatively, without the need for Matlab or Image Acquisition Toolbox, one can use the pre-compiled installers, made on a Windows 7 Pro 64 bit computer, available at:
In order to run them, one may be asked to also install the free Matlab Win64 Compiler Runtime (MCR), version 8.3 (in case it’s not already installed). This is a free 475 Mb installation from Mathworks, which will support any executable issued from Matlab compiler, providing all the necessary libraries. KEV3.exe is a standalone executable supposing that the MCR is already installed or you take care of the MCR installation. KEV3_Installer.exe is a Windows installation version which verifies OS compatibility, downloads and installs for you the appropriate MCR.
A demo on using this program is available on YouTube under: “KEV3: Kinect integration with the EV3 robot”:
The application was made for Kinect V1 for Windows and the robot Lego Mindstorms EV3, used mainly as a car, eventually with an arm moving up and down. This arm operates a smart phone as an IPCamera to see where the robot goes. The robot has to be already BT (Bluetooth) paired with the computer. To start, press "Start camera“, wait until that button changes to “Stop Camera” and then press "BT connect". Wait few seconds for that operation to complete (one should hear a beep from EV3 and the "Stop EV3" button becomes active). Position yourself about 1 m away from Kinect and your skeleton should overlay in red your image. For a better flexibility, there is a tilt slider on the left of the camera image to adjust the Kinect viewing angle. Once the skeleton is properly acquired and the dial graph (on top of the logo) shows movements made by your right hand, you're ready to drive. The driving commands sensed by Kinect are now ready for transferring to the sliders and then to be picked up by the robot control. The driving gestures are:
- Left hand starts the robot: move the left hand horizontally to the right, for less than a second, and beyond the neck position. This allows the Kinect to sense your moves, store them on sliders, ready to be picked up by the robot code. Practically this operation starts the robot movements. The same thing is done by pressing GO. One can stop it by pressing HALT, or move again the left hand the same way.
- Left hand drives the robot: upper than elbow drives forward, lower than elbow drives backwards. The motor power is proportional with the moves. It’s shown in the vertical slider, on the left of the lab logo.
- Right hand turns left-right the robot - it has to operate like turning a driving wheel. It should follow the dial indication.
In any moment one can stop the robot with the “Stop EV3” button in the EV3 panel. The IPCamera viewing angle can be changed up and down with the Tilt++ or Tilt– buttons. It’s necessary to stop the robot driving during this operation to avoid unexpected robot movements.
References and inspirations
This code was partly inspired by the work or suggestions of other people:
- Isabela Albuquerque, Raymundo Cassani, Tiago Falk from MusaeLab
update link to the installers
update on using the KEV3_Installer
add beep comment
remove bad reference link
update QUT link
update motor IDs
added video link
Download apps, toolboxes, and other File Exchange content using Add-On Explorer in MATLAB.