Kinect Infrared (IR) and Depth Image Simulator
Updated 22 Dec 2017
This zip-file contains functions to simulate noisy Kinect infrared (IR) and depth images from a user inputted CAD model.
This work was motivated by an extensive study of the sensor's underlying mechanisms and performance characteristics, as well as newly constructed empirical models for the intensity, speckle, and detector noise of the received IR dot pattern. The simulator models the physics of the transmitter/receiver system, unique IR dot pattern, disparity/depth processing technology, and random intensity speckle and IR noise in the detectors. Our model accounts for important characteristics of Kinect's stereo triangulation system, including depth shadowing, IR dot splitting, spreading, and occlusions, correlation-based disparity estimation between windows of measured and reference IR images, and sub-pixel refinement.
The Kinect simulator functions require the CAD model to include vertex, facet, and facet normal arrays. The structure of the arrays must be:
vertex - 3xn, n vertices of each 3D coordinate that defines the CAD model.
face - 3xm, m facets, each represented by 3 vertices that defines the CAD model.
normalf - 3xm, m facets, representing the normal direction of each facet.
Note, the single CAD model input could be an aggregate collection of multiple CAD models of interesting objects and background structures. I also provide the option to include a CAD model of a flat wall to function as the scene’s background.
This package requires the Matlab wrapper for OPCODE, which is a collision detection/ray casting library for triangular 3D meshes. The wrapper was written by Vipin Vijayan, and can be found here:
This code was used to simulate line-of-sight vectors emanating from the transmitter coordinate system, which in effect represent the IR laser system that transmits the dot pattern onto the given scene. An idealized binary representation of the Kinect dot pattern is used as default for the simulator, which was constructed by Andreas Reichinger, and can be found here:
The downloadable package includes a demo to exhibit the functionality of the IR and depth image simulator options, which generates images of numerous CAD models (also included).
For a detailed description of how this simulator was developed, please refer to
M. J. Landau, B. Y. Choo, P. A. Beling, “Simulating Kinect Infrared and Depth Images,” IEEE Transactions on Cybernetics, vol. 46, no. 12, pp. 3018 - 3031, Dec. 2016.
This paper can be found here:
If this simulator is used for publication, please cite this in your references.
Michael Landau (2023). Kinect Infrared (IR) and Depth Image Simulator (https://www.mathworks.com/matlabcentral/fileexchange/50357-kinect-infrared-ir-and-depth-image-simulator), MATLAB Central File Exchange. Retrieved .
MATLAB Release Compatibility
Platform CompatibilityWindows macOS Linux
- Test and Measurement > Image Acquisition Toolbox > Image Acquisition Toolbox Supported Hardware > Kinect For Windows Sensor >
- Test and Measurement > Image Acquisition Toolbox > Image Data Acquisition > Acquisition Using Kinect for Windows Hardware >
Inspired by: Ray casting for deformable triangular 3D meshes
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!Start Hunting!
Discover Live Editor
Create scripts with code, output, and formatted text in a single executable document.
Reference citation updated in description