Code covered by the BSD License  

Highlights from
Simulink Support for Kinect

4.92857

4.9 | 14 ratings Rate this file 118 Downloads (last 30 days) File Size: 1.53 MB File ID: #32318
image thumbnail

Simulink Support for Kinect

by

 

25 Jul 2011 (Updated )

Simulink Support for Kinect is a set of Simulink blocks interfaced with OpenNI and Kinect SDK.

Editor's Notes:

This file was selected as MATLAB Central Pick of the Week

| Watch this File

File Information
Description

Simulink Support for Kinect consists of IMAQ, Image, IR, Depth, Motion and Skeleton blocks to achieve real-time image data acquisition from Kinect devices such as Microsoft Kinect or ASUS Xtion PRO under Simulink simulation environment.
It works with 32bit/64bit version of MATLAB R2010b or later in Windows7 32bit/64bit and it requires the following 3rd party software:

[C MEX Compiler]
- Microsoft Visual Studio 2010 Express Edition (VC++)
- Microsoft Windows SDK 7.1

[To use OpenNI (http://www.openni.org/)]
- OpenNI
- PrimeSense NITE
- PrimeSensor Module

[To use Microsoft Kinect SDK (http://www.microsoft.com/en-us/kinectforwindows/)]
- Microsoft Kinect for Windows SDK v1.6

For more detailed information, please refer to document in the zip file and help of the blocks.

[Optional MATLAB products]
- Computer Vision System Toolbox (highly recommended)
- DSP System Toolbox (to acquire Kinect Microphone Array data)
- Simulink Coder (to generate *.exe file)

Acknowledgements

Kinect Matlab and Kinect Microsoft Sdk inspired this file.

This file inspired Simulink For Pcv (Point Cloud Viewer).

Required Products Simulink
MATLAB release MATLAB 7.11 (R2010b)
Tags for This File   Please login to tag files.
Please login to add a comment or rating.
Comments and Ratings (102)
26 Jun 2014 Anshul Joshi

Sometimes you might get an error even when you've installed correctly:

### Generating C MEX file for Simulink Support for Kinect...
kinectSDK_wrapper.cpp
C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\INCLUDE\intrin.h(26) : fatal error C1083: Cannot open include file: 'ammintrin.h': No such file or directory

C:\PROGRA~1\MATLAB\R2011A\BIN\MEX.PL: Error: Compile of 'kinectSDK_wrapper.cpp' failed.

??? Error using ==> mex at 208
Unable to complete successfully.

Error in ==> makecmexfile4nid at 62
mex('sfun_nid.cpp', 'kinectSDK_wrapper.cpp', 'wrapper_common.cpp', ['-L'
KinectSDKPathLib], '-lKinect10', ['-I' KinectSDKPathInclude]);

Error in ==> setup_kinectsdk at 76
makecmexfile4nid;

This is because it cannot find the file ammintrin.h. Run CMD prompt as administrator and use:
echo. > "C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\include\ammintrin.h"

to generate that file. (This instruction is given at http://www.chromium.org/developers/how-tos/build-instructions-windows)

30 Apr 2014 Flavius Domide

Hello, i have a question

When i try to run nid_cvst_motion i have the error : Error evaluating 'InitFcn' callback of NID IMAQ block (mask) 'nid_cvst_motion/NID IMAQ1'.

Caused by:
Error in S-function 'nid_cvst_motion/NID IMAQ1': S-Function 'sfun_nid' does not exist

I unlocked the library that i used.

Thank you !

19 Apr 2014 discuter harry

阿达

02 Apr 2014 yashin

Hello;

i have errors when i execute programs of skeleton

what's this problem ?

Can you help me

Thank you very much

02 Apr 2014 yashin

hello ;

I succeed to working the kinect.
Now,I want to get the positions X,Y,Z of my quadrortor by using the program of Simulink Support For Kinect
you know the good program can help me ? and i can't execute some programs like "nid_Cvst_point_could.mdl" nid_cvst_skelton"nid_cvst_corner_detection"
Have you an idea about this problem

Thank you very much
Have a good day
Yassine

05 Mar 2014 Malstheet

Hi i have a question, i got the depth data and color data image what should i do to get a good segmentation

24 Feb 2014 Takashi Chikamasa

Important - OpenNI SDK software downloads will continue to be available until April 23rd, 2014 at which time the OpenNI website will be closed.

http://www.openni.org/openni-sdk/

24 Feb 2014 Takashi Chikamasa

Hi yashin,

You might need to connect a Kinect before executing MATLAB as Dghy advised.

P.S. Thanks!, Dghy

23 Feb 2014 Dghy

Hi Yassine,
I had same problem. I think it is caused by MATLAB not recognizing Kinect. In my case, I do:
1. Connect Kinect to PC before running MATLAB/Simulink.
2. In Matlab run mux -setup and SETUP_KINECTSDK again.
3. Open Simulink.

It helped with my problem.

Daniel.

20 Feb 2014 yashin

Hello takashi;

I successfully run SETUP_KINECTSDK but when I ran some example this message appears.
Error reported by S-function 'sfun_nid' in 'nid_cvst_image/NID IMAQ':

NID Error: Failed to initialize NID. It might be caused by
- NID is not connected.
- NID unsupported block is used in the model

please help me to solve this problem

thank you in advance

Yassine

14 Feb 2014 Takashi Chikamasa

Hi Johanna,

>I have one more question to make everything >clear, the kinect has to be used in normal mode >in simulink and there is no way then to run the >kinect and get image in real time?
You could run the Kinect blocks in accelerator mode in Simulink, even you could generate native excutable (*.exe) by Simulink coder. The exception is only the Skeleton Viewer because it is written in M file S-Funciton.

13 Feb 2014 Johanna

Takashi, thank you for your answer.

I have one more question to make everything clear, the kinect has to be used in normal mode in simulink and there is no way then to run the kinect and get image in real time?

Thanks again

13 Feb 2014 Takashi Chikamasa

Hi Johanna,

Unfortunately, the skeleton viewer is written in M file S-Function. Therefore, Code generation products could not work with it. Sorry for that.

11 Feb 2014 Johanna

Hi Takashi,

I am using this support for kinect and it is great! I however have a problem with running it in real time, I am trying to use external mode to run it in real time but I get the following error message.

The corresponding 'skeleton_viewer.tlc' file for the MATLAB S-function 'skeleton_viewer' in block 'nid_cvst_skeleton/Skeleton Viewer' must be located in the current working directory, the MATLAB S-function directory 'C:\Work\johannai\slkinect\Lib', or the directory 'C:\...\slkinect\Lib\tlc_c'

There doesnt seen to be a skeleton_viewer.tlc file in the package? Do you have any idea how I can solve this problem?

I really appreciate your work, thanks!

07 Feb 2014 kn ra

Hi takashi,

I install PrimeSense v 1.5, OpenNI v 1.5 and SDK v 1.7 and the Kinect works very well.but when I run (nid_cvst_pattern_tracking). matlab does not include NID block IMAQ.
My question is is there a block that I must download and put it in matlab?

05 Feb 2014 Dghy

Hello Takashi,

I really appreciate your fast response. Thank you for advice.

I am throwing 80 mm ball which is about 2.5m/3m away from Kinect. As you advised, I tried throwing bigger object and it worked. Now depth camera can recognize and obtain correct Z coordinate of my flying object (I don't have bigger ball at the moment, so I used box).
(Size of throwed box: 240mm x 120mm x 80 mm).

Thank you very much for your help. You gave me fast and good solution for my problem which I was trying to solve for a big amount of time.
I only regret that I haven't asked you for help sooner, I would save a lot of time.
Thank you again.

Daniel.

05 Feb 2014 Takashi Chikamasa

In terms of data acquisition for both of RGB and Depth, the way of implementation which I used collects both data (RGB and Depth) at the same time, so I think there should be no delay between those data.

How much is the size of the flying ball? Kinect uses a lot of infrared spots to measure depth. But I am not sure how much small ball Kinect depth sensor can measure. I guess that bigger ball should be more stable and accurate.

04 Feb 2014 Dghy

Hello Takashi,

Thank you for your fast reply, I really appreciate that you have found a moment for my problem.

I have looked at those check boxes and they are ON at all times. Also looked at sample that you recommended. I noticed that X, Y, Z coordinates of Depth are received form NID Depth block. In my program, for ball recognition and obtaining it coordinates, I am using Blob Analysis (Centroid) (just like in your "nid_cvst_pattern_tracking.mdl"). It gives me X and Y coordinates in pixels. In next step, I take those X,Y coordinates and use them to find Z coordinate form Depth data (output form NID Depth is given as "Depth").
Despite of usage "Adjust view point to Image", sometimes (especially in parallel to X axis throws) my Z coordinates are wrong.

Is it possible that coordinates from RGB image are not accurate enough, or Depth image has some kind of Delay ( maybe weak PC ) ?
I could use X,Y,Z coordinates from NID Depth block but in that case, I have severe fps drops, so fast flying ball is not recognized.

Thank you again for your time.

Looking forward for your replay,
Daniel.

04 Feb 2014 Takashi Chikamasa

Dear Dghy,

To synchronize Depth and RGB image, please make sure the following block dialog parameters are checked.
1. Adjust view point to Image check box: ON in NID IMAQ block
2. Real world coordinates check box: ON in NID Depth block

nid_cvst_point_cloud.slx sample might also help you to understand how to synchronize RGB and Depth.

03 Feb 2014 Dghy

Hello Takashi,

I have made system that can recognize red ball and calculate its trajectory. During tests, I noticed something: RBG image and Depth image are not well synchronized. It doesn't make much difference when I am throwing ball towards Kinect, but when ball is flying parallel to X axis, it is a problem. Z coordinates of ball are collected by mean of X and Y coordinates form RGB image. In this case, my Z coordinate colected forom Depth data is all wrong. Here is an image that will help you understand my problem:
http://imgur.com/JKIwqYr

Can you advise any solution ?
Thanks in advance,
Dghy.

14 Jan 2014 Takashi Chikamasa

Hi Kelvin,

The path to the OpenNI should be like
"C:\Program Files\OpenNI" (for 64bit Win).
Please make sure that you specified to OpenNI folder during set up. OpenNI versions which you pointed should be fine.

12 Jan 2014 KELVIN YAP

Hi Takashi,

I uninstalled the previous version of openNI, PrimeSense-Nite and PrimeSense Sensori and downloaded the following version:

1. openNI 1.5.2.23
2. PrimeSense-Nite 1.5.2.21
3. PrimeSensor v5.1.0.41

Previously I have typed wrongly. I am using Matlab 2011a and not 2013a.

I have followed the steps as stated but I keep getting the same error message:

??? Error using ==> setup_openni>GuiBasedSetUp at 210
ERROR: OpenNI might not be installed or version mismatch.
Error in ==> setup_openni at 51
[openNI_dir] = GuiBasedSetUp();

Also, I am unsure what it meant when it request to specify OpenNI Installation path. I entered the folder where the openNI files are kept. Not sure if that the correct way to do it.

Can you please advise me on solving this issue?

Thanks

Kelvin

10 Jan 2014 KELVIN YAP

Hi Takashi,

I have downloaded the following:

1. openNI 1.5.4.0
2. PrimeSense-Nite 1.5.2.21
3. PrimeSense Sensor 5.1.2.1

I have downloaded the Microsoft Visual Studio 2010 Express Edition and Microsoft Windows SDK 7.1 and had set it as C MEX compiler.

However when I ran the setup-openni, I kept getting the following error message:

Error using setup_openni>GuiBasedSetUp (line 210)
ERROR: OpenNI might not be installed or version mismatch.
Error in setup_openni (line 51)
[openNI_dir] = GuiBasedSetUp();

By the way, I am using matlab 2013a

I do not know where is wrong. Hope that you can advise me on how to solve it.

Thanks :)

Kelvin

24 Dec 2013 Takashi Chikamasa

Hi Jose,

What kind of error message have you seen? if you are OK, please copy and paste the message in the comments.

21 Dec 2013 jose

Hi Takashi
It's possible to use the IMAQ and SKELETON blocks by using Real-Time Windows Target???
The thing is that I want to control using the kinect a Parrot AR Drone through the "AR Drone Simulink Development-Kit V1 by David Escobar Sanabria (http://www.mathworks.com/matlabcentral/fileexchange/43719-ar-drone-simulink-development-kit-v1)" but when i tried to execute, it gives a lot of errors. I would be grateful if you could give any idea. Thanks!!!
PS: My english is not good, because i'm from Ecuador and i barely write in English

25 Nov 2013 Takashi Chikamasa

Hi Vincent,

NID IMAQ block has a real time synchronization capability, therefore, it seems that the real time Synchronization block is not needed in your model.

20 Nov 2013 Johanna

Hi, I have done the compilation but how do I after that get the blocks into simulink? Or did something go weong since they are not there?

thanks

05 Nov 2013 Vincent

Hi,

I am using the kinect simulink blocks from the library together with a real time synchronization. The real time Synchronization block is constantly missing ticks and when looking at the profiler report, the reason is that the NID IMAQ block uses all its sample time, as if it is idling and waiting for the next image. This happens at 0.04 s as well as 0.2 s or more.
Do you know how this time can be set free for other calculations to be done?

Thank you in advance,
Vincent

05 Nov 2013 Takashi Chikamasa

Hi Guanjiu,
"video viewer" and "frame rate display" are blocks in Computer Vision System Toolbox™. Simulink support for Kinect works without the toolbox, so if it is annoying, you can remove the blocks.

02 Nov 2013 Guanjiu

Hi,
I was confused about every time when I launch each mdl files it shows two missing blocks, one is the "video viewer" and the other is "frame rate display", do you know what is the issue?

Thanks

10 Oct 2013 Agustin

OK, I've just found the solution, it was just a silly mistake. Thanks for the great files.

26 Sep 2013 Dave

Ok, thank you for your help, I can now confirm that this submission works correctly with Visual Studio 2012 and Windows 8 and 8.1.

Yet I have one question regarding my project, is there any way to capture an infrarred signal with the IR emitter turned off? Thank you for your help.

24 Sep 2013 Takashi Chikamasa

Hi Dave,
The setup scripts are assumed that Lib directory is located at the same place where the scripts are placed. Is this valid for your case?

20 Sep 2013 Dave

Hello Takanashi,
I have already installed everything mentioned here and I'm using the Kinect SDK v1.8 but no matter what I try, i keep getting this output:

### Confirming Simulink Support for Kinect installations with Kinect SDK...
### Setting MATLAB path to .\Lib and .\Lib\doc_en directories.
Warning: Name is nonexistent or not a directory: C:\Users\dave\Lib
> In path at 110
In addpath at 87
In setup_kinectsdk at 54
Warning: Name is nonexistent or not a directory: C:\Users\dave\Lib\doc_en
> In path at 110
In addpath at 87
In setup_kinectsdk at 55
### Generating C:\Users\dave\Lib\simulinkfornidinfo.m
Error using fclose
Invalid file identifier. Use fopen to generate a valid file identifier.

Error in setup_kinectsdk (line 68)
fclose(fid);

How could I fix this problem? Thank you for your time and consideration.

18 Sep 2013 Takashi Chikamasa

The latest Kinect SDK v1.8 also works with the Simulink Support for Kinect.

16 Jul 2013 Takashi Chikamasa

Hi Ryan,

Instrument Control Toolbox™ might help you. If you are familiar with S-Function in Simulink, you can write your own driver block to control other hardware.

14 Jul 2013 Ryan Robinson

Hi Takashi,

I am trying to get a robot arm to respond to what the Kinect sees in real time. Do you know a good way to get Simulink to communicate with other hardware (e.g. Arduino) while running your code?

Thanks,
Ryan

11 Jun 2013 Piña Patiño Ambrocio

Hiii!! Takashi Chikamasa

My name is Ambrocio Piña, From México:

Excuse me, my english is not good, but I need your help with sensor kinect and simulink, I can´t successfully install this drivers.

I install the software in the following order:

1.- GRMSDKX_EN_DVD (SDK for Windows 7.1)

2.- KinectSDK-v1.6-Setup ((Kinect SDK 1.6)

3.- vc_web (Visual c++)

4.- in matlab my compiler already SDK

but, when I run slkinect/setup_kinectsdk.m
I have the following error:

>> setup_kinectsdk
### Confirming Simulink Support for Kinect installations with Kinect SDK...
### Setting MATLAB path to .\Lib and .\Lib\doc_en directories.
### Generating C:\Program Files\MATLAB\Kinect-Simulink\slkinect\Lib\simulinkfornidinfo.m
Error using fclose
Invalid file identifier. Use fopen to generate a valid file identifier.

Error in setup_kinectsdk (line 68)
fclose(fid);

PLEASE HELP TO ME, I NEED FOR A SCHOOL PROYECT, IS URGENT.

FOR YOUR CONSIDERATION, THANK YOU VERY MUCH

AMBROCIO PIÑA.

12 Mar 2013 Christian

Hello everybody,

First of all thanks Takashi for this great submission.
Second, I successfully ran SETUP_KINECTSDK and setup_openni.m, but when I try to run some example this message appears.

Error reported by S-function 'sfun_nid' in 'nid_cvst_image/NID IMAQ':

NID Error: Failed to initialize NID. It might be caused by
- NID is not connected.
- NID unsupported block is used in the model
- NID unsupported IMAQ Resolution and FPS is configured.

Some clue to solve this.

Thanks in advance

Christian

13 Feb 2013 Samuel

Dan,

You could connect the signal output of the Skeleton block to a "To workspace" Simulink block. This will save the output of your simulation to your current MATLAB workspace. You can then access joint data as a 20x3xN matrix (where N is the number of frames in your simulation and assuming you are using the Windows kinect runtime).

13 Feb 2013 Dan

Thank you for providing this code. Great resource. Question, how would you go about saving the XYZ for each joint? I am not familiar with Simulink. Thanks

07 Feb 2013 Takashi Chikamasa

Hi all,

Please make sure about the following steps to solve installation issues:
1. Make sure that all SDK (VC++, Windows SDK, Kinect SDK/OpenNI) are installed properly
2. Make sure that VC++ is set as mex compiler
>> mex -setup
3. Read document in the Simulink support Kinect.

Additionally, I recommend to install MATLAB under the directory without spaces because of mex compilation with third party SDKs.

Concerning Windows8, I have never used Win8, so unfortunately, I can't answer your question.

Takashi

03 Feb 2013 Maleeha

Hi Takashi,
I am trying to run the setup_kinectsdk.m file on the Matlab, however it gives me the following error
"
>> setup_kinectsdk
### Confirming Simulink Support for Kinect installations with Kinect SDK...
### Setting MATLAB path to .\Lib and .\Lib\doc_en directories.
### Generating C:\Users\Shahzada\Documents\MATLAB\simulinkfornidinfo.m
### Generating C MEX file for Simulink Support for Kinect...

C:\PROGRA~1\MATLAB\R2012A\BIN\MEX.PL: Error: 'sfun_nid.cpp' not found.

Error using mex (line 206)
Unable to complete successfully.

Error in makecmexfile4nid (line 62)
mex('sfun_nid.cpp', 'kinectSDK_wrapper.cpp', 'wrapper_common.cpp', ['-L' KinectSDKPathLib],
'-lKinect10', ['-I' KinectSDKPathInclude]);

Error in setup_kinectsdk (line 77)
makecmexfile4nid;

"

Please let me know if you can help at your earliest convenience.

Thank you

02 Feb 2013 sahar

Hi Takashi,

Can the 'Simulink support for kinect' work on Windows 8 64bit??

31 Jan 2013 Takashi Chikamasa

Concerning OpenNI, you need to use an older version of OpenNI (http://www.openni.org/openni-sdk/openni-sdk-history-2/). The latest OpenNI 2.1 beta is totally different from the older versions, thus, it does not work with Simulink support for Kinect.

28 Jan 2013 Mehwish

Also, after I run the setup file, I'm supposed to get both files, sfun_nid.mexw32 and simulinkinfo.m, right? Because I've only been able to get simulinkinfo.m file so far.

28 Jan 2013 Mehwish

Hi Takashi,

I'm still having the same problems.

1) Initially I was using the setup_kinectsdk.m file. When I run the .m file, I'm asked to specify the location of the Kinect installers. I couldn't solve the problem even when I used C:\Program Files\Microsoft SDKs\Kinect\v1.6

2) I know my Kinect SDK was installed properly. But just incase, I now tried with setup_openni.m file. OpenNI is installed correctly, as verified by Matlab as 'ok' and the camera is capturing images correctly when used as a webcam. But I'm experiencing the same errors as before. Problem with directory or path...

Do I have to change the code for any of the files to make it work for me?

>> dir1 = 'C:\Program Files\OpenNI';
>> setup_openni(dir1);
### Verifying OpenNI: OK
### Confirming Simulink Support for Kinect installations with OpenNI...
### Setting MATLAB path to .\Lib and .\Lib\doc_en directories.
Warning: Name is nonexistent or not a directory: C:\Users\EDP\Documents\MATLAB\Lib
> In path at 110
In addpath at 87
In setup_openni at 78
Warning: Name is nonexistent or not a directory:
C:\Users\EDP\Documents\MATLAB\Lib\doc_en
> In path at 110
In addpath at 87
In setup_openni at 79
### Generating C:\Users\EDP\Documents\MATLAB\Lib\simulinkfornidinfo.m
Error using fclose
Invalid file identifier. Use fopen to generate a valid file identifier.

Error in setup_openni (line 94)
fclose(fid);

24 Jan 2013 Takashi Chikamasa

OK. You specified the wrong path for
SIMULINKFORNIDKINECTSDKPATH = 'C:\Users\EDP\Documents\MATLAB\simulink_support_for_kinect_0.6.0\slkinect\Lib';

It has to be the path to MS Kinect SDK (not Simulink support for Kinect).
In my case,
SIMULINKFORNIDKINECTSDKPATH = 'C:\Program Files\Microsoft SDKs\Kinect\v1.6';

Please check the MS Kinect SDK could be installed properly.

23 Jan 2013 Mehwish

However, I'm getting the following error:

>> setup_kinectsdk
### Confirming Simulink Support for Kinect installations with Kinect SDK...
### Setting MATLAB path to .\Lib and .\Lib\doc_en directories.
Warning: Name is nonexistent or not a directory:
C:\Users\EDP\Documents\MATLAB\simulink_support_for_kinect_0.6.0\Lib
> In path at 110
In addpath at 87
In setup_kinectsdk at 54
Warning: Name is nonexistent or not a directory:
C:\Users\EDP\Documents\MATLAB\simulink_support_for_kinect_0.6.0\Lib\doc_en
> In path at 110
In addpath at 87
In setup_kinectsdk at 55
### Generating C:\Users\EDP\Documents\MATLAB\simulink_support_for_kinect_0.6.0\Lib\simulinkfornidinfo.m
Error using fclose
Invalid file identifier. Use fopen to generate a valid file identifier.

Error in setup_kinectsdk (line 68)
fclose(fid);

23 Jan 2013 Mehwish

Hi Takashi,

I checked that file simulinkfornidinfo.m. And yes it says the following:

% This file is automatically generated by setup_kinectsdk.m
SIMULINKFORNIDDRIVER = 'KINECT_SDK';
SIMULINKFORNIDKINECTSDKPATH = 'C:\Users\EDP\Documents\MATLAB\simulink_support_for_kinect_0.6.0\slkinect\Lib';

23 Jan 2013 Takashi Chikamasa

Hi Mehwish,
Could you check the simulinkfornidinfo.m? if it was generated correctly, the file constains something like following:

% This file is automatically generated by setup_kinectsdk.m
SIMULINKFORNIDDRIVER = 'KINECT_SDK';
SIMULINKFORNIDKINECTSDKPATH = 'C:\Program Files\Microsoft SDKs\Kinect\v1.6';

22 Jan 2013 Mehwish

Hi, I'm following the steps in the Simulink_Support_for_Kinect_En.pdf. I'm stuck on Step 5, which states to Run slkinect/setup_kinectsdk.

If everything is installed properly (which I'm sure it is), sfun_nid.mexw32/.mexw64 and simulinkfornidinfo.m should be generated.

However, following is the error I'm getting.

>> setup_kinectsdk
### Confirming Simulink Support for Kinect installations with Kinect SDK...
### Setting MATLAB path to .\Lib and .\Lib\doc_en directories.
### Generating C:\Users\EDP\Documents\MATLAB\simulink_support_for_kinect_0.6.0\slkinect\Lib\simulinkfornidinfo.m
Error using setup_kinectsdk (line 69)
### ERROR: Failed to generate
C:\Users\EDP\Documents\MATLAB\simulink_support_for_kinect_0.6.0\slkinect\Lib\simulinkfornidinfo.m

19 Jan 2013 Mehwish  
06 Dec 2012 Samuel

Hi Takashi,

Thanks for your quick answer. OpenNI and NITE do work on OSX with the PrimeSensor module for OSX (unstable). I was thinking that the Simulink interface should work too then. Thanks for the great work!

Sam

04 Dec 2012 Jiri Zemanek

Hi Takashi,

thank you for the advice it was a stupid mistake :(. Anyway, you did a great job with this blockset! Thanks for that.

Jiri

27 Nov 2012 Takashi Chikamasa

Hi Samuel,

>I'm trying the latest version with an Asus Xtion >Pro Live, but the NID Skeleton block always >causes MATLAB to crash.
ASUS Xtion PRO Live does not support for Mac OS X, only support for Win/Linux according to ASUS, so it is a known restriction by ASUS (neither OpenNI nor Simulink for NID)

27 Nov 2012 Samuel

I'm trying the latest version with an Asus Xtion Pro Live, but the NID Skeleton block always causes MATLAB to crash. The depth and video blocks work fine however, and everything works fine with a Kinect.

I'm working with OpenNI on OSX.

26 Nov 2012 Takashi Chikamasa

Hi Jiri,

You need to use the Kinect SDK 1.6 (not 1.0).

Takashi

22 Nov 2012 Jiri Zemanek

Hi,

unfortunately, I'm not able to compile the cpp files. I have Kinect for Win (SDK 1.0), Matlab R2010b 64bit, MS VS 2010 running on Win7 64bit and after calling mex() I am getting a long list of errors (attached bellow). Do you have any idea what's wrong? Thanks.

kinectSDK_wrapper.cpp(54) : error C2143: syntax error : missing ';' before '*'
kinectSDK_wrapper.cpp(54) : error C4430: missing type specifier - int assumed. Note: C++ does not support default-int
kinectSDK_wrapper.cpp(54) : error C4430: missing type specifier - int assumed. Note: C++ does not support default-int
kinectSDK_wrapper.cpp(830) : error C2039: 'NuiGetCoordinateMapper' : is not a member of 'INuiSensor'
C:\Program Files\Microsoft SDKs\Kinect\v1.0\inc\NuiSensor.h(508) : see declaration of 'INuiSensor'
...

02 Nov 2012 Samuel  
03 Oct 2012 Takashi Chikamasa

>Can we use this one for multiple kinects?
Sorry, Simulink for NID does not support multiple kinect devices. It requires huge changes in the code.

03 Oct 2012 Takashi Chikamasa

>I am trying to output xyz coordinate depth data over >time. I would like to output these coordinates in real->time. Is this possible with this code? If so, how
>(exactly) can I implement this?
nid_cvst_point_cloud.mdl might be a good example to understand how to acquire xyz coordinates depth data. This model uses a Frame Rate
Display of CVST, but it has nothing with the depth data, so you can remove it and you can manipulate the xyz depth data in Simulink without CVST.

21 Sep 2012 Tharindu

Can we use this one for multiple kinects?

19 Sep 2012 Mike Powell

I am trying to output xyz coordinate depth data over time. I would like to output these coordinates in real-time. Is this possible with this code? If so, how (exactly) can I implement this?

What I have tried so far (among many other things):
1. Open the nid_depth.mdl block
2. Double click the NID Depth block and change output to XYZ.
3. Check "Real World Coordinates"
4. Double click on the selector. Change the number of input dimensions to 3.
5. Insert a simulink/Sinks/To Workspace block instead of the scope.
6. Change the time from inf to 1.5.
7. Double click the wire between the NID Depth block and the To Workspace block. Name the wire using a name without spaces.
7. Hit run.

When I do this, the output is a 1x1x46 timeseries. This is not what I am looking for.

To do what I am asking, do I need the computer vision system toolbox (which I do not currently have)?

17 Sep 2012 marchienne

Thank you very much for your help. Your works are very useful !

13 Sep 2012 Takashi Chikamasa

Hi marchienne,
You can create MATLAB Guide based viewer for any types of Kinect acquired data. I am going to bundle some example which uses this technique in the next update (maybe in October after MS released the SDK 2.0).

13 Sep 2012 marchienne

Hi,

Is someone able to see the first examples? I have an error message that says "Unable to load block diagram 'vipsnks'". I saw that we had to use a toolbox named "Computer Vision System Toolbox" but there is a way to do it without this toolbox ?

Thank you for your help.

06 Sep 2012 Takashi Chikamasa

Hi Mike,
Thank you for your information. I will add your tips to the documents in the next update.

05 Sep 2012 Mike Powell

Actually, I just fixed the mex problem and it ran successfully using the recommedation listed at
http://www.mathworks.com/matlabcentral/answers/27759#answer_36115

"I had to copy "mspdb100.dll" from the "..\Microsoft Visual Studio 10.0\Common7\IDE\" to the "..\Microsoft Visual Studio 10.0\VC\bin\"."

by Stephen Yip.

05 Sep 2012 Mike Powell

(and I wanted to add that I have a Simulink license)

Amanda

05 Sep 2012 Mike Powell

Hi,

I have installed:
- Microsoft SDK 7.1
- Visual Studio C++ 2010 Express
- OpenNI Stable Build for Windows x86 v1.5.2.23 Redist Edition
- PrimeSense NITE Stable Build for Windows x86 v 1.5.2.21 Redist Edition
- PrimeSensor Module Stable Build x86 v 5.1.0.41
- Kinect for Windows SDK v1.5

However, when I open setup_kinectsdk.m in MATLAB and hit run, I get the following error:

C:\PROGRA~1\MATLAB\R2012A\BIN\MEX.PL: Error: Compile of 'sfun_nid.cpp' failed.

Error using mex (line 206)
Unable to complete successfully.

Error in makecmexfile4nid (line 62)
mex('sfun_nid.cpp', 'kinectSDK_wrapper.cpp', 'wrapper_common.cpp', ['-L' KinectSDKPathLib], '-lKinect10', ['-I'
KinectSDKPathInclude]);

Error in setup_kinectsdk (line 76)
makecmexfile4nid;

I have tried changing the compiler using mex -setup, but neither compiler works. The compiler options I tried were:

[1] Lcc-win32 C 2.4.1 in C:\PROGRA~1\MATLAB\R2012a\sys\lcc
[2] Microsoft Software Development Kit (SDK) 7.1 in C:\Program Files\Microsoft Visual Studio 10.0

Any assistance or suggestions would be appreciated.

Amanda

20 Aug 2012 Akhand Tripathy  
16 Aug 2012 Takashi Chikamasa

I forgot to inform important thing about 0.5.1:
- Support OpenNI in MAX OS X! and Linux!

13 Jul 2012 Takashi Chikamasa

Hi Francisco,

I assume you have Simulink license. Is this correct?

03 Jul 2012 Francisco

Hello everyone. I am having the same problems as Jno navarro. Any solution to this problem?

c:\users\jno\desktop\kinect project\simulink_nid_0.4.1\slnid\lib\sfun_nid.h(16) : fatal error C1083: Cannot open include file: 'simstruc.h': No such file or directory

C:\PROGRA~1\MATLAB\R2010B\BIN\MEX.PL: Error: Compile of 'sfun_nid.cpp' failed.

??? Error using ==> mex at 208
Unable to complete successfully.

Error in ==> makecmexfile at 54
mex('sfun_nid.cpp', 'kinectSDK_wrapper.cpp', 'wrapper_common.cpp', ['-L'
KinectSDKPathLib], '-lKinect10', ['-I' KinectSDKPathInclude]);

Error in ==> setup_kinectsdk at 78
makecmexfile;

22 May 2012 Takashi Chikamasa

What's new 0.4.2 in detail
-Support Skeleton tracking in the Kinect SDK v1.5 Near mode and Seated mode
-Support multiple person skeleton tracking (up to two persons)
-Support multi-instantiation of NID Device blocks in a model

[In Japanese]
-Kinect SDK v1.5の最新機能である、Near mode、Seated modeにおける骨格トラッキングに対応
-最大2人までの複数ユーザーの同時骨格トラッキングに対応
-同一NID Deviceブロックのモデル内複数配置に対応

26 Apr 2012 Takashi Chikamasa

Hi jno,

Great to hear your success in your project. As far as I tested with a Kinect for Windows HW, depth and image with Near mode could work fine. So it sounds strange. In case of Near mode, I am sure that skeleton tracking does not work in v1.0 Kinect SDK. According to MS, they will release v1.5 in May and it seems to support skeleton tracking in Near mode. Anyways, your 3D scanner should be absolutely interesting for us. If you published some of your work as video such as at YouTube, please let me know.

26 Apr 2012 jno navarro

Hi Takashi,
Thank you for your fast answers, my 3d scanner is working quite well, the only thing is that i am not able to use near mode, i am using kinect sdk last version and everything seems to work well except that whe i turn on near mode after using my application it says fail to use near mode and it doesnt work. Do you know why this may happen?
Thank you

18 Apr 2012 Takashi Chikamasa

Hi jno,

>so the element 1,1 of the matrix in real
>world coordinates is in the case of the
>x matrix, the x position of the point in
>that pixel, is that correct?
In my experience with the point cloud sample. You assumption should be correct. The point cloud viewer is mapping image on each pixel onto real world XYZ coordinates at the same pixel.

18 Apr 2012 jno navarro

Hi Takashi,
I have readed the help format, but what i uderstando is that the depth block gives me, in the case x/y/z, the x y and z position of each pixel, formed by a 640*480 matrix, so the element 1,1 of the matrix in real world coordinates is in the case of the x matrix, the x position of the point in that pixel, is that correct??

08 Apr 2012 Takashi Chikamasa

Hi jno,

Please see the help of Depth block. You can find description about depth format.

06 Apr 2012 jno navarro

Hi Takashi,
I found the error in another blog, there is a problem with my matlab and it is already fixed. Your program runs great, can you tell me in which format you get the points of depth?

02 Apr 2012 Takashi Chikamasa

Hi jno,

Did you install Simulink on your PC? Simuilnk for NID is for Simulink, not for MATLAB.

27 Mar 2012 jno navarro

Hi everyone,
I have Matlab 7.11.0 (r2010b) in a win 7 64 bits. Each time I try to compile the archives using vs2010 professional I get the following error:
sfun_nid.cpp
c:\users\jno\desktop\kinect project\simulink_nid_0.4.1\slnid\lib\sfun_nid.h(16) : fatal error C1083: Cannot open include file: 'simstruc.h': No such file or directory

C:\PROGRA~1\MATLAB\R2010B\BIN\MEX.PL: Error: Compile of 'sfun_nid.cpp' failed.

??? Error using ==> mex at 208
Unable to complete successfully.

Error in ==> makecmexfile at 54
mex('sfun_nid.cpp', 'kinectSDK_wrapper.cpp', 'wrapper_common.cpp', ['-L'
KinectSDKPathLib], '-lKinect10', ['-I' KinectSDKPathInclude]);

Error in ==> setup_kinectsdk at 78
makecmexfile;

This is really urgent, I am trying to use my kinect to generate 3d models through matlab but I am stuck with that error, I would appreciate anyway to get images through matlab
Thank you in advance

21 Mar 2012 Tharindu

Great code. thanks for sharing.
and thanks for the quick responses to the questions

11 Mar 2012 Takashi Chikamasa

Hi lev,

I think ASUS Xtion does not have an accelerator sensor, so we can't get the tilt angles.

10 Mar 2012 lev

Hi Takashi
Can we use ASUS Xtion to get the tilt angles instead of Kinect.

28 Feb 2012 Takashi Chikamasa

Hi Michael and lev,
The latest version (0.4.1) supports your requests. It will be published soon.

26 Feb 2012 lev

Hi Takashi
please add an outport on IMAQ to get the tilt angles.

24 Feb 2012 Michael

Thank you sir. If you could do that, it would be much appreciated.

23 Feb 2012 Takashi Chikamasa

Hi Michael,

>Can you advise me on an easy way to
>access the vector data pertaining to the
>skeletal joint points?
point_cloud_viewer.m might be useful to understand how to access the data easily. I am sure both of eML and Simulink, you can access the data pretty easy. I also consider adding a demo based on your question.

23 Feb 2012 Takashi Chikamasa

Hi Lev,

>if you can provide the heading and tilt angles.
The Kinect tilt angle can be acquired by the Kinect SDK, but heading angle is not supported by the SDK. If the tilt angle is sufficient, I can add an outport on IMAQ to get the tilt angle.

23 Feb 2012 lev  
22 Feb 2012 Michael

Can you advise me on an easy way to access the vector data pertaining to the skeletal joint points?

Pretty much what I am trying to do is use the joint 3d data to control servos, making a robotic arm mimic your movements.

22 Feb 2012 lev

Hi Takashi,
Can I obtain the accelerometer data using this Simulink, I deeply appreciate if you can provide the heading and tilt angles.

16 Feb 2012 Takashi Chikamasa

A new feature of Kinect for Windows hardware "Near mode" also worked fine with the latest Simulink for NID.

[Japanese]
Kinect for Windowsハードウェアの新機能である"Near Mode"も正常に動作することを確認しました。

08 Feb 2012 Takashi Chikamasa

[Japanese]
ver.0.4.0から、マイクロソフトのKinect for Windows SDK v1.0にも対応しました。DSP System Toolboxを使用することで、Kinect for Windowsの4chマイクロフォンアレイ生データもSimulink上で取得することができます。

11 Nov 2011 Emanuele Ruffaldi

Hi Takashi,

I am referring to the ONI files that can be recorded with NiViewer provided with OpenNI. These ONI files contain the various Kinect streams (depth and/or image). Using this patch it is possible to keep the same Simulink model and select to get data from a recorded ONI file or from the real sensor. It is useful for testing a model with consistent input (e.g. different users). The patch works by adding a FILE_BLOCK (7) to the S-Function dispatcher, and selecting it if the filename parameter of the mask is not empty.

It is even possible to have NiViewer recording data while using the real Kinect from Simulink.

Best,
Emanuele

09 Nov 2011 Takashi Chikamasa

Hi Emanuele,

Thank you for your post. Next update will support both of 32bit/64bit version of MATLAB. Concerning data recording, there are several options in Simulink:
1. To/From Workspace blocks
If you also have a Computer Vision System Toolbox, you can use
2. To/From Multimedia File blocks
3. Video To/From Workspace blocks

09 Nov 2011 Emanuele Ruffaldi

I've made an update with two changes:
- support for MATLAB 64bit
- support for using recorded ONI files as input

There is now a new block for file input.
Another one allows mixed situation: if the filename is empty it uses the real-time input otherwise the recorded file.

Thanks for this greate package

Source code: http://dl.dropbox.com/u/146279/simulink_nid_0.3.1file.zip

18 Oct 2011 Takashi Chikamasa

[English]
ver.0.3.1 supports QVGA(320x240): 60FPS IMAQ for ASUS Xtion PRO series devices.

[Japanese]
ver.0.3.1では、ASUS Xtion PROシリーズデバイスを対象に、QVGA(320x240): 60FPSのIMAQをサポートしました。

20 Sep 2011 Takashi Chikamasa

%% Description in Japanese
Simulink for NID は、IMAQ, Image, IR, Depth, Motion および Skeletonブロックから構成されており、OpenNIが対応しているデバイスからのリアルタイムデータ取得が可能です。Simulink for NIDの使用に際しては、32bitバージョンのR2010b以降のMATLAB製品をWindows7 32bit/64bit環境にインストールする必要があります。また、次のサードパーティソフトウェアが環境構築に必要になります:
- Microsoft Visual Studio 2010 (VC++) Express Edition
- OpenNI SDK
- PrimeSense NITE
- OpenNIに対応したNID向けデバイスドライバ
※詳細については、ZIPファイルに同梱されている日本語readme_ja.textおよび各ブロックの日本語ヘルプを参照してください。

[推奨MATLAB製品]
- Computer Vision System Toolbox
- Simulink Coder

Updates
13 Sep 2011

Updated to ver. 0.3.0.

16 Oct 2011

Support QVGA(320x240): 60FPS for NID IMAQ in addtion to VGA(640x480): 30FPS.
Note that QVGA(320x240): 60FPS is supported by only ASUS Xtion PRO series devices.

09 Dec 2011

-Support 64bit MATLAB
-Support both of pose less/PSI pose calibration for skeleton tracking
-Added new computer vision demos

06 Feb 2012

What's new in 0.4.0 (02/07/2012)
- Support Microsoft Kinect for Windows SDK in addition to OpenNI as NID SDK
- Added demos using Kinect Microphone Array (DSP System Toolbox is required)

27 Feb 2012

What's new in 0.4.1
- Support Kinect tilt angle acquisition feature on IMAQ block.
- Added samples to show how to access Skeleton data.

22 May 2012

What's new in 0.4.2
- Support Kinect SDK for Windows v1.5 and its new features and more.

16 Aug 2012

What's new in 0.5.1
- Kinect sensor angle could be changed during simulation
- Added new parameters on IMAQ block

09 Nov 2012

What's new in 0.5.2
- Support Kinect SDK 1.6 new features
(Extended depth, Infrared camera)

03 Dec 2012

What's new in 0.6.0
- Just changed package name to Simulink Support for Kinect (formerly, called Simulink for NID)

11 Jun 2013

What's new in 0.6.1
- Support Simulink Coder packNGo feature

Contact us