Discover MakerZone

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn more

Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply Today

To resolve issues starting MATLAB on Mac OS X 10.10 (Yosemite) visit: http://www.mathworks.com/matlabcentral/answers/159016

Calling User32.dll mouse_event

Asked by Eelco Galestien on 16 Jun 2011

Hi,

I don't know if this is the right place to ask, but here goes:

I want to control the mouse using Matlab (version R2010B). I tried the java robot code, but it does not work in my application. It has to be done by calling the user32.dll. I can move the mouse using "setCursorPos". But now I also want to emulate a mouse click (left button only for now).

So basically my question is: How do I successfully call mouse_event in user32 to perform a left click ?

Here is my code so far:

clc
screenSize = get(0, 'screensize');
if(libisloaded('user32')==0)
      disp('loading dll...')
      loadlibrary('C:\WINDOWS\system32\user32.dll','user32.h');
  else
      disp('dll already loaded')
  end
calllib('user32','SetCursorPos',screenSize(3)/2,screenSize(4)/2); %center cursor on screen

So far so good. But when I add something like this at the end, it just crashes and I have to restart matlab:

MouseEventLeftDown = 2;
MouseEventLeftUp = 4;
calllib('user32','mouse_event',MouseEventLeftDown,0,0,0,0);
calllib('user32','mouse_event',MouseEventLeftUp,0,0,0,0);

Thanks in advance,

Eelco

0 Comments

Eelco Galestien

Products

4 Answers

Answer by Eelco Galestien on 18 Jul 2011
Accepted answer

Hi,

@jan: As I said, the java robot solution does not work in my application. I want to control the mouse in a video game. The game runs in full screen mode and somehow it overrides the java robot.

Anyway, the solution I came up with is this: First create a mex file, include "windows.h" and "winbase.h" Not sure if you really need both. They don't have to be in the Matlab working directory. Matlab should find them in windows/system.

Next pass some numbers inside the mex to the function SetCursorPos(x,y). X and Y should be int32 if I recall correctly. This positions the mouse at the pixel coordinate (x,y) on the screen. I used get(0, 'ScreenSize') in the m-file that calls the mex file because the video game changed the screen resolution on start up.

Last I used the process thread to simulate a mouse click. Inside the mex file I wrote: PostMessage(GetForegroundWindow(),WM_LBUTTONDOWN,0,0); Sleep(20); PostMessage(GetForegroundWindow(),WM_LBUTTONUP,0,0); Sleep(20);

If I understand correctly this posts the code for a mouse click in the message queue of the active window. I used postmessage, because I didn't want to wait for it to actually do so. I you want to wait until the message is actually posted, use sendmessage instead. WM_LBUTTONDOWN and UP are already defined in the header files so they are automatically the right type. GetForegroundWindow() is a pointer the the active window, the application inside which I wanted to click the mouse.

Compile the mex-file and call it from an m-file or from the workspace. The postmessage and sendmessage functions allow you to send anything to any process. It does not even need to be in focus. Just create a pointer to the application you want to control.

1 Comment

Eelco Galestien on 18 Jul 2011

forgot to mention, the Sleep(20); is very important. It gives the computer the necessary time to actually process your command. Without Sleep(20) you will get very strange behavior, or none at all.

Eelco Galestien
Answer by victoria vincze on 18 Jul 2011

Hy

I'm about to try to do something like this, but your solution doesn't work for me. It cannot find user32.h. Does somebody know how can I solve this?

V

2 Comments

Jan Simon on 18 Jul 2011

Did you install a C-compiler?

victoria vincze on 20 Jul 2011

Of Course.

I've a mex file, which sets the cursor, but it also works incorrectly cause it sets the cursor on the left upper corner instead of the center of the screen.

victoria vincze
Answer by Chirag Gupta on 18 Jul 2011

I haven't looked at user32.h, but I will take a guess with checking the function syntax in the header file for mouse_event.

When you pass MouseEventLeftDown = 2; you are actually passing a double value (MATLAB defaults to double) [Check using whos]. If the mouse_event expects an int32, it will crash. Try and cast the value to the type expected.

calllib('user32','mouse_event',int32(MouseEventLeftDown),0,0,0,0); 

2 Comments

Jan Simon on 18 Jul 2011

@Chirag: But the last argument is a ULONG_PTR. Does the DOUBLE 0 match this on 32-bit Matlab's also?

Chirag Gupta on 18 Jul 2011

@Jan: I haven't seen the header file. But if it's a ULONG_PTR then the best course is to create a libpointer of type uint32.

Chirag Gupta
Answer by deng on 19 May 2013
Edited by Walter Roberson on 19 May 2013
clc
screenSize = get(0, 'screensize');
if(libisloaded('user32')==0)
    disp('loading dll...')
    loadlibrary('C:\WINDOWS\system32\user32.dll','user32.h');
else
    disp('dll already loaded')
end
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
loading dll...
??? Error using ==> loadlibrary>lFullPath at 553
Could not find file user32.h.
Error in ==> loadlibrary at 221
        header=lFullPath(header);
Error in ==> matlabdiaoyongwindowshanshu at 5
    loadlibrary('C:\WINDOWS\system32\user32.dll','user32.h');

1 Comment

Walter Roberson on 19 May 2013

Which operating system are you using, and are you running the operating system in 32 bit or 64 bit mode? Are you using 32 bit MATLAB or 64 bit?

deng

Contact us