Discover MakerZone

MATLAB and Simulink resources for Arduino, LEGO, and Raspberry Pi

Learn more

Discover what MATLAB® can do for your career.

Opportunities for recent engineering grads.

Apply Today

Thread Subject:
Detecting Forward/Backward vs Lateral movement from a Continuous Video Stream

Subject: Detecting Forward/Backward vs Lateral movement from a Continuous Video Stream

From: D V

Date: 27 Apr, 2010 16:43:04

Message: 1 of 10

Hey Everyone!

I'm interested in mounting a webcam on a moving object (of which i have no control) and detecting the direction of that object's movement from the continuous video stream. I'd like to separately detect the magnitude of lateral (left/right & up/down) motion & optic flow (forward/backwards) motion.

One strategy I've tried is just taking doing diff(frame(i) - frame(i-1)) so see where the pixel by pixel change, but I'm essentially just getting edge detection and I can't think of a way to separate this difference into lateral vs forward/backwards. I thought maybe an algorithm with locates the focus of expansion and I could asses forward/backwards strength based on how confidently it identifies that center?

I've looked at some of the motion detection m-files posted by users in Matlab Central, but most of them deal with object tracking, something which I do not need. I'm just looking for a way to detect the magnitude of lateral change and magnitude of forward/backwards change (to a scaling factor, of course).

I've also cursorily looked at the motion detection literature but much of it is on object detection as well.

Excited for your help!

best,

Don

Subject: Detecting Forward/Backward vs Lateral movement from a Continuous

From: ImageAnalyst

Date: 27 Apr, 2010 20:05:14

Message: 2 of 10

Don:
I thought you said that you could do optical flow. That, to me, seems
the way to go (if it's fast enough for you - might have to operate on
a reduced resolution image to get the speed up).
ImageAnalyst

Subject: Detecting Forward/Backward vs Lateral movement from a Continuous

From: D V

Date: 27 Apr, 2010 20:42:05

Message: 3 of 10

ImageAnalyst <imageanalyst@mailinator.com> wrote in message <397ad22a-6038-4c74-b455-50cf779e77bd@b33g2000yqc.googlegroups.com>...
> Don:
> I thought you said that you could do optical flow. That, to me, seems
> the way to go (if it's fast enough for you - might have to operate on
> a reduced resolution image to get the speed up).
> ImageAnalyst

Hi ImageAnalyst

I'm aware that there are algorithms for detecting optic flow and the focus of expansion, but I'm not aware of how to actually do it. Do you have any suggestions or particular methods you find to work the best? Also, if I am able to detect optic flow, how would I then detect lateral motion? Since movement is a combination of lateral movements and forward/backwards motion most of the time, I'd like to calculate optic flow and then somehow calculate lateral motion as well.

Thanks for the response!

best,

Don

Subject: Detecting Forward/Backward vs Lateral movement from a Continuous Video Stream

From: Jerry Mouse

Date: 27 Apr, 2010 20:59:07

Message: 4 of 10

"D V" <donaldavaughn@gmail.com> wrote in message <hr746o$hoi$1@fred.mathworks.com>...
> Hey Everyone!
>
> I'm interested in mounting a webcam on a moving object (of which i have no control) and detecting the direction of that object's movement from the continuous video stream. I'd like to separately detect the magnitude of lateral (left/right & up/down) motion & optic flow (forward/backwards) motion.
>
> One strategy I've tried is just taking doing diff(frame(i) - frame(i-1)) so see where the pixel by pixel change, but I'm essentially just getting edge detection and I can't think of a way to separate this difference into lateral vs forward/backwards. I thought maybe an algorithm with locates the focus of expansion and I could asses forward/backwards strength based on how confidently it identifies that center?
>
> I've looked at some of the motion detection m-files posted by users in Matlab Central, but most of them deal with object tracking, something which I do not need. I'm just looking for a way to detect the magnitude of lateral change and magnitude of forward/backwards change (to a scaling factor, of course).
>
> I've also cursorily looked at the motion detection literature but much of it is on object detection as well.
>
> Excited for your help!
>
> best,
>
> Don

Hello,
as soon as I know, it is grail of all scientist in AI .... to solve such problem.... gut luck !

Subject: Detecting Forward/Backward vs Lateral movement from a Continuous Video Stream

From: D V

Date: 27 Apr, 2010 22:00:19

Message: 5 of 10

Yeah, it seems like a holy grail of the field, but I'm not looking for a perfect algorithm, just something that works decently in determining motion direction. Are there no m-files or algorithms that can be implemented in MATLAB that I could use?

"Jerry Mouse" <jerry.mouse.email@seznam.cz> wrote in message <hr7j6r$i08$1@fred.mathworks.com>...
> "D V" <donaldavaughn@gmail.com> wrote in message <hr746o$hoi$1@fred.mathworks.com>...
> > Hey Everyone!
> >
> > I'm interested in mounting a webcam on a moving object (of which i have no control) and detecting the direction of that object's movement from the continuous video stream. I'd like to separately detect the magnitude of lateral (left/right & up/down) motion & optic flow (forward/backwards) motion.
> >
> > One strategy I've tried is just taking doing diff(frame(i) - frame(i-1)) so see where the pixel by pixel change, but I'm essentially just getting edge detection and I can't think of a way to separate this difference into lateral vs forward/backwards. I thought maybe an algorithm with locates the focus of expansion and I could asses forward/backwards strength based on how confidently it identifies that center?
> >
> > I've looked at some of the motion detection m-files posted by users in Matlab Central, but most of them deal with object tracking, something which I do not need. I'm just looking for a way to detect the magnitude of lateral change and magnitude of forward/backwards change (to a scaling factor, of course).
> >
> > I've also cursorily looked at the motion detection literature but much of it is on object detection as well.
> >
> > Excited for your help!
> >
> > best,
> >
> > Don
>
> Hello,
> as soon as I know, it is grail of all scientist in AI .... to solve such problem.... gut luck !

Subject: Detecting Forward/Backward vs Lateral movement from a Continuous Video Stream

From: Jerry Mouse

Date: 28 Apr, 2010 07:09:06

Message: 6 of 10

Hi lucky man :-),

I have NO knowledge of such library in Matlab. I can recommend to you several scientific papers:

*SEEING THE WORLD BEHIND THE IMAGE, Spatial Layout for 3D Scene Understanding, Derek Hoiem
*A Global, Camera-based Mobile Robot Localization, István Engedy, Gábor Horváth
*Recovering Occlusion Boundaries from a Single Image, Derek Hoiem∗ Andrew N. Stein *Alexei A. Efros Martial Hebert, Robotics Institute
Carnegie Mellon University
*COARSE GLOBAL VISUAL LOCALIZATION OF A MOBILE ROBOT, Almasi Maguya

You can get more if you can try to find "Visual SLAM" or "visual robot localization" theme in GOOGLE.
 
For video recognition in this way you need a special hardware so called programmable autocorrelation processor for PCI-e bus:

http://www.robotics.hiroshima-u.ac.jp/hyper_human_vision/index-e.html

If you have NO such HW you are not able to work with video stream.

Good luck unfortunate man ! :-))

Subject: Detecting Forward/Backward vs Lateral movement from a Continuous Video Stream

From: David Young

Date: 28 Apr, 2010 10:06:05

Message: 7 of 10

This is quite a complex question, and as has been indicated, there is a huge literature on it. You probably need to make the assumption that only the robot is moving, the rest of the world being fixed, and it's also important to think about what information you really need to extract from the images. A very important question is whether or not your camera (or what it is mounted on) is subject to rotation, and if so whether or not you have any way of estimating the rate of rotation.

Anyway, here is a file exchange submission which might help you:

http://www.mathworks.com/matlabcentral/fileexchange/27093-affine-optic-flow

It estimates the overall first-order optic flow for an image pair, without doing tracking. The dilation component can (subject to some assumptions) tell you something about forward/backward motion. If your camera does not rotate, the x and y components will tell you something about sideways motion - it sounds like these are the things you want to measure.

Subject: Detecting Forward/Backward vs Lateral movement from a Continuous

From: ImageAnalyst

Date: 28 Apr, 2010 11:48:01

Message: 8 of 10

On Apr 27, 6:00 pm, "D V" <donaldavau...@gmail.com> wrote:
> Yeah, it seems like a holy grail of the field, but I'm not looking for a perfect algorithm, just something that works decently in determining motion direction.  Are there no m-files or algorithms that can be implemented in MATLAB that I could use?
----------------------------------------------------------------------------------------------

I personally haven't used any optical flow code, so all I can do is
give you a link on a professor (John Barron) who seems to be one of
the renowned experts in the field:
http://www.csd.uwo.ca/faculty/barron/
Lots of interesting applications there on his site.
Good luck.

Subject: Detecting Forward/Backward vs Lateral movement from a Continuous Video Stream

From: Jerry Mouse

Date: 28 Apr, 2010 18:00:21

Message: 9 of 10


... by the way... if you have money enough... send me email... I will help you with this problem .... but, you still need the correlation processor... :-)
I am able to do it in CLI/C++...

jerry.mouse.email@seznam.cz

Subject: Detecting Forward/Backward vs Lateral movement from a Continuous Video Stream

From: David Young

Date: 28 Apr, 2010 21:46:04

Message: 10 of 10

"Jerry Mouse" <jerry.mouse.email@seznam.cz> wrote in message <hr9t3l$82k$1@fred.mathworks.com>...
> .... but, you still need the correlation processor... :-)

I'm not sure that this is necessarily correct. It depends on many things, like the frame rate and the image resolution. It also depends very much on whether one is measuring

(a) lots of optic flow vectors, for a set of different points in the image

or

(b) some parameters of a model of the overall flow field, e.g. the first-order flow parameters (two components of translation, the dilation rate, the rotation rate and two components of shear rate).

It's possible to do (b) much faster than (a). Try some timing tests on the file exchange submission referenced above! What I think the original post is asking for is just (b).

Tags for this Thread

What are tags?

A tag is like a keyword or category label associated with each thread. Tags make it easier for you to find threads of interest.

Anyone can tag a thread. Tags are public and visible to everyone.

Contact us