Code covered by the BSD License  

Highlights from
High accuracy optical flow

4.07143

4.1 | 15 ratings Rate this file 114 Downloads (last 30 days) File Size: 1.71 MB File ID: #17500
image thumbnail

High accuracy optical flow

by

 

13 Nov 2007 (Updated )

High accuracy optical flow using a theory for warping

| Watch this File

File Information
Description

This code implements two variations of the paper
"High accuracy optic flow using a theory for warping" presented at ECCV 2004 by Dr. Thomas Brox.

Its variants appeared in "Particle Video"
presented at CVPR 2006 by Dr. Peter Sand.

The files optical_flow_brox.m and
optical_flow_sand.m are the files you need to run.

optical_flow_brox.m implements the traditional algorithm. Just replace the first lines where images are read, with your image, and start the process.

optical_flow_sand.m implements a variant presented in CVPR 2006. Again just replace the lines "img1 = imread('...')" and "img2 = imread('...')" with your images and the rest is automatic.

The variable num_levels controls the number of images in the laplacian pyramid to be constructed. Change it to suit your convenience.

The way to tweak / use the code is described in the documentation pdf that is attached with the code. It can also be accessed from

http://perception.inrialpes.fr/~chari/myweb/Software/

In order to tweak the code, however, you need to read the two papers...

Required Products Image Processing Toolbox
MATLAB release MATLAB 7 (R14)
Tags for This File   Please login to tag files.
Please login to add a comment or rating.
Comments and Ratings (35)
20 Jun 2012 Andy

Well done.

17 May 2012 Kassem Al Ismaeil

Hey,
Just I was wondering if this code could work for a grays-cale images, or there is another version of the code. what I am doing now is to convert the gray images to rgb and then estimate motion. I want to save more time by doing it directly on the gray images if possible, thanks for the help.

03 May 2011 shreyas

Hey,
Its a really nice piece of work, I am getting stuck at one part because, I want to reduce the number of points/resolution for which optical flow is computed,I believe that this will make the code faster.
Would be grateful,if anyone who has done this provides me with some help.

14 Mar 2011 rh_sh

Hi all,I have a question.I compute the optical flow(u ,v) using LK and HS method. Now I want to compute the average angular error(AAE).But I don't know how.Do I need the ground truth optical flow (in .pcm file format)for compare the flow ?
Or any other way to compute the angular error?
thank u all.

30 Mar 2010 Omid Aghazadeh

I have made some minor modifications to your implementation of Brox's algorithm:
- Reflecting boundary conditions corresponds to mirror-extending the images. This can remove some of "unwanted" spurious flows at image boundaries.
- Added color channels (and different color spaces)
- 500 SOR Iterations is too much! If I remember correctly, Brox mentions much less SOR iterations. You can safely decrease it to around 50(or even less).

Let me know if you want me to send the modified code to you(Visesh).

Regarding the higher errors on the results:
- If I'm not mistaken, this implementation does not assume spatio-temporal smoothness(it's not 3D) and does not work on more than 2 images . In order to get nicer results you have to extend it to the 3D case.

I recently realized that Brox has already put his binaries(both for the ECCV04 and CVPR09 papers) online!

20 Mar 2010 Brian

Thank you for this implementation. It is valuable in an educational way. But given that the bugs mentioned above are still in this package this gets only 3 stars from me. Also, you might be interested in this implementation which is a lot faster and produces very similar results:
http://people.csail.mit.edu/celiu/OpticalFlow/

22 Feb 2010 Hu Yu

Very usefull tool ! I'm looking forward to a MEX version.

30 Oct 2009 Visesh Chari

Hi Johan,

I have the code, but its not documented yet... I could give it to you if you want to have a look, but I haven't thoroughly proofread it for bugs, and might not have time before december....

10 Oct 2009 Johan

Visesh, you mentioned a MEX version of the algorithm. Have you made it available somewhere? I'm trying to run it on rather large images, so I could appreciate some improved speed. Thanks.

01 Sep 2009 tlg  
31 Aug 2009 tlg

Hi. First of all thanks for your share.
I am trying to understand your code and I am almost finished but 'constructMatrix' function is still waiting to be explored. Could you, if possible, give some explanations on construction of the matrices in this script.

29 Jul 2009 Omid Aghazadeh

Great work. Thanks for sharing. Haven't gone through the documentation yet, but the preset configurations worked reasonably well.

01 Jul 2009 Naveen

Really great work!
I am using this code for few of my images. I am not able to get the (u, v) of the same dimension as that of input images. I am getting (u,v) equal to the next resolution in the pyramid dependng on the scale_factor I give. For Instance, If I am giving two 120*120 images, scale_factor = 0.5, no. of levels = 3, I am getting final output (u, v) of the resolution 60*60. How to obtain (u,v) in the resolution of input images. Am I wrong in specifying some parameters?

31 May 2009 De

Great Piece of Work !!!!

27 May 2009 Ayaz Ahmed Shaikh

Great work!! Hi Vishesh, When I run the simulation of brox, in the lower left corner of quiver plot, there is some additional unwanted vector fields, i want to cope with them, so do you have any idea how to cope and why the are occuring?

01 May 2009 Assaf laufer

Great! do you have the implementation in c?

22 Mar 2009 Mo Star

Thanks for sharing. very well documented and extermely useful.

04 Mar 2009 Visesh Chari

can you tell me what doesnt work ? i have received no complains so far... may be you did not read the reference document i sent... if possible post the error so that i can check

29 Jan 2009 UMD Mishra

Brox implementation does not work. There is something wrong with it..

05 Nov 2008 Assaf cohen

Thank you, i am waiting. can't i resize u and v a little bit to be bigger? in the end, the original image is 148*148 and the im2_hr is a little bit less, so i want to resize the u and v and then warp the original image?

05 Nov 2008 Visesh Chari

Hi, the basic smoothing operation that is done in the algorithm is to reduce the effects of noise in the data, so it cannot be avoided. If you really have noiseless data, and do not need much smoothing, you can change the appropriate parameters. You can refer to my documentation, but will eventually have to tweak the code to reduce the variance of the gaussian used for smoothing. If this is not too urgent (u can wait a week or so), I should be able to post additional documentation material, which you can use...

03 Nov 2008 Assaf cohen

Hi, i have a question: the registration is GREAT, but aventully i get an image that is not as "focused" as the original image, it looks like filtered with gaussian filter... so, the question is how can i obtain aventually the original imag warped, and not the im2_hr ? thank you very much

01 Nov 2008 Visesh Chari

it should work, so long as u and v are formatted like those in my code, which is just two 2D arrays with flow values....

30 Oct 2008 Assaf cohen

Hello i have a question, can i use the mywarp() function for any U,V i have that wasn't comuted by the algorithm? i mean, U,V are calculated not through this algorithm, but i want to use the warp function, will it work? or this is specially designed for this algorithm?

19 Sep 2008 shifang xu

It is a very usefull demo as a state-of-the-art optical flows method. Thanks for sharing it!

27 Jun 2008 visesh chari

Hi all,
Thanks for your comments. I am sorry to have posted a wrong version of the code, hence the wrong limits for inner and outer iterations. I did not notice that while uploading. Thanks for pointing it out. I will upload the corrected version in some time.

You are right about the parameters. They are very important, and have to be carefully tuned to get the desired results, as per the author.

The algorithm I have coded differs from the original formulation in some of the boundary conditions and smoothing operations.. (will post a documentation about it in the next 2 months). This is possibly the reason why the figures reported in the literature are not achieved by this code. Otherwise the code follows the algorithm faithfully, to my knowledge.

Also, do watch this space for a MEX version of the above algorithm. Its faster than the current version, with exactly the same results.

07 Jun 2008 Hui Fang

Although there are a couple of bugs in it, it's a good implementation anyway. You need fix the bugs when you are trying to use it. The two parameters,the smoothness one and the gradient one, are very important.When I did an experiment for face tracking and I set the smoothness weight to a big value or a small value, the error is unbearable.But finally I got a reasonable result when the parameter of this one and the gradient one are set suitable for my sequence.

05 Jun 2008 Konrad S.

Christian,

on Yosemite I get the Sand&Teller version to an angular error of AAE=3.4 degrees and endpoint error of AEP=0.17 pixels, excluding a 10 pixel border (like most evaluations do to limit mean boundary effects). This is after fixing the bug mentioned in your posting, and setting parameters

alpha_global=5; alpha_local=10;
outer_iter = 20; inner_iter = 20;
mu = 0.95;
% mu ... pyramid scale factor, hardcoded in original source

The errors are not quite as low as those reported by Brox, but competitive with other recent entries in the Middlebury flow evaluation.

cheers,
Konrad

16 May 2008 Christian B.

First of all, it is a great relief that finally someone published an implementation to some state-of-the-art optical flows.

However, I am a little bit curious about your implementation.

First there are some bugs in it.
In the Sand-Version your most outer loop does not rescale your solution to the initial resolution of the input images, as your loop variable stops at 8 but should go to 1.

A similar bug is in the brox version, where i stops at 1 but should stop at 0.

I tried your implementation on the yosemite sequence but I got horrible results. Using the sand-version I got an angular error of 12 degree and using the brox-version I got an angular error of 20, which is one magnitude larger than it should be!
I would be interested in the explanation for this, because it is rather strange.
Did you already make tests with the yosemite sequence by yourself? Should one choose other parameters maybe? If so, what parameters work well for this sequence.
Without comparable angular errors it is hard to decide, whether your implementation is correct.

06 Feb 2008 Visesh Chari

Hi Samuel,

I am using Matlab 7.0.0.19901 (R14). In this
case the line of code I wrote and the one
you have written given the same results.

Is there something I am not following ? Is there a difference for different versions ? I do not have access to other matlab versions so I cannot be sure. Please let me know if I am missing something...

Thanks,
Visesh

01 Feb 2008 Samuel Foucher

I think there is a bug line in gaussianRescaling line 31.

This line:

img_scaled = imresize(img_smooth, scale_factor, 'bilinear', 0);

should be:

img_scaled = imresize(img_smooth, [fix(size(img_smooth,1)*scale_factor) fix(size(img_smooth,2)*scale_factor)], 'bilinear', 0);

20 Jan 2008 mathild L.

I am to some extent friendly with different optical flow and warping techniques and I think that this code is worth a PhD project.
The algorithm makes use of high mathematical background.
I myself could understand the flow of the codes in less than an hour without reading help!
Ofcourse I do not recommend this code for those who are novice in the field of motion estimation and registration while the algorithm relies on newly developed penalizations and warping strategies.

Thanks for sharing this, Visesh!
Five stars from me!!

15 Nov 2007 visesh chari

Hi Mr Moisy,

Thank you very much for your comments. Have incorporated all of them in the current version.

V.

15 Nov 2007 F Moisy

OK, I have seen the (little) help of the revised version of this submission, thanks ; however I think it should still be improved to be really useful to other users:

a) changing the filenames directly in the script is awful: you can't automatize the processing for several files! Just make it a function, in the form M = optic_flow(IM1, IM2)

b) It still does not work for me. optic_flow_sand returns the error message "??? Undefined function or method 'getalphaImg' for input arguments of type 'uint8'." and optic_flow_brox returns "??? Undefined function or method 'split' for input arguments of type 'double'." (I use Matlab 7.5).

c) I understand that for copyright reasons you cannot include the two refered papers into the submission; however you should explain a little bit what does each algorithm.

F.

14 Nov 2007 F Moisy

How to get started? Please provide some help and examples! I was not present at the ECCV 2004 conference so I don't know how to start with those 20 files. As it stands, this submission is useless.

Updates
14 Nov 2007

I am adding a screenshot of the results of the current algorithm/code.

14 Nov 2007

Updating information on how to get started.

15 Nov 2007

Added a some documentation.

23 Jun 2009

Updated with license removed

Contact us