How to convert optical flow to feature matrix in videos

1 view (last 30 days)
I want to classify videos using optical flow.
After using this code, I got this result.
Which data can I save as matrix for classification?
I think that I can use optical flow data, but there are 4 matrix(Vx, Vy, Orientation, Magnitude). How can I deal with these data?
I will save the data of opticalFlow and use dimensionality reduction. However, I don't know which property can be used for this process.
clear all
close all
vidReader = VideoReader('shahar_walk.avi');
opticFlow = opticalFlowFarneback
h = figure;
hViewPanel = uipanel(h,'Position',[0 0 1 1],'Title','Plot of Optical Flow Vectors');
hPlot = axes(hViewPanel);
while hasFrame(vidReader)
frameRGB = readFrame(vidReader);
frameGray = rgb2gray(frameRGB);
flow = estimateFlow(opticFlow,frameGray);
hold on
plot(flow,'DecimationFactor',[5 5],'ScaleFactor',2,'Parent',hPlot);
hold off

Accepted Answer

Raunak Gupta
Raunak Gupta on 30 Apr 2020
I am unsure about what you want to classify in the video using the optical flow estimation workflow. As per my understanding the above code estimate the direction and magnitude of the movement of any pixel in two consecutive frames. So essentially you can use the algorithm to tell if any object which consists of many pixels is moving or not.
For this you may use any of the pair (Vx,Vy) or (Orientation,Magnitude) as a reference to check if an object is moving. These two pairs are related as for every pixel in the image,
Orientation = atan(Vy./Vx);
Magnitude = sqrt(Vx.^2 + Vy.^2);
So basically, you may cluster the points which have same (Vx,Vy) in a region that might represent an object because for a single object its velocity for every point will remain same.
I would suggest saving any of the pair (Vx,Vy) or (Orientation,Magnitude) for dimensionality reduction as that will bring all the data available from optical flow estimation.
  1 Comment
耶 王
耶 王 on 6 Jan 2021
Hello, how to cluster after getting the optical flow vector of each pixel by dense optical flow method? Thank you very much.

Sign in to comment.

More Answers (0)

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!