I checked out a couple of projects out there, e.g. videoIO - but I didn't get lucky with it, so I did some very basic things on my own:
* Most stupid way of implementation
The source provides the slowest and most stupid way for video processing, but it works: It starts ffmpeg (must be installed) and writes pictures of every single frame into the dir /tmp/*cputimehash*. Then it reads the pictures iteratively. Video Information is extracted via 'aviinfo' if it is a avi.
* both MATLAB video object and uint8 matrix
The objects builds a MATLAB movie object with colormap and a convenient matrix holding all your pixel ready to process. There you can change pixels easily and convert it back (v = v.matrix2movie, v = v.movie2matrix)
* Fancy coverflow visualization
Yes, I seriously map single frames on 3d objects as a texture. There's nothing wrong about it. You can even zoom, drag and edit every object in there interactively! (see screenshot)
needs to have ffmpeg installed