I need to analyse the frames of a video along with the audio of the said frames. VideoFileReader's `step' function has worked nicely but when I try to use it with an mp4 file with AAC-encoded audio it gives weird results (the audio frames are 6 columns wide and don't really seem to represent any audio). If I convert the video to WMV things are good. If I use `audioread' the AAC-encoded audio is read fine.
Is there a way to make VideoFileReader work with AAC, and if not, what is the easiest way to analyse audio and video frame-by-frame? Converting the video each time would be highly impractical for me.