AAC Audio and VideoFileReader compatibility

Hello!

I need to analyse the frames of a video along with the audio of the said frames. VideoFileReader's `step' function has worked nicely but when I try to use it with an mp4 file with AAC-encoded audio it gives weird results (the audio frames are 6 columns wide and don't really seem to represent any audio). If I convert the video to WMV things are good. If I use `audioread' the AAC-encoded audio is read fine.

Is there a way to make VideoFileReader work with AAC, and if not, what is the easiest way to analyse audio and video frame-by-frame? Converting the video each time would be highly impractical for me.

Many Thanks,

Joachim

[Walter Roberson]

Is it possible that the .AAC is in Dolby 5.1 format?

[Joachim]

It's stereo, but even if it was 5.1, the audio that VideoFileReader gives me is shown in the image on this link http://imgur.com/gbQmpjS, when I use the converted .wmv file, this image shows up http://imgur.com/1grnBl6. I wonder if what I get actually represents any audio...

videoFReader = vision.VideoFileReader('B_r3_livroom.wmv','AudioOutputPort',true);

videoPlayer = vision.VideoPlayer;

allAudio = [];

while ~isDone(videoFReader)  %until the end of videoFReader object
  [videoFrame,audioFrame] = step(videoFReader); % takes vF and the multiple aF for each frame and moves forward
  allAudio = [allAudio; audioFrame];
i=i+1;
end
  plot(allAudio)

[Walter Roberson]

What does wavfinfo() say about the file?

[Joachim]

[m d] = wavfinfo('B_r3_livroom.MP4');
>> d

d =

Not a WAVE file

>> m

m =

       ''

[Walter Roberson]

Okay, how about http://www.mathworks.com/help/matlab/ref/mmfileinfo.html ?

[Joachim]

Below is the output, it might be a red herring since it's not recognizing the video type for the .mp4 file even if videofilereader reads the video just fine.

I also tried it with .wmv, which works fine here...

info = mmfileinfo('B_r3_livroom.MP4')

info = 

      Filename: 'B_r3_livroom.MP4'
          Path: 'C:\Users\Joachim Horn\Desktop\Image Processing\D Video Hit Capture and Analysis'
      Duration: 3.2032
         Audio: [1x1 struct]
         Video: [1x1 struct]

>> info.Audio

ans = 

                Format: ''
      NumberOfChannels: []

>> info.Video

ans = 

      Format: ''
      Height: []
       Width: []

>> info = mmfileinfo('B_r3_livroom.wmv')

info = 

      Filename: 'B_r3_livroom.wmv'
          Path: 'C:\Users\Joachim Horn\Desktop\Image Processing\D Video Hit Capture and Analysis'
      Duration: 3.2040
         Audio: [1x1 struct]
         Video: [1x1 struct]

>> info.Audio

ans = 

                Format: 'WMAUDIO'
      NumberOfChannels: 2

[Walter Roberson]

Hmmm... When I look at VideoFileReader it is not clear that it supports AAC audio on any system. It does support H.264 video for MPEG-4 but that is a video standard, not an audio standard, and AAC is not listed under audio for any platform.

I am not sure which tool you could use to look at the audio channel information. Possibly Windows Explorer file properties might be able to tell you, or possibly QuickTime.

[Joachim]

Hello Walter, here's the response from Matlab support:

"

The ability to decode audio from an MP4 video file is not available in Computer Vision System Toolbox 5.2. Our development team are aware of this issue and are working on a solution for a future release.

Unfortunately there is no work-around for this issue other than to convert the video into a different format.

Having had a search on MATLAB Central I have found an entry, M4AREAD, which decodes the AAC audio from an .MP4 file using an open-source AAC decoder. This can be found at the following URL:

http://www.mathworks.co.uk/matlabcentral/fileexchange/28745-m4aread-read-mpeg4-aac-or-m4a-compressed-audio-files

"

Thank you for your comments and answers!