Importing Audio and Video

Get Information about Audio or Video Files

To get information about files that contain audio data, video data, or both, use mmfileinfo. The mmfileinfo function returns the duration, format, number of audio channels, and height and width of video, as applicable.

To get more information about files that contain only video data, such as the number of frames, create a multimedia object with VideoReader and use the get method. For more information, see Getting Information about Video Files.

Characteristics of Audio Files

The audio signal in a file represents a series of samples that capture the amplitude of the sound over time. The sample rate is the number of discrete samples taken per second and given in hertz. The precision of the samples, measured by the bit depth (number of bits per sample), depends on the available audio hardware.

MATLAB audio functions read and store single-channel (mono) audio data in an m-by-1 column vector, and stereo data in an m-by-2 matrix. In either case, m is the number of samples. For stereo data, the first column contains the left channel, and the second column contains the right channel.

Typically, each sample is a double-precision value between -1 and 1. In some cases, particularly when the audio hardware does not support high bit depths, audio files store the values as 8-bit or 16-bit integers. The range of the sample values depends on the available number of bits. For example, samples stored as uint8 values can range from 0 to 255 (2⁸ – 1). The MATLAB sound and soundsc functions support only single- or double-precision values between -1 and 1. Other audio functions support multiple data types, as indicated on the function reference pages.

Back to Top

Read Audio Files

The easiest way to read audio data from a file is to use the Import Wizard, a graphical user interface. The Import Wizard can read WAV, AU, or SND files. To start the Import Wizard, select File > Import Data or double-click the file name in the Current Folder browser. To import WAV files without invoking a graphical user interface, use wavread.

Back to Top

Record and Play Audio

This section discusses the following topics:

• Record Audio

• Play Audio

• Recording or Playing Audio within a Function

Record Audio

To record data from an audio input device (such as a microphone connected to your system) for processing in MATLAB:

1. Create an audiorecorder object.

2. Call the record or recordblocking method, where:

   • record returns immediate control to the calling function or the command prompt even as recording proceeds. Specify the length of the recording in seconds, or end the recording with the stop method. Optionally, call the pause and resume methods.

   • recordblocking retains control until the recording is complete. Specify the length of the recording in seconds.

3. Create a numeric array corresponding to the signal data using the getaudiodata method.

For example, connect a microphone to your system and record your voice for 5 seconds. Capture the numeric signal data and create a plot:

% Record your voice for 5 seconds.
recObj = audiorecorder;
disp('Start speaking.')
recordblocking(recObj, 5);
disp('End of Recording.');

% Play back the recording.
play(recObj);

% Store data in double-precision array.
myRecording = getaudiodata(recObj);

% Plot the samples.
plot(myRecording);

Specifying the Quality of the Recording.  By default, an audiorecorder object uses a sample rate of 8000 hertz, a depth of 8 bits (8 bits per sample), and a single audio channel. These settings minimize the required amount of data storage. For higher quality recordings, increase the sample rate or bit depth.

For example, typical compact disks use a sample rate of 44,100 hertz and a 16-bit depth. Create an audiorecorder object to record in stereo (two channels) with those settings:

myRecObj = audiorecorder(44100, 16, 2);

For more information on the available properties and values, see the audiorecorder reference page.

Play Audio

After you import or record audio, MATLAB supports several ways to listen to the data:

• For simple playback using a single function call, use sound or soundsc. For example, load a demo MAT-file that contains signal and sample rate data, and listen to the audio:

  load chirp.mat;
  sound(y, Fs);

• For more flexibility during playback, including the ability to pause, resume, or define callbacks, use the audioplayer function. Create an audioplayer object, then call methods to play the audio. For example, listen to the gong demo:

  load gong.mat;
  gong = audioplayer(y, Fs);
  play(gong);

  For an additional example, see Recording or Playing Audio within a Function.

If you do not specify the sample rate, sound plays back at 8192 hertz. For any playback, specify smaller sample rates to play back more slowly, and larger sample rates to play back more quickly.

Note   Most sound cards support sample rates between approximately 5,000 and 48,000 hertz. Specifying sample rates outside this range can produce unexpected results.

Recording or Playing Audio within a Function

Unlike graphics handles, if you create an audioplayer or audiorecorder object inside a function, the object exists only for the duration of the function. For example, create a player function called playFile and a simple callback function showSeconds:

function playFile(myfile)
   load(myfile);
   
   obj = audioplayer(y, Fs);
   obj.TimerFcn = 'showSeconds';
   obj.TimerPeriod = 1;
   
   play(obj);
end

function showSeconds
   disp('tick')
end

Call playFile from the command prompt to play the demo file handel.mat:

playFile('handel.mat')

At the recorded sample rate of 8192 samples per second, playing the 73113 samples in the file takes approximately 8.9 seconds. However, the playFile function typically ends before playback completes, and clears the audioplayer object obj.

To ensure complete playback or recording, consider the following options:

• Use playblocking or recordblocking instead of play or record. The blocking methods retain control until playing or recording completes. If you block control, you cannot issue any other commands or methods (such as pause or resume) during the playback or recording.

• Create an output argument for your function that generates an object in the base workspace. For example, modify the playFile function to include an output argument:

  function obj = playFile(myfile)

  Call the function:

  h = playFile('handel.mat');

  Because h exists in the base workspace, you can pause playback from the command prompt:

  pause(h)

Back to Top

Read Video Files

To import video data from a file, construct a reader object with VideoReader and read selected frames with the read function.

For example, import all frames in the demo file xylophone.mpg:

xyloObj = VideoReader('xylophone.mpg');
vidFrames = read(xyloObj);

It is not necessary to close the multimedia object.

For more information, see:

• Getting Information about Video Files

• Processing Frames of a Video File

• Reading Variable Frame Rate Video

• Supported Video File Formats

Getting Information about Video Files

VideoReader creates an object that contains properties of the video file, including the duration, frame rate, format, height, and width. To view these properties, or store them in a structure, use the get method. For example, get the properties of the demo file xylophone.mpg:

xyloObj = VideoReader('xylophone.mpg');
info = get(xyloObj)

The get function returns:

info = 
          Duration: 4.7020
              Name: 'xylophone.mpg'
              Path: [1x75 char]
               Tag: ''
              Type: 'VideoReader'
          UserData: []
      BitsPerPixel: 24
         FrameRate: 29.9700
            Height: 240
    NumberOfFrames: 141
       VideoFormat: 'RGB24'
             Width: 320

To access a specific property of the object, such as Duration, use dot notation as follows:

duration = xyloObj.Duration;

Note   For files with a variable frame rate, VideoReader cannot return the number of frames until you read the last frame of the file. For more information, see Reading Variable Frame Rate Video.

Processing Frames of a Video File

A typical video contains many frames. To save memory, process a video one frame at a time. For faster processing, preallocate memory to store the video data.

For example, convert the demo file xylophone.mpg to a MATLAB movie, and play it with the movie function:

xyloObj = VideoReader('xylophone.mpg');

nFrames = xyloObj.NumberOfFrames;
vidHeight = xyloObj.Height;
vidWidth = xyloObj.Width;

% Preallocate movie structure.
mov(1:nFrames) = ...
    struct('cdata', zeros(vidHeight, vidWidth, 3, 'uint8'),...
           'colormap', []);

% Read one frame at a time.
for k = 1 : nFrames
    mov(k).cdata = read(xyloObj, k);
end

% Play back the movie once at the video's frame rate.
movie(mov, 1, xyloObj.FrameRate);

Reading Variable Frame Rate Video

Some files store video at a variable frame rate, including many Windows Media Video files. For these files, VideoReader cannot determine the number of frames until you read the last frame.

For example, consider a hypothetical file VarFrameRate.wmv that has a variable frame rate. A call to VideoReader to create the multimedia object such as

obj = VideoReader('VarFrameRate.wmv')

returns the following warning and summary information:

Warning: Unable to determine the number of frames in this file.

Summary of Multimedia Reader Object for 'VarFrameRate.wmv'.

  Video Parameters:  23.98 frames per second, RGB24 1280x720.
                     Unable to determine video frames available.

Counting Frames.  To determine the number of frames in a variable frame rate file, call read with the following syntax:

lastFrame = read(obj, inf);

This command reads the last frame and sets the NumberOfFrames property of the multimedia object. Because VideoReader must decode all video data to count the frames reliably, the call to read sometimes takes a long time to run.

Specifying the Frames to Read.  For any video file, you can specify the frames to read with a range of indices. Usually, if you request a frame beyond the end of the file, read returns an error.

However, if the file uses a variable frame rate, and the requested range straddles the end of the file, read returns a partial result. For example, given a file with 2825 frames associated with the multimedia object obj, a call to read frames 2800 - 3000 as follows:

images = read(obj, [2800 3000]);

returns:

Warning: The end of file was reached before the 
requested frames were read completely.  
Frames 2800 through 2825 were returned. 

Supported Video File Formats

The VideoReader function reference page lists file formats that VideoReader usually can read, including AVI, MPEG-1, and Motion JPEG 2000. Sometimes VideoReader can read files in unlisted formats, and sometimes it cannot read files in listed formats.

For video data, the term "file format" often refers either to the container format or the codec. A container format describes the layout of the file, while a codec describes how to code/decode the data. Many container formats support multiple codecs.

To read a video file, any application must:

• Recognize the container format (such as AVI). The VideoReader function reference page lists the supported container formats.

• Have access to the codec associated with the particular file. Some codecs are part of standard Windows and Macintosh system installations, and allow you to play video in Windows Media Player or QuickTime. VideoReader can access most, but not all, of these codecs.

• Properly interpret the codec. VideoReader cannot always read files associated with codecs that were not part of your original system installation.

To see the codec associated with a video file, use mmfileinfo and view the Format field. For example, given a hypothetical AVI file that uses the Indeo^® 5 codec, the following code:

info = mmfileinfo('myfile.avi');
info.Video.Format

returns

ans = 
IV50

Back to Top

Convert Between Image Sequences and Video

This example shows how to convert between video files and sequences of image files using VideoReader and VideoWriter.

The sample file named shuttle.avi contains 121 frames. Convert the frames to image files using VideoReader and the imwrite function. Then, convert the image files to an AVI file using VideoWriter.

Setup

Create a temporary working folder to store the image sequence.

workingDir = tempname;
mkdir(workingDir);
mkdir(workingDir,'images');

Construct a VideoReader Object

Create a VideoReader object to use for reading frames from the file.

shuttleVideo = VideoReader('shuttle.avi');

Create the Image Sequence

Loop through the video, reading each frame into a width-by-height-by-3 array named img. Write out each image to a JPEG file with a name in the form imgN.jpg, where N is the frame number:

      img1.jpg
      img2.jpg
      ...
      img121.jpg

for ii = 1:shuttleVideo.NumberOfFrames
    img = read(shuttleVideo,ii);

    % Write out to a JPEG file (img1.jpg, img2.jpg, etc.)
    imwrite(img,fullfile(workingDir,'images',sprintf('img%d.jpg',ii)));
end

Read and Sort the Image Sequence into MATLAB

Find all the JPEG file names in the images folder. Convert the set of image names to a cell array.

imageNames = dir(fullfile(workingDir,'images','*.jpg'));
imageNames = {imageNames.name}';

Notice that the image file names are not in numeric order.

disp(imageNames(1:10));

    'img1.jpg'
    'img10.jpg'
    'img100.jpg'
    'img101.jpg'
    'img102.jpg'
    'img103.jpg'
    'img104.jpg'
    'img105.jpg'
    'img106.jpg'
    'img107.jpg'

To sort the file names, extract the frame numbers from the file names and use them to sort the array.

First, match any file names that contain a sequence of numeric digits. Convert the strings to doubles.

imageStrings = regexp([imageNames{:}],'(\d*)','match');
imageNumbers = str2double(imageStrings);

Sort the frame numbers from lowest to highest. The sort function returns an index matrix that indicates how to order the associated files.

[~,sortedIndices] = sort(imageNumbers);
sortedImageNames = imageNames(sortedIndices);

The sequence file names are now sorted.

disp(sortedImageNames(1:10));

    'img1.jpg'
    'img2.jpg'
    'img3.jpg'
    'img4.jpg'
    'img5.jpg'
    'img6.jpg'
    'img7.jpg'
    'img8.jpg'
    'img9.jpg'
    'img10.jpg'

Create a New Video with the Image Sequence

Construct a VideoWriter object, which creates a Motion-JPEG AVI file by default.

outputVideo = VideoWriter(fullfile(workingDir,'shuttle_out.avi'));
outputVideo.FrameRate = shuttleVideo.FrameRate;
open(outputVideo);

Loop through the image sequence, load each image, and then write it to the video.

for ii = 1:length(sortedImageNames)
    img = imread(fullfile(workingDir,'images',sortedImageNames{ii}));

    writeVideo(outputVideo,img);
end

Finalize the video file.

close(outputVideo);

View the Final Video

Construct a reader object.

shuttleAvi = VideoReader(fullfile(workingDir,'shuttle_out.avi'));

Create a MATLAB movie struct from the video frames.

mov(shuttleAvi.NumberOfFrames) = struct('cdata',[],'colormap',[]);
for ii = 1:shuttleAvi.NumberOfFrames
    mov(ii) = im2frame(read(shuttleAvi,ii));
end

Resize the current figure and axes based on the video's width and height, and view the first frame of the movie.

set(gcf,'position', [150 150 shuttleAvi.Width shuttleAvi.Height])
set(gca,'units','pixels');
set(gca,'position',[0 0 shuttleAvi.Width shuttleAvi.Height])

image(mov(1).cdata,'Parent',gca);
axis off;

Play back the movie once at the video's frame rate.

movie(mov,1,shuttleAvi.FrameRate);

Credits

Video of the Space Shuttle courtesy of NASA.

Back to Top

Importing Images

Importing Binary Data with Low-Level I/O

Recommended Products

• MATLAB
• Signal Processing Toolbox
• Optimization Toolbox
• Statistics Toolbox
• Control System Toolbox

Get MATLAB trial software

Includes the most popular MATLAB recorded presentations with Q&A sessions led by MATLAB experts.

Get the Interactive Kit