Code covered by the BSD License  

Highlights from
apply2VideoFrames

image thumbnail

apply2VideoFrames

by

 

27 Jul 2011 (Updated )

Apply any image processing function applied to all frames of input video.

outVideoFile=apply2VideoFrames(varargin)
function outVideoFile=apply2VideoFrames(varargin)
%%  outVideoFile
% This function applies some image function applied to all frames of input video. Similar
%  to cellfun.
%  
%% Syntax:
%  outVideo=apply2VideoFrames('inVideo', videoFile, 'hAppliedFunc', @function,...
%     'flagConcat', true, 'compression', 'NONE', 'PARAMS', param1, param2...);
%  outVideo=apply2VideoFrames('inVideo', videoFile, 'hAppliedFunc', @function,...
%     'PARAMS', param1, param2...);
%  
%% Description:
% This functions allows the user to apply a function of his choice to all video frames-
%  on a frame by frame basis.
%  Consider a case you have a function you are used to apply to a single image like
%  "imfilter", "imresize", "imadjust", "imcrop" etc... (see Matlab Image Processing
%  Toolbox for additional functions), and now you want to apply it to a video stored in a
%  file. 
%  Well now you can use this function with the video, and get the resulting video in a
%  single command. While this disregards the relation between the video frames, and treats
%  each frame as an individual image, this is a good starting point in testing methods for
%  video enhancement & analysis methods.
%  A similar functionality exists in the Matlab Computer Vision Toolbox- it is the step
%  command each object has.
%  To name a few good qualities of this function:
%   - Easy and intuitive to use, moving from image to video function in a single command.
%   - Versatility- operates on any function that fulfills certain basic requirements (see
%       following hAppliedFunc Input arguments description).
%   - Can be applied to the relevant frames (this can dramatically reduce run time).
%   - Allows frame data preprocessing- contrast, resizing denoising, various
%       transformations such as color space, geometrical etc...
%   - Supports Bayer video.
%   - Supports functions with non image data output.
%   - Good for easy video annotation, using image processing functions.
%  
%% Input arguments (defaults exist):
%   inVideo- input video file names
%  
%   hAppliedFunc- function handle to the function which will be applied to relevant video
%     frames. The function to which hAppliedFunc points must fit a few simple rules:
%        1) Operate on matrices with images data as inputs (usually NxMx3 UINT8).
%        2) The image variable is the first input parameter.
%        3) The function is expected to return resulting image matrix (usually NxMx3
%           UINT8), so it can be stored as frame in outVideoFile.
%        4) The rest parameters the function needs will be specified after the key string
%           'PARAMS'. The above parameters will be used as inputs for hAppliedFunc.
%   isCellInput- a flag that should be set according to the used function (hAppliedFunc).
%     The flag should be enabled when the used function received multiple images input as
%     a single cell array, and disabled otherwise. Default value is false!
%  
%   PARAMS- the parameters to be fed to the hAppliedFunc. Note all inputs specified
%     after the keyword 'PARAMS' will be ignored by this program, and forwarded as is to
%     hAppliedFunc, so it should the last among all inputs.
%  
%   compression- compression method used. Uncompressed video demands large disk space,
%     thus a compression should be used. As we deal with image processing, compression
%     artifacts are highly unwanted- so a lossless codec must be used. We have found
%     'LAGS'- Lagarith Lossless Video Codec http://lags.leetcode.net/codec.html to do the
%     trick. Note:  Verify that the needed codec is installed.
%  
%   conv2FuncFormat- a handle to a function responsible for converting the video frame
%     matrix (usually NxMX3 UINT8) into a format suitable for the hAppliedFunc function.
%     This can be both numerical type/class conversion, color space conversions, and
%     normalization (divide/multipl by 255 etc..). By utilization of this function handle
%     wide range of image processing functions are supported. Default function handle is
%     unity function y(x)=x.
%  
%   conv2FuncFormat- a handle to a function responsible for converting the hAppliedFunc
%     function outputs image format to a video frame matrix (usually NxMX3 UINT8). This
%     function handle should point to an inverse function of conv2FuncFormat. This can be
%     both numerical type/class conversion, color space conversions, and normalizations
%     (divide/multiply by 255 etc..). By utilization of this function handle wide range of
%     image processing functions are supported. Default function handle is unity function
%     y(x)=x.
%  
%   flagConcat- flag enabling creation a video file concatenated from input and
%           resulting videos. Note you must have the concatVideo2D function in your path.
%           You can download the function from Matlab File Exchange:
%           http://www.mathworks.com/matlabcentral/fileexchange/33951-concatenate-video-files-subplot-style
%  
%   flagBrowseFiles- when enabled, user can choose input video files via browser.
%  
%   outVideoFile-    user defined video file name (path+name+extension), to store the
%           resulting video. It is highly recommended to store in AVI format.
%  
%   iStart-    the first frame of the processed frames sequence. Default value is 1.
%  
%   iEnd-      the last frame of the processed frames sequence. Default value is video
%           length (last video frame).
%  
%   framesList- a vector of integers describing the list of frames to be embedded.
%           Expected to be sorted from low to high. Default value- 1:NVideoFrames
%  
%   isSaveAllFrames- when enabled (true)- saves all original video frames, changing only ones
%           selected by user (via framesList or iStart, iEnd). When disabled- saves only
%           user chosen frames, ignoring other frames. Default value- true (enabled)
%           (which results in embedded video of same number of frames as input video).
%   isBayer- when enabled, only first a 2D matrix defined by frame(:, :, 1) is used, while
%           remaining color channels is ignored. Default value is false;
%  
%   isFrameOut- when true result of an image is expected from hAppliedFunc. Otherwise a
%           numeric result is expected.
%  
%% Output arguments:
%   outVideoFile-    newly created video file name (path+name+extension). Note that you
%     might choose a file type different from the input file extension.
%  
%% Issues & Comments:
% - Some errors accuring in hAppliedFunc are not catched and reported properly for some
%     reason.
% - The duality of avifile and VideoWriter makes the code cumbersome. Unfortunately no
%     single function capable of encoding all video files extensions  is available.
%     Another issue is that VideoWriter and VideoReader/mmreader mean different things
%     under the VideoFormat concept. My good advice is to use avifile with .AVI
%     files until those issues are resolved, ...
% - .mpg file extension writing results in error (can be fixed if all outputs videos are
%     AVI).
% - dot (.) used in file name cause errors, as misinterpret by the fileparts function.
%  
%% Example I:
% matlabVfile='rhinos.avi'; % 'xylophone.mpg'; % no other input video files in Matlab default path was found
% outVideoFileCrop=apply2VideoFrames('inVideo', matlabVfile, 'hAppliedFunc', @imcrop, 'PARAMS', [10,10,120,180]);
% outVideoFileRotate=apply2VideoFrames('inVideo', matlabVfile, 'isSaveAllFrames', false, 'framesList', 10:100,'hAppliedFunc', @imrotate, 'PARAMS', 180);
% implay(matlabVfile);
% implay(outVideoFileCrop);
% implay(outVideoFileRotate);
%  
%% Example II (with multiple input videos of varying type):
% note apply2VideoFrames must be available
% ( http://www.mathworks.com/matlabcentral/fileexchange/37400 )
% matlabVfile1='rhinos.avi';  % no other input video files in Matlab default path were found
% matlabVfile2='xylophone.mpg';
% outVideoFileFuse12=apply2VideoFrames('inVideo', {matlabVfile2, matlabVfile1},...
%    'isSaveAllFrames', false, 'hAppliedFunc', @imagesFusion, 'isCellInput', true);
% implay(outVideoFileFuse12);
%  
%% See also
%  - cellfun
%  - Matlab image processing tools: imresize, imadjust, imfilter
%  
%% Revision history:
% First version: Nikolay S. 2010-12-07.
% Last update:   Nikolay S. 2013-03-13.
%  
%% *List of Changes:*
% 2013-03-13
% - some minor improvement, like reducing readFramesList for cases when the output is not
%   a framne to be stored.
% 2012-12-06
%  - isBayer frlag and functionality is added.
%  - filesListFromInput function replaced a code section
% 2012-08-07
%  - Warning message in case AviFile used for file over 2GB is issued.
% 2012-07-09
%  - link to "Concatenate video files subplot style" added
%  - waitbar name changed.
%  - defaultCompression changed to NONE
% 2012-07-08
%  - conv2FuncFormat and conv2FuncFormat function handle inputs were added to suppot image
%     processing function desighned for image formats other the UINT8x3-RGB, supported so
%     far (consider YCbCr, DOUBLE [0:1] etc...).
%  - Multiple files input was added to support multi image functions (see imagesFusion)
%  - isCellInput aded to support multiple video inputs options for different image
%     processing functions.
%  -  Utility function 'getfourcc' by Dirk-Jan Kroon is added for video compressor search
%      http://www.mathworks.com/matlabcentral/fileexchange/28706-list-video-codecs-fourcc
% 2012-05-22
%  - framesList and isSaveAllFrames added- allowing working on picked video frames (and
%     not continuous video section), and storing all frames, or only subset.
% 2012-02-19
%  - iStart and iEnd added- allowing working on video section.
% 2012-02-19
%  - 2D images support added
% 2011-07-26:
%  - VideoWriter added to replace avifile for non AVI files.
%  - VideoReader replaced mmreader.
% 2011-05-15:
%  - Header documentation style change (support publish)


%% Default parameters
isCellInput=false;
inVideo=[];
flagBrowseFiles=false;
flagConcat=false;
feedForwardVarargin={};
isEncodeWithAvifile=false;
iStart=1;
iEnd=Inf;
nFrames=Inf;
framesList=[];
isSaveAllFrames=[];
isBayer=false;
isFrameOut=true;
conv2FuncFormat=@(x) x;
convFromFunc2RGB=@(x) x;

if nargin>=2
    % automatically get all input pairs (name, value) and store in local vairables
    for iArg=1:2:length(varargin)
        if strcmpi(varargin{iArg},'PARAMS') % get PARAMS  to transfer them to hAppliedFunc
            feedForwardVarargin=varargin(iArg+1:end);
            break;
        end
        % eval([varargin{iArg},'=varargin{iArg+1};']);  % TODO get read of EVAL.
        assignin_value(varargin{iArg},varargin{iArg+1});
    end
else
    error('apply2VideoFrames:argChk', 'Function handle undefined.');
end

if isempty(isSaveAllFrames) 
    % when isFrameOut is dissabled, there is no use in going through all frames
    if isFrameOut
        isSaveAllFrames=true;
    else
        isSaveAllFrames=false;
    end
end
    
defaultCompression='LAGS'; % NONE LAGS is much better option...
videoFormats= VideoReader.getFileFormats();
videosFilesExtList={videoFormats.Extension};

if isempty(inVideo)
    flagBrowseFiles=true;
end
inVideo=filesListFromInput( inVideo, flagBrowseFiles, videosFilesExtList,...
    'Files list', 'Select input video files (Ctrl for multiple files).' );

nFiles=length(inVideo);
videoPath=cell(nFiles, 1);
videoName=cell(nFiles, 1);
videoExt=cell(nFiles, 1);
ObjInVideo=cell(nFiles, 1);
for iFile=1:nFiles
    [videoPath{iFile}, videoName{iFile}, videoExt{iFile}] = fileparts(inVideo{iFile});
    assert(exist(inVideo{iFile}, 'file')==2, 'No such video file is found.');
    
    ObjInVideo{iFile} = VideoReader([videoPath{iFile}, filesep, videoName{iFile},...
        videoExt{iFile}]);
    
    nFrames=min( nFrames, ObjInVideo{iFile}.NumberOfFrames );
    iEnd=min( iEnd, nFrames ); % maximal avalible stop index
    
    if isempty(framesList)
        framesList=iStart:iEnd;
    else
        framesList(framesList > iEnd)=[]; % remove frame indexes out of video file limits
    end
end

inpFileNames= strcat( videoName, repmat({' '}, [size(videoName)]) );
inpFileNames{end}=inpFileNames{end}(1:end-1);
fprintf('\nVideos: "%s" frames update via %s started: %s \n', strcat(inpFileNames{:}),...
    func2str(hAppliedFunc), datestr(now,'dd-mmm-yyyy HH:MM:SS'));

if isFrameOut
    if exist('outVideoFile','var')~=1 % generate outVideoFile name if user didn't supply one
        outVideoFile=[func2str(hAppliedFunc),'_', inpFileNames{:}, videoExt{1}];
        outVideoFile=[videoPath{1}, filesep, outVideoFile];
    else
        [outVideoPath, outVideoName, outVideoExt] = fileparts(outVideoFile);
        if isempty(outVideoPath) % if no path was mentioned- use inputs path
            outVideoFile=[videoPath{1}, filesep, outVideoName, outVideoExt];
        end
    end
    
    if exist('compression','var')~=1 % when compression undefined inherit from first parent file
        % mmfileinfo uses mmreader, which supposed to be removed starting R2011b
        inVideoFileInfo = mmfileinfo(inVideo{1});
        compression=inVideoFileInfo.Video.Format;
        % unfortunately inVideoFileInfo.Video.Format names differ from videoWrCompMethods
    end % if exist('compression','var')~=1
    
    profiles = VideoWriter.getProfiles();
    videoWrCompMethods={profiles.Name};
    % videoWrCompMethods={'Motion JPEG AVI','Uncompressed AVI','Motion JPEG 2000','Archival'};
    isVideoWriterCompression=any(~cellfun(@isempty, strfind(videoWrCompMethods, compression)));
    [~, ~, outVideoExt] = fileparts(inVideo{1});
    if strcmpi(outVideoExt,'.AVI') && ( ~isVideoWriterCompression );
        % if an a file with AVI extention, and codec used in not supported by VideoWriter,
        isEncodeWithAvifile=true; % enable encoding with avifile, instead of VideoWriter
        % turn encoding warning off, for encoders not integrated into Matlab (like LAGS)
        warning('off', 'MATLAB:aviset:compressionUnsupported');
        
        % http://www.mathworks.com/matlabcentral/fileexchange/28706-list-video-codecs-fourcc
        if exist('getfourcc', 'file')==2 % if utility function 'getfourcc' by Dirk-Jan Kroon is in path
            L = getfourcc; % List of installed video codecs, though LAGS is missed for some reason
            if ~any(strcmpi(compression, {L.name}))
                compression=defaultCompression;
            end
        end
        
        if any(strcmpi(compression, {'RGB 24', 'RGB24', 'MJPG'})) %
            compression=defaultCompression;
        end
    end
    
    if isEncodeWithAvifile % VIDEOWRITER
        ObjOutVideo = avifile(outVideoFile, 'fps', ObjInVideo{1}.FrameRate,...
            'compression', compression);
        outVideoFile=ObjOutVideo.Filename;
    else
        % if compression is supported by VideoWriter, use it
        if isVideoWriterCompression
            ObjOutVideo = VideoWriter(outVideoFile, compression);
        else
            ObjOutVideo = VideoWriter(outVideoFile, 'Archival'); % default compression
        end
        ObjOutVideo.FrameRate=ObjInVideo{1}.FrameRate;
        outVideoFile=strcat(ObjOutVideo.Path, filesep, ObjOutVideo.Filename);
        % currently compression is supported in Motion JPEG 2000 files
        %    if strcmpi(ObjOutVideo.VideoCompressionMethod,'Motion JPEG 2000')
        %       ObjOutVideo.LosslessCompression=true;
        %    end
        open(ObjOutVideo);
    end % if isEncodeWithAvifile
else
    outFrameCount=1;
end % if isFrameOut

h=waitbar( 0,sprintf('Please wait, video frames update via %s in progress:',...
    func2str(hAppliedFunc)),'Name',strcat('apply2VideoFrames: ',func2str(hAppliedFunc)) );
hTicApply2VideoFrames=tic;

try
    if isSaveAllFrames
        readFramesList=1:nFrames;
    else
        readFramesList=framesList;
    end
    nAppliedFrames=length(readFramesList);
    
    for iFrame = readFramesList % loop through all frames
        currFrame=cell(nFiles, 1);
        if ~any(iFrame==framesList)
            if isSaveAllFrames
                updatedFrame=read(ObjInVideo{1}, iFrame);
            else % if only processed frames to be saved- skip saving frame to video
                continue;
            end
        else % if ~any(iFrame==framesList)
            for iFile=1:nFiles % Loop thorugh all input video files
                
                if isBayer
                    tmpFrame=read(ObjInVideo{iFile}, iFrame);
                    currFrame{iFile}=tmpFrame(:, :, 1);
                else
                    currFrame{iFile}=read(ObjInVideo{iFile}, iFrame);
                end
            end % for iFile=1:nFiles
            
            % apply hAppliedFunc with user specified params to get outFrame
            inputImg=cellfun(conv2FuncFormat, currFrame, 'UniformOutput', false);
            if isCellInput
                updatedFrameFuncFormat=hAppliedFunc(inputImg, feedForwardVarargin{:});
            else
                updatedFrameFuncFormat=hAppliedFunc(inputImg{:}, feedForwardVarargin{:});
            end
            
            if isFrameOut
                updatedFrame=convFromFunc2RGB(updatedFrameFuncFormat);
                
                if ~strcmpi('UINT8', class(updatedFrame)) % convert all non UINT8 images to UINT8
                    updatedFrame=uint8(updatedFrame);
                end
                
                % add updated frame to outVideo
                if isEncodeWithAvifile
                    ObjOutVideo = addframe(ObjOutVideo, updatedFrame);
                else
                    writeVideo(ObjOutVideo, updatedFrame);
                end
                
            else
                if iFrame == framesList(1) % init resulting array
                    updatedFrame=repmat(updatedFrameFuncFormat, [length(framesList), 1]);
                end

                updatedFrame(outFrameCount, :)=updatedFrameFuncFormat;
                outFrameCount=outFrameCount+1;
            end % if isEncodeWithAvifile
        end % any(iFrame==framesList)
        
        % Present waitbar- a bar with progress, time passed and time remaining
        waitbarProgress=find(iFrame == readFramesList)/nAppliedFrames;
        waitbarTimeRemaining(h, hTicApply2VideoFrames, waitbarProgress);
    end % for iFrame = 1:nFrames % loop through all frames
    
    close(h);
    if isFrameOut
        if isEncodeWithAvifile
            ObjOutVideo = close(ObjOutVideo);
            fileInfo=dir(outVideoFile);
            if (fileInfo.bytes/2^30) > 2 % Verify size, issue warning if file too large
                warning('MATLAB:apply2VideoFrames',...
                    'File of over 2GB, encoded with Avifile will not be readable. Use VideoWriter, or different codec.')
            end
        else
            close(ObjOutVideo);
        end
    else
        outVideoFile=updatedFrame; % no Video file, just the data
    end % if isFrameOut
catch exception  % In case of error remeber to close open files- or esle you'll get another error nvideoExt time ;)
    close(h);
    if isFrameOut
        if isEncodeWithAvifile
            ObjOutVideo = close(ObjOutVideo);
            fileInfo=dir(outVideoFile);
            if (fileInfo.bytes/2^30) > 2 % Verify size, issue warning if file too large
                warning('MATLAB:apply2VideoFrames',...
                    'File of over 2GB, encoded with Avifile will not be readable. Use VideoWriter, or different codec.')
            end
        else
            close(ObjOutVideo);
        end
    elseif exist('updatedFrame', 'var')==1 
        outVideoFile=updatedFrame;  % no Video file, just the data
    end % if isFrameOut
    fprintf('\nError in frame No'' %d!!!\n\n', iFrame);
    rethrow(exception);
end

if isFrameOut && flagConcat
    concatFilesList= cat(2, inVideo, outVideoFile);
    concatVideo2D('fileNames', concatFilesList);
end

Contact us