How to record and analyze audio in real-time?

Hi all,

I would like to have audio recorded continously, and at each time interval, grab the audio data and analyze the data without interrupting the recording.

I can imagine using two threads, one for recording, the other for analyzing. I know how to do recording and analyzing, but I don't know how to implement two threads. Does matlab support multiple threads for this kind of task? Could someone give me an example of how to do this? I would greatly appreciate that.

Thanks,

[Geoff Hayes]

Yanping - when I've used the audio recorder object to record from my microphone, I just add a timer object to the recorder. Then (for example) every second, the timer fires, and I process the most recent one second of data (since the lsat time the timer fired). Something like

 recorder = audiorecorder(8192,8,1);
 set(recorder,'TimerPeriod',1,'TimerFcn',{@audioTimer});

[Yanping]

Hi Geoff,

This is awesome! Looks like it's exactly what I was looking for. I will go ahead and try it immediately. Many thanks for the answer!

[Geoff Hayes]

Cool - I'll just copy the above as an answer in case someone else has a similar question.

[Yanping]

I tried it, and it works beautifully! Thanks, Geoff.

[Geoff Hayes]

Glad that it worked out, Yanping!

[Basava Chetan]

Its not working, how to analyze this. Please suggest me matlab code for digital hearing aid application. thank you. Is there any code like it does real time noise cancellation along with recording? Please suggest me

[Geoff Hayes]

Basava - please clarify what you mean by it's not working. Also, you seem to be asking a new question so please post it as such.

[Oskar Kilgus]

Hey guys,

im working on a project where i need pretty much the same as Yanping described at first.

I need a audiorecorder with (later sometihng like 15 min recording time) and i want to pitch-analyze every some seconds. So i´d need a continously recording with the ability to analyze a moving, lets say 60sec part.

I tried to understand it with the help of this post but really didnt get behind the Callback Functions, could you help me?

Thanks in advance!

[Geoff Hayes]

@Oskar Kilgus - what part of the callback is confusing?

[Oskar Kilgus]

@Geoff Hayes - i understand the idea of using them in this context but i didnt ever use them before and therefore dont know how to implement them in this case...

[Walter Roberson]

audiorecorder is only suitable for the case where you record everything first and then process the signal. If you are doing real-time work you should use the facilities from the Audio System Toolbox

[Oskar Kilgus]

From @Geoff Hayes linked code

https://www.mathworks.com/matlabcentral/answers/uploaded_files/21814/myAudioRecording.m

i tried to transfer this to pitch analysis. The problem with pitch seems to be the windowlength/overlaplength so you cant plot the XData/YData due to different sizes/shapes.

Do you guys have an idea how to make that work?

function [recorder] = myAudioRecording(Fs,durationSecs)
    
    % add an extra half-second so that we get the full duration in our
    % processing
    durationSecs = durationSecs + 0.5;
    
    % index of the last sample obtained from our recording
    lastSampleIdx = 0;
    
    % start time of the recording
    atTimSecs     = 0;
    
    % create the audio recorder
    recorder = audiorecorder(Fs,8,1);
    
    % assign a timer function to the recorder
    set(recorder,'TimerPeriod',1,'TimerFcn',@audioTimerCallback);
    
    % create a figure with two subplots (changed it to one)
    hFig   = figure;
    hAxes1 = subplot(1,1,1);
    

    % create the graphics handles to the data that will be plotted on each
    % axes (changed it to one)
    hPlot1 = plot(hAxes1,NaN,NaN);
    
    
    drawnow;
    
    % start the recording
    record(recorder,durationSecs);
    
    % define the timer callback
    function audioTimerCallback(hObject,~)
        
        % get the sample data
        samples  = getaudiodata(hObject);
        
        % skip if not enough data
        if length(samples)<lastSampleIdx+1+Fs
            return;
        end
        
        % extract the samples that we have not performed pitch on
        X = samples(lastSampleIdx+1:lastSampleIdx+Fs);
        
        % compute the pitch
        Y = pitch(X,Fs);
        
        % plot the data
        t = linspace(0,1-1/Fs,Fs) + atTimSecs;  %t wird zu einem zeitvektor von einer sekunde in 8000 Fs Teilen
        set(hPlot1,'XData',t,'YData',Y);

        % increment the time in seconds "counter"
        atTimSecs     = atTimSecs + 1; 

    end

    % do not exit function until the figure has been deleted
    waitfor(hFig);
end

This is the slightly changed code where i tried to do the exact same thing with pitch-plot. With fs of 8000, t becomes a 1x8000 vector, Y becomes a 95x1 vector.

Thanks :)

[Walter Roberson]

@Oskar Kilgus

Don't. Rewrite the code in terms of https://www.mathworks.com/help/audio/ref/audiodevicereader-system-object.html

If you record

half_samples = round(durationseconds * frequency / 2)

per frame, then the previous buffer plus the newest buffer concatenated together make a full current buffer. (If the overlap is not half of a buffer then the buffer strategy can be modified.)

Answer 1

[Geoff Hayes]

Yanping - when I've used the audio recorder object to record from my microphone, I just add a timer object to the recorder. Then (for example) every second, the timer fires, and I process the most recent one second of data (since the lsat time the timer fired). Something like

 recorder = audiorecorder(8192,8,1);
 set(recorder,'TimerPeriod',1,'TimerFcn',{@audioTimer});

Answer 2

[Nissim Gean]

Geoff - According to my understanding, in order to start recording and/or "fire" the timer we need to add a third line to your suggestion above, which is:

    record( recorder );

But then in order to grab the audio data we need first to stop the recorder object and start it again (after grabbing it) in the audioTimer callback function for the next iteration, something like this:

      function audioTimer(recorder)
      ...
        stop(recorder);
        data = getaudiodata(recorder);
        record(recorder);
      ...

Since we stop/start the recorder every time for frame-by-frame analysis we cannot get a continuous recording. Could you please explain how exactly you grab the audio data and analyze the data without interrupting the recording?

Thanks!

[Geoff Hayes]

Nissim - yes, the record( recorder ); is implied. I don't understand your comment But then in order to grab the audio data we need first to stop the recorder object and start it again. Why do you need to stop the recorder? Just grab the latest one second of data (from wherever you left off last time). See the example code at https://www.mathworks.com/matlabcentral/answers/uploaded_files/21814/myAudioRecording.m.

[Nissim Gean]

Geoff - Thank you for the example. I have tried to run it with my R2010b MATLAB version but got the same error:

??? Cannot retrieve audio data while recording is in progress.

Error in ==> myAudioRecording>audioTimerCallback at 54
        samples  = getaudiodata(hObject);

This is mainly the reason for my comment about stopping the recorder before grabbing. I thought it is mandatory to stop the recorder before grabbing the data, but now I understand that it is not have to be that way and it probably works differently in newer versions of MATLAB.

Are you aware of any of that?

Thanks!

[Geoff Hayes]

Nissim - no, I'm not aware of how older versions of MATLAB behave with respect to the audio recorder. But given the error message, it seems that you cannot get the audio data while it is being recorded. Perhaps the r2010b documentation suggests an alternative approach?

[Hamza Ashraf]

sir am using the same example to reacord and extract data at the same but am getting the following error i dont know how to solve it

Undefined function or variable 'durationSecs'

[Walter Roberson]

? None of the code posted on this page includes anything about duration, so we do not know what code you are talking about.

[Hamza Ashraf]

% Function that records sound from the microphone at a specified sampling
% rate for a fixed number of seconds
%
% Fs           - the sampling rate (Hz)
% durationSecs - the (optional) duration of the recording (seconds)
% N            - the (optional) FFT N-point block size
function [recorder] = myAudioRecording(Fs,durationSecs,N)

    if ~exist('durationSecs','var')
        % default to five minutes of recording
        durationSecs = 300;
    end
    
    if ~exist('N','var')
        % default to the sampling rate
        N = Fs;
    end
    
    % add an extra half-second so that we get the full duration in our
    % processing
    durationSecs = durationSecs + 0.5;
    
    % index of the last sample obtained from our recording
    lastSampleIdx = 0;
    
    % start time of the recording
    atTimSecs     = 0;
    
    % create the audio recorder
    recorder = audiorecorder(Fs,8,1);
    
    % assign a timer function to the recorder
    set(recorder,'TimerPeriod',1,'TimerFcn',@audioTimerCallback);
    
    % create a figure with two subplots
    hFig   = figure;
    hAxes1 = subplot(2,1,1);
    hAxes2 = subplot(2,1,2);
    
    % create the graphics handles to the data that will be plotted on each
    % axes
    hPlot1 = plot(hAxes1,NaN,NaN);
    hPlot2 = plot(hAxes2,NaN,NaN);
    
    drawnow;
    
    % start the recording
    record(recorder,durationSecs);
    
    % define the timer callback
    function audioTimerCallback(hObject,~)
        
        % get the sample data
        samples  = getaudiodata(hObject);
        
        % skip if not enough data
        if length(samples)<lastSampleIdx+1+Fs
            return;
        end
        
        % extract the samples that we have not performed an FFT on
        X = samples(lastSampleIdx+1:lastSampleIdx+Fs);
        
        % compute the FFT
        Y = fft(X,N);
        
        % plot the data
        t = linspace(0,1-1/Fs,Fs) + atTimSecs;
        set(hPlot1,'XData',t,'YData',X);
        
        f = 0:Fs/N:(Fs/N)*(N-1);
        set(hPlot2,'XData',f,'YData',abs(Y));
         
        % increment the last sample index
        lastSampleIdx = lastSampleIdx + Fs;
        
        % increment the time in seconds "counter"
        atTimSecs     = atTimSecs + 1; 
    end

    % do not exit function until the figure has been deleted
    waitfor(hFig);
end

this code acctually i want to extract the data samples at the same time of recording

[Walter Roberson]

That code defines the durationSecs parameter if you do not pass anything in. Are you sure that the error is inside the function, and not when you try to call the function? What parameters are you passing to the function?

[Hamza Ashraf]

oki i got that i was not passing durationSecs to the fucntion. i have another problem if you can help. i want to pass these following samples in above code to a trained neural network instead of taking fft.

X = samples(lastSampleIdx+1:lastSampleIdx+Fs);

can you tell me how to do that