Moving average filter for large dataset

Asked by Kumar on 2 Sep 2019

Latest activity Commented on by Kumar on 2 Sep 2019

5 views (last 30 days)

Hello everyone.

I have a large dataset with for eg. 3500 Hz (i.e 3500 samples every second) for lets say 10 minutes. so in total I have 3500*60*10 = 2100000 samples.

example question: X = 2100000x1 double.

and now I want to perform moving average for 1 minute (i.e 60*3500 = 210000 samples).

I have worked out the solution using conv. code is written below.

Y = conv(X, ones(1,210000)/210000, 'valid');

This does exactly what I want, but it is taking too long to compute (~3-4 minutes).

Are there any other methods which are faster?

Thanks.

2 Comments

Adam on 2 Sep 2019

Your code takes just a few seconds to run for me if I create a vector of random data of that size. Also

Y = movmean( X, 210000, 'Endpoints', 'discard' );

is also almost instantaneous and gives the same result.

Note: movmean is only available from R2016a

Kumar on 2 Sep 2019

Ah, that's strange.

Anyways, thanks for the movmean thing. It works perfectly fine.

Products

MATLAB

1 Answer

Link

Answer by Dimitris Kalogiros on 2 Sep 2019

Edited by Dimitris Kalogiros on 2 Sep 2019

Hi Ajay

You can try an iir filter like this

clear; clc;
%supose x contains your data.
% here , for simplicity, x is a constant with some gaussian noise
x= 5+randn(1,3500*60*10);
% output of iir filter (moving average)
y=zeros(1,length(x));
% time constant 
L=1/(60*3500);
% filtering process
for n=2:length(x)
    y(n)=(y(n-1)+L*x(n))/(1+L);
end
% plots of output and input
figure;
plot(x, '-b'); hold on;
plot(y, '-r', 'LineWidth', 2); zoom on; grid on;
legend('x: input', 'y: output');

1 Comment

Kumar on 2 Sep 2019

Thanks Dimitris. I will try this.

Products

MATLAB

You are now following this question

Kumar (view profile)