Algorithm for a moving irregular time window?

Asked by Carsten Ridder on 27 Feb 2018

Latest activity Commented on by Carsten Ridder on 27 Feb 2018

11 views (last 30 days)

The code in movmean, movmedian etc. is extremely fast even when I use a duration row vector as directional window length, e.g.: "m = movmedian(tabdata, [days(30) 0],'SamplePoints',datum);" which calculates medians for the last 30 days, even if the datum (datetime) is not regular. It takes - in my case - 35 times longer to run through all dates in a loop, like: "idx = datum >= datum(ii) - days(30) & datum <= datum(ii)". Does anyone know the algorithm used in movmean, movmedian etc.?

0 Comments

1 Answer

Link

Answer by John D'Errico on 27 Feb 2018

The "algorithm"?

type movmean
'movmean' is a built-in function.

So movmean is a compiled function. It uses a loop, scanning through the data, maintaining a window of the desired size. Because the loop is compiled code, carefully written in a lower level language, it will be pretty efficient, more so than a loop in parsed MATLAB code, even with a good parsing capability as is built into MATLAB.

Anyway, as you wrote it, that is not even close to how I would try to write it even in MATLAB. you are using find on EVERY iteration. UGH. A bad idea. Instead, by keeping track of the points which will be in the window, even if the points are irregular, as long as they are sorted, you will do better than a repeated call to find. This is certainly how I would write the code in a lower level language.

1 Comment

Carsten Ridder on 27 Feb 2018

Thanks for your answer! I know that movmean is a built-in function; that's really why I asked my question ... But I didn't know that the compilation of the code itself gave such a huge time saving! And BTW I don't use "find" in the loop!

My problem is that the window size changes for every datapoint. The data points are - in average - measured each 3rd day, but there a many exceptions to that! I have to be sure, that I always make calculations on data from exactly the last 30 days. It is normally between ca. 8 and 30 points.

I would be very gratefull, if you could outline, how you would write efficient code in this case?

You are now following this question

Carsten Ridder (view profile)