File Exchange

image thumbnail

movingstd & movingstd2

version 1.4 (3.54 KB) by

A (fast) windowed std on a time series (or array)

4.78571
16 Ratings

29 Downloads

Updated

View License

Occasionally I see a request for computation of a running, windowed standard deviation. This is easily accomplished using filter and the alternative formula for the standard deviation:
std = sqrt((sum(x.^2) - n*xbar.^2)/(n-1)).
movingstd allows you to specify forward, backward or central windows of any desired length. It patches the ends, shortening the window as necessary.
movingstd2 is now included, allowing a moving window standard deviation on 2-dimensional arrays. (I suppose I could easily enough implement movingstdn if that becomes of interest.)

Comments and Ratings (31)

bharat yadav

bharat yadav

E. Cheynet

E. Cheynet (view profile)

John D'Errico

John D'Errico (view profile)

Just uploaded a new release which includes movingstd2. That tool only allows central windows. It would seem to make no sense at all to have a window that is a forward window in one dimension and central in the other dimension.

As well, the window size is always square, although that would be easily remedied if I see requests. The moving window is truncated around the edges as necessary.

Finally, which I did not write movingstdn, that too would be a trivial extension if there are people asking.

Robert

Robert (view profile)

Another nice function John, thanks. As suggested by Ka Ho Tam, would be nice if this works on 2D data. Would that be possible?

Ka Ho Tam

Would be nice if this works on 2D data.

Nathan Orloff

John... For what's worth, your functions kick butt!

Nathan Orloff

John... For what's worth, your functions kick butt!

John D'Errico

John D'Errico (view profile)

Uploaded a new version to replace strmatch

Erik

Erik (view profile)

Very nice function and quite useful with the little extras.

Please note the strmatch function being used will be replaced in a future version of MATLAB. Replacing it with strncmp fixes the warning. Please update the file some time in the future.

Maty

Maty (view profile)

Sorry my mistake

Maty

Maty (view profile)

The vector s contains all NaN. Probably the file needs an update for the newest MATLAB version.

John D'Errico

John D'Errico (view profile)

Keith - I just submitted a fix for that case. The new version should appear online in the morning. It happens because the result was essentially zero for some elements, but floating point arithmetic errors can drive a value less than zero, then a sqrt is taken.

Keith

Keith (view profile)

Great function, but I just ran into a complex standard deviation. Not sure if it's my end or not.

John D'Errico

John D'Errico (view profile)

Lennart - No, filter definitely does not assume that. However, nothing stops you from using this code as is with a minor trick.

1. Copy the correct number of points (based on the window size) at each end of the series to the other end.

2. Compute the windowed SD

3. Delete the created end point values.

Lennart

If my series are periodical, the ends meet and the special case defined in this function is not correct. Does the filter function assume this and can I just delete the special case part?

John D'Errico

John D'Errico (view profile)

Nayomi - If x is an image, I assume that you mean it is an array. Note that this function is defined explicitly for use on vector input x. If you desire to use it on each row (or column) of an array, you would use an external loop over the rows or columns.

I used this function as follows.
Image for x.
k=7
windowmode=backward
I= movingstd(x,k,windowmode)

Following error is generated while executing this code.
Error in ==> movingstd at 89
x = x - mean(x);

Please Can you suggest a solution for this problem.

John D'Errico

John D'Errico (view profile)

I'm unsure what is meant here. The behavior of filter at the edges is not material, since I explicitly repair those elements along the edges as is appropriate. This is already noted in the help:

% All windowmodes adjust the window width near the ends of
% the series as necessary.

The internal comments also indicate what was done to repair the behavior of filter at the ends.

So I think/thought that this question is already covered, but perhaps the help should be more explicit to reflect what is done. I'll submit a new version that does this.

Very nice. Performs as advertised, and quickly!
Is a reflection condition imposed implicitly by "filter.m" at the edges ? If so it would be nice to mention, just so that users are aware of all assumptions.

John D'Errico

John D'Errico (view profile)

Simplest is to replace the nans by interpolation. Use my inpaint_nans (also on the file exchange) to do this. Then apply this code to compute the moving standard deviation. High variance sections of the curve will still be high variance after this process. The virtue of this scheme is that both parts are fully vectorized codes.

The alternative is to use a loop, computing a windowed standard deviation on varying numbers of points by excluding the nan elements. This scheme may be tripped up if you have enough nan elements. Of course, with very many nan elements, no scheme will be perfect.

Rosie

Rosie (view profile)

This is just what I need except I have NaNs in my timeseries...how can I deal with these?

Adam

Adam (view profile)

Ralph

Ralph (view profile)

Sophia Zhao

Israel Solares-Moya

fantastic.. just saved me a few hours!

Mikko Pirinen

well made and very useful

John O'Leary

Michael Robbins

Neat

Updates

1.4

movingstd2 included

1.3

strncmpi instead of strmatch for tests
Pre-centered and pre-scaled now

1.2

Fix the case of complex std results due to floating point arithmetic.

1.1

Documentation change to explicitly state the behavior at the end of the series.

MATLAB Release
MATLAB 7.0.1 (R14SP1)
Acknowledgements

Inspired: SLIDEFUN

Download apps, toolboxes, and other File Exchange content using Add-On Explorer in MATLAB.

» Watch video

Win prizes and improve your MATLAB skills

Play today