File Exchange

image thumbnail

FilterM

version 1.0 (19.9 KB) by

A faster FILTER and FILTFILT: Speedup factor 2.5 to 25

4.88889
10 Ratings

36 Downloads

Updated

View License

FilterM, FiltFiltM: Fast digital filter

These functions are compatible to MATLAB's FILTER and FILTFILT commands,
but they are faster (see screenshot):
FilterM: 30%-40% of FILTER runtime
FiltFiltM: 4%-20% of FILTFILT runtime

ADDITIONAL FEATURES:
- The dimension to operate on can be specified for FiltFiltM.
- FilterM can process the signal in backward direction. (This is the
main part of the acceleration of FiltFiltM, because it avoids to
reverse the signal two times.)
- For signals of type SINGLE, the intermediate values are stored in
DOUBLE precision to increase the accuracy. The output is converted
to SINGLE again.
- The Signal Processing Toolbox is *not* needed.

CALLING:
Y = FiltFiltM(b, a, X, Dim)
[Y, Zf] = FilterM(b, a, X, Zi, Dim, Reverse)

b, a: Filter parameters as DOUBLE vectors.
X: Signal as DOUBLE or SINGLE vector or array.
Zi, Zf: Initial and final conditions as DOUBLE or SINGLE array.
Optional, default: Zeros.
Dim: Dimension to operate on. Optional, default: 1st non-singelton.
Reverse: Flag to process the signal in reverse direction.
Optional, default: FALSE.
Y: Filtered signal, same size and type as X.
While FilterM filters in forward direction, FiltFiltM processes
the signal forward and reverse direction for a zero phase
distortion.

INTENTION:
To accelerate my FEX submission FiltFiltM, I've implemented a filter as
C-Mex, which works in reverse order. To my surprise this was faster than
running Matlab's FILTER forward, e.g. 3.7 times for a [10000 x 1] vector,
5th order Butterworth filter (Matlab 2009a, WinXP 32 bit, single core).
Therefore I've expanded the Mex such that the direction can be defined
as input. The algorithm is a direct form II transposed structure.
A future version will be mutli-threaded.

INSTALLATION:
Setup the compiler if not done before: mex -setup.
Auto-compilation: Call FilterM without inputs to start the compilation.
A pre-compiled Mex can be downloaded: http://www.n-simon.de/mex
Run the unit-tests uTest_FilterM and uTest_FiltFiltM to check validity and speed.

Tested: Matlab 6.5, 7.7, 7.8, WinXP, 32bit
Compiler: LCC2.4/3.8, BCC5.5, OWC1.8, MSVC2008
Assumed Compatibility: higher Matlab versions, Mac, Linux, 64bit

This is faster and more powerful than my former submission "FiltFiltM", which will be removed soon.

Comments and Ratings (28)

Jan Simon

Jan Simon (view profile)

@Danijel: The inputs are the same as for CoreSingleN, where you find the documentation.
MX: Number of elements in the 1st dimension
NX: Number of columns, considers mutliple dimensions.

Danijel

Hi, when using CoreSingle5() in my C project, what are the parameters MX and NX? Can't figure out. My input is 64 float samples.

dhaba india

Kevin C

Hi Jan,
Do you still plan to implement multithreading in an upcoming version? If so, do you have any idea when that version will be released? It would be a big help for me.

Kevin C

Andrea Libri

Royi Avital

Hi,
Used "FilterM" in my project:
https://github.com/RoyiAvital/FastGuassianBlur

The project is about finding the most efficient method to apply a Gaussian Blur on an image.

It really made things much much easier.

Thank You.

Jan Simon

Jan Simon (view profile)

I've received the files. Thanks! I see that I've made a mistake with the matlab.2015 address - sorry!

Michele S.

Hi Jan,
sorry to bother you, but I had to struggle quite a bit to find your e-mail address :S
Eventually I sent some files at: matlab.2010@...
Could you confirm that you received them? Thanks!

Jan Simon

Jan Simon (view profile)

@Michele S.: The currently published version does not consider the special cases a==1 or b==1. I have a new version with a special treatment of these cases, but still struggle with the multi-threading. Please send me the data by email for some experiments (you find the address in the help section of the code). Thanks.

Michele S.

Hello Jan,
I found out that in my actual use case, FilterX performs significantly worse than the built-in filter (~ x4 slower). In my case a = 1, b is a 1445x1 double array, X is a 97280x1 double array.
I am running on r2011b, 32 bit, and the mex file is compiled with MSVC2008.
Is this to be expected due to the specific signal/coefficients sizes? Any comments or suggestions would be higly appreciated.
I can share some data and examples with you if you are willing to look further into this.
Thanks

Royi Avital

Jan,
Why not add an optional parameter of the number of threads to use?
Then you move the "Learning and Optimization" process to be the responsibility of the user.

We'd love to have a multi threaded version of this.

Thank You.

Jan Simon

Jan Simon (view profile)

@Jim: I'm sorry that my FEX submissions makes you nervous. I attach the unit-tests functions with speed comparisons to reduce this kind of stress.
I'm gald that this function helps you to save time and energy.

@Yair: I use (sulking) camel-case (this means: some uppercase letters) on purpose to avoid name space collisions with Matlab's builtin functions.
I have some succes with the multi-threaded version, but I still struggle with a general way to determine the most efficient number of threads. If the memory transfer is the bottle neck, even two threads on a single core can be useful. For data with short columns but many rows the strategy differs substantially for machines with few or many cores. The best would be an automatic learning process during the installation, which adjust the parameters for the number of threads depending on the data size and filter length. But this is an overkill and a source of bugs.
It would be useful to have a wide-spread feedback from FEX users with real-world cases, but this is seldom: only 1 of 20'000 downloaders starts a discussion with me.

Jim Hokanson

Jim Hokanson (view profile)

Hi Jan,

I'll be honest I'm always a bit nervous about your performance specs given updated Matlab versions and my use cases but in my very limited testing (single channel, 1 GB data, filtfiltM only) it seems to do much better. I'm getting a runtime that is 40% of the original and even better, the memory usage is roughly around 40% of the original. Given the large memory requirements of the original this feature is quite helpful.

Thanks,
Jim

Yair Altman

Yair Altman (view profile)

Still 20-30% faster than Matlab's built-in, even after the performance boost to the built-in filter function in R2014a.

Nice documentation, unit-tests and optimization for good performance.

Suggestion: rename the functions filterm, filterx etc. (without capitals). The current FilterM file breaks because it refers to filterX instead of FilterX for example. Now that Matlab is case-sensitive, better make all functions lowercase, as is the Matlab convention.

Aside from this minor issue, a very useful utility. Well done!

Any progress with the promised multi-threaded version?

Rami Khushaba

Rami Khushaba (view profile)

Thanks for these great functions. Besides an improvement in speed, they apparently solve some instability of default filtfilt function causing NaN in the output on a randomic basis.

Royi Avital

@Jan, try even longer filters. I'd be happy if you could release an update for long FIR's even before the threaded version.
It would be great!

Thank you for this great function.

Jan Simon

Jan Simon (view profile)

@Royi Avital: The currently published version does not consider the reduced calculations for FIR filters. I get these measurements for 2011b/64 under Windows, MSVC 2008:
x=rand(1e5, 1); B=repmat(1/21, 1, 21);
tic; for i=1:100, y=filter(B,1,x); clear('y'); end, toc
>> 0.56 sec
tic; for i=1:100, y=FilterX(B,1,x); clear('y'); end, toc
>> 0.42 sec (still faster)
The new version of FilterX considers A=1 explicitely and requires 0.27 sec only. It will be included in the next version, which is multi-threaded also. See: http://www.mathworks.com/matlabcentral/answers/24384-multi-threaded-mex-functions-in-the-fex

Royi Avital

@Jan, for FIR Filters with more than 20 coefficients FilterM lags behind MATLAB's implementation.
Do you think you could optimize for longer filters as well?
Thank you.

Jan Simon

Jan Simon (view profile)

Sorry, currently only real values are considered. The implementation of a complex parameters is not trivial in C. It is planned later.

Royi Avital

Jan, there's an issue with complex coefficients, Could you check that?

I'm using the MEX file directly (For "Classic" use as Filter).

Thank you.

Jan Simon

Jan Simon (view profile)

@Royi: I'm working on a multi-threaded version at the moment. Unfortunately this will work under Windows only, because I cannot test the pthread libs under Linux and MacOS.

Royi Avital

Thank you, this is great!
Do you think further optimization could be made?
Do you intentions to do so?

Jan Simon

Jan Simon (view profile)

@Vassilis: I do not get the point. FilterM produces exactly the same results as Matlab's FILTER (except for the higher accuracy for SINGLE), therefore it leads to the same phase shift. FiltFiltM (from this submission) produces the same results as Matlab's FILTFILT. The former submission 25337 "Patch FiltFilt" is slower, depends on the SPT and does not work with modern Matlab versions - but it creates the same results as FiltFiltM.

If you get different results between FilterM and FILTER or between FiltFiltM and FILTFILT, please send me the output of unit-test functions by mail.

vkehayas

Very good addition Jan. I observed however that, as you might know, it produces different results than your FiltFiltM as well as from the built-in filtfilt function. I think this is because of the zero-padding you apply, while in the previous version you were using Gustafsson's method -- am I right? The most prominent difference is a time-lag to the signal, plus some minor distortions in the signal.

vkehayas

MATLAB Release
MATLAB 7.8 (R2009a)
Acknowledgements

Inspired: Fast Guassian Blur, filter1

Download apps, toolboxes, and other File Exchange content using Add-On Explorer in MATLAB.

» Watch video