FilterM, FiltFiltM: Fast digital filter
These functions are compatible to MATLAB's FILTER and FILTFILT commands,
but they are faster (see screenshot):
FilterM: 30%-40% of FILTER runtime
FiltFiltM: 4%-20% of FILTFILT runtime
- The dimension to operate on can be specified for FiltFiltM.
- FilterM can process the signal in backward direction. (This is the
main part of the acceleration of FiltFiltM, because it avoids to
reverse the signal two times.)
- For signals of type SINGLE, the intermediate values are stored in
DOUBLE precision to increase the accuracy. The output is converted
to SINGLE again.
- The Signal Processing Toolbox is *not* needed.
Y = FiltFiltM(b, a, X, Dim)
[Y, Zf] = FilterM(b, a, X, Zi, Dim, Reverse)
b, a: Filter parameters as DOUBLE vectors.
X: Signal as DOUBLE or SINGLE vector or array.
Zi, Zf: Initial and final conditions as DOUBLE or SINGLE array.
Optional, default: Zeros.
Dim: Dimension to operate on. Optional, default: 1st non-singelton.
Reverse: Flag to process the signal in reverse direction.
Optional, default: FALSE.
Y: Filtered signal, same size and type as X.
While FilterM filters in forward direction, FiltFiltM processes
the signal forward and reverse direction for a zero phase
To accelerate my FEX submission FiltFiltM, I've implemented a filter as
C-Mex, which works in reverse order. To my surprise this was faster than
running Matlab's FILTER forward, e.g. 3.7 times for a [10000 x 1] vector,
5th order Butterworth filter (Matlab 2009a, WinXP 32 bit, single core).
Therefore I've expanded the Mex such that the direction can be defined
as input. The algorithm is a direct form II transposed structure.
A future version will be mutli-threaded.
Setup the compiler if not done before: mex -setup.
Auto-compilation: Call FilterM without inputs to start the compilation.
A pre-compiled Mex can be downloaded: http://www.n-simon.de/mex
Run the unit-tests uTest_FilterM and uTest_FiltFiltM to check validity and speed.
Tested: Matlab 6.5, 7.7, 7.8, WinXP, 32bit
Compiler: LCC2.4/3.8, BCC5.5, OWC1.8, MSVC2008
Assumed Compatibility: higher Matlab versions, Mac, Linux, 64bit
This is faster and more powerful than my former submission "FiltFiltM", which will be removed soon.
@Danijel: The inputs are the same as for CoreSingleN, where you find the documentation.
MX: Number of elements in the 1st dimension
NX: Number of columns, considers mutliple dimensions.
Hi, when using CoreSingle5() in my C project, what are the parameters MX and NX? Can't figure out. My input is 64 float samples.
Do you still plan to implement multithreading in an upcoming version? If so, do you have any idea when that version will be released? It would be a big help for me.
Used "FilterM" in my project:
The project is about finding the most efficient method to apply a Gaussian Blur on an image.
It really made things much much easier.
I've received the files. Thanks! I see that I've made a mistake with the matlab.2015 address - sorry!
sorry to bother you, but I had to struggle quite a bit to find your e-mail address :S
Eventually I sent some files at: matlab.2010@...
Could you confirm that you received them? Thanks!
@Michele S.: The currently published version does not consider the special cases a==1 or b==1. I have a new version with a special treatment of these cases, but still struggle with the multi-threading. Please send me the data by email for some experiments (you find the address in the help section of the code). Thanks.
I found out that in my actual use case, FilterX performs significantly worse than the built-in filter (~ x4 slower). In my case a = 1, b is a 1445x1 double array, X is a 97280x1 double array.
I am running on r2011b, 32 bit, and the mex file is compiled with MSVC2008.
Is this to be expected due to the specific signal/coefficients sizes? Any comments or suggestions would be higly appreciated.
I can share some data and examples with you if you are willing to look further into this.
Why not add an optional parameter of the number of threads to use?
Then you move the "Learning and Optimization" process to be the responsibility of the user.
We'd love to have a multi threaded version of this.
@Jim: I'm sorry that my FEX submissions makes you nervous. I attach the unit-tests functions with speed comparisons to reduce this kind of stress.
I'm gald that this function helps you to save time and energy.
@Yair: I use (sulking) camel-case (this means: some uppercase letters) on purpose to avoid name space collisions with Matlab's builtin functions.
I have some succes with the multi-threaded version, but I still struggle with a general way to determine the most efficient number of threads. If the memory transfer is the bottle neck, even two threads on a single core can be useful. For data with short columns but many rows the strategy differs substantially for machines with few or many cores. The best would be an automatic learning process during the installation, which adjust the parameters for the number of threads depending on the data size and filter length. But this is an overkill and a source of bugs.
It would be useful to have a wide-spread feedback from FEX users with real-world cases, but this is seldom: only 1 of 20'000 downloaders starts a discussion with me.
I'll be honest I'm always a bit nervous about your performance specs given updated Matlab versions and my use cases but in my very limited testing (single channel, 1 GB data, filtfiltM only) it seems to do much better. I'm getting a runtime that is 40% of the original and even better, the memory usage is roughly around 40% of the original. Given the large memory requirements of the original this feature is quite helpful.
Still 20-30% faster than Matlab's built-in, even after the performance boost to the built-in filter function in R2014a.
Nice documentation, unit-tests and optimization for good performance.
Suggestion: rename the functions filterm, filterx etc. (without capitals). The current FilterM file breaks because it refers to filterX instead of FilterX for example. Now that Matlab is case-sensitive, better make all functions lowercase, as is the Matlab convention.
Aside from this minor issue, a very useful utility. Well done!
Any progress with the promised multi-threaded version?
Thanks for these great functions. Besides an improvement in speed, they apparently solve some instability of default filtfilt function causing NaN in the output on a randomic basis.
@Jan, try even longer filters. I'd be happy if you could release an update for long FIR's even before the threaded version.
It would be great!
Thank you for this great function.
@Royi Avital: The currently published version does not consider the reduced calculations for FIR filters. I get these measurements for 2011b/64 under Windows, MSVC 2008:
x=rand(1e5, 1); B=repmat(1/21, 1, 21);
tic; for i=1:100, y=filter(B,1,x); clear('y'); end, toc
>> 0.56 sec
tic; for i=1:100, y=FilterX(B,1,x); clear('y'); end, toc
>> 0.42 sec (still faster)
The new version of FilterX considers A=1 explicitely and requires 0.27 sec only. It will be included in the next version, which is multi-threaded also. See: http://www.mathworks.com/matlabcentral/answers/24384-multi-threaded-mex-functions-in-the-fex
@Jan, for FIR Filters with more than 20 coefficients FilterM lags behind MATLAB's implementation.
Do you think you could optimize for longer filters as well?
Sorry, currently only real values are considered. The implementation of a complex parameters is not trivial in C. It is planned later.
Jan, there's an issue with complex coefficients, Could you check that?
I'm using the MEX file directly (For "Classic" use as Filter).
@Royi: I'm working on a multi-threaded version at the moment. Unfortunately this will work under Windows only, because I cannot test the pthread libs under Linux and MacOS.
Thank you, this is great!
Do you think further optimization could be made?
Do you intentions to do so?
@Vassilis: I do not get the point. FilterM produces exactly the same results as Matlab's FILTER (except for the higher accuracy for SINGLE), therefore it leads to the same phase shift. FiltFiltM (from this submission) produces the same results as Matlab's FILTFILT. The former submission 25337 "Patch FiltFilt" is slower, depends on the SPT and does not work with modern Matlab versions - but it creates the same results as FiltFiltM.
If you get different results between FilterM and FILTER or between FiltFiltM and FILTFILT, please send me the output of unit-test functions by mail.
Very good addition Jan. I observed however that, as you might know, it produces different results than your FiltFiltM as well as from the built-in filtfilt function. I think this is because of the zero-padding you apply, while in the previous version you were using Gustafsson's method -- am I right? The most prominent difference is a time-lag to the signal, plus some minor distortions in the signal.