filtfilt function returning NaN at certain frequencies
67 views (last 30 days)
I'm hoping someone can help me sort this out. I'm trying to use the filtfilt function with a butterworth filter, and the maximum frequency range I can specify depends on the sampling frequency Fs. For example:
Fs=16kHz, my maximum bandpass range is [121 7944]. Any frequency lower than 121 or higher than 7944 causes filtfilt to return NaN.
Fs=32kHz, my maximum bandpass range is [209 15900]. Any frequency lower than 209 or higher than 15900 causes filtfilt to return NaN.
Similar questions have been asked in the forum before, but the answers were usually to check the data for NaN or Inf values. I have neither in my data.
The program I'm developing needs to be generic enough to accept any sampling rate and user-specified bandpass. I want to add a warning if the user specifies a bandpass that will cause problems, but I'm not sure how to calculate the limit. I'm sure a large part of my problem is unfamiliarity with filters. Is there some nuance in filter design that I'm missing? Any help would be appreciated. Thanks.
I found the source of my problem. I am using the EEGLab toolbox. It appears that it causing a problem with the native filtfilt fxn. Removing the EEGlab toolbox from my paths list fixed the problem.
Jan on 28 Aug 2013
Reply to Jeremy's comment:
x = randn(35000,1);
[b, a] = butter(10, 60/5000, 'high');
x = filtfilt(b,a,x);
Inside filtfilt the reflection method for the start values need to solve the linear system (from version 2009a):
zi = sp \ (b(2:nfilt).' - a(2:nfilt).'*b(1));
But for the filter parameters a and b of the Butterworth filter, the condition of sp is 2.3e19. As a rule of thumb you loose k significant digits in the solution of the linear system, when the matrix has the condition 10^k. In this case loosing 19 digits mean, that the results are random. Huge values and overflows can be expected.
Obviously the reflection method proposed by Fredrik Gustafsson (Determining the initial states in forward-backward filtering, IEEE Transactions on Signal Processing, pp. 988--992, April 1996, Volume 44, Issue 4) has a limitation here. Unfortunately I do not have access to this paper to find out, what is said in the discussion about this problem. Sending an enhancement request to TMW would be a good idea.
The FiltFiltM implemtation suffers from exactly the same problem, but it does not use a sparse method to solve this system, such that a warning message appears. I'm not sure, why this does not happen for the sparse version.
Jan on 7 Feb 2013
Edited: Jan on 7 Feb 2013
filtfilt is based on the calculations of filter, which is a deterministic function. When you call it with finite values, infinite values are only replied in case of overflows.
I would be helpful if you post the relevant part of the code instead of describing it by some text. E.g. your maximum bandpass range of [121 7944] looks mysterious. How do you define the filter parameters exactly? Usually the cut-of frequencies are normalized by the Nyquist frequency, such that only values between 0 and 1 are accepted. Perhaps your test for finite values of the input has a bug and inspite of your assumption the input does contain NaNs.
Anyhow, whenever I've seen reports of such magic behaviour, there have been very real bugs at unexpected locations. Usually Matlab's toolbox functions have been shadowed by user-defined functions or variables. And without any doubt, Matlab's filter accepts the full range of possible frequencies from 0 to 1 (exclusively for numerical reasons). You can see an implementation as M-file with exactly the same results here: Answers: Use Filter Constants. So you can use this M-function instead of Matlab's built-in filter() and check in every step, where any why the non-finite values occur.
Cameron on 3 Mar 2013
I am having the same problem. I am using OSX 10.8.2 with 2012a. filtfilt was working fine but now using the 'butter' filter is only returning NaN's. If I used the code in the example found in doc filtfilt the 'equiripple' works fine but the the 'butter' again returns garbage. I wonder is this has something to do with a Java update or OSX Update.
x=ecg(500)'+0.25*randn(500,1); %noisy waveform
d=design(h,'equiripple'); %Lowpass FIR filter
y=filtfilt(d.Numerator,1,x); %zero-phase filtering
y1=filter(d.Numerator,1,x); %conventional filtering
legend('Zero-phase Filtering','Conventional Filtering');
d1 = design(h,'butter');
y = filtfilt(d1.sosMatrix,d1.ScaleValues,x);
plot(x,'b-.'); hold on;
legend('Noisy ECG','Zero-phase Filtering','location','NorthEast');
Right, that is my point. It used to work for me and then post this last OSX update where apple mucked around with Java because of the security issue it is no longer working.
MatLab Version 126.96.36.1999 (R2012a) OSX 10.8.2
java version "1.7.0_05" Java™ SE Runtime Environment (build 1.7.0_05-b06) Java HotSpot™ 64-Bit Server VM (build 23.1-b03, mixed mode)
Carlo E.D. Riboldi on 16 Dec 2013
I had a very similar issue on dataset sampled at very high frequency and very long. NaN showed up without apparent regularity when applying filtfilt to the signal, changing the order of the Butterworth filter (butter) between 4 and 8 and the cut-off frequency.
I tried the implementation of the filter through filter objects (fdesign), but this didn't fix the problem.
I downloaded the small package FilterM from Matlab Central, and this apparently fixed it all (<http://www.mathworks.com/matlabcentral/fileexchange/32261-filterm)>. I recompiled the included Mex-file with MSVC2008.
I am using Matlab 7.9.0, R2009b.