I am trying to design a Chebyshev type 2 filter in the discrete-time domain in order to have control over the placement of the filters zero. My attempt is almost successful, but I seem to be missing one last condition for the coefficients.

The following parameters are given:

Fs=250

Fnotch=7

H(f=0Hz)=1

H(f=fs/2)=R

B can be calculated directly from Fnotch and Fs:

B=poly([exp(2*pi*i*Fnotch/Fs) conj(exp(2*pi*i*Fnotch/Fs))])

A can be calculated from the two conditions for H(z):

f=0Hz -> z=1 (b0+b1+b2)/(1+a1+a2)=1 -> a1+a2=b0+b1+b2-1

and

f=fs/2Hz -> z=-1 (b0-b1+b2)/(1-a1+a2)=A -> -a1+a2=(1/A)*(b0-b1+b2)-1

This gives a nice linear equation system for a1 and a2 and ultimately the vector of filter coefficients A:

R=db2mag(-40);

A=[1 1; -1 1]\[sum(B)-1; (1/R)*(B(1)-B(2)+B(3))-1];

A=[1 A.']

freqz(B, A, 4096, Fs)

As you can see, the filter [B, A] fulfills all the conditions for notch position and gain at DC and fs/2, but it certainly doesn't look anything like a cheby2 filter in the frequency domain. And why? Because B isn't

B=poly([exp(2*pi*i*Fnotch/Fs) conj(exp(2*pi*i*Fnotch/Fs))])

but actually

vvv

B= G * poly([exp(2*pi*i*Fnotch/Fs) conj(exp(2*pi*i*Fnotch/Fs))])

^^^

There is a scale factor in B that requires an additional equation. Does anyone know what equation this is?

(If I use cheby2 to design a filter with a notch at Fnotch and insert the resulting B(1) as G into the above equation, the result for B and A will look much better.)