Filter Design Toolbox

errpdf

Calculate the probability density function (pdf) of the quantization error

Syntax

• (qf,x) = errpdf(q)
  (qf) = errpdf(q,x)
  

Description

(qf,x) = errpdf(q) returns qf, the pdf of the quantization error, evaluated at the values returned in x. When you do not provide x as an input vector to define the values at which to calculate qf, errpdf uses 128 equally spaced points between (-2*eps) and (2*eps) as the values at which it calculates qf.

(qf) = errpdf(q,x) returns qf, the pdf of the quantization error, evaluated at the values specified in vector x. Values in qf result from using q to quantize a signal. The error generated by the quantization process is random and uniformly distributed around zero.

When the precision of your signal is close to the precision of your quantizer, qf may not match the theoretical precision. When your signal has infinite extent and infinite precision, the value calculated for qf matches the theoretical value of the pdf of the uniformly distributed quantization error.

For most purposes, when the difference in precision between a signal and the quantizers is greater then 16 bits, the result calculated by errpdf is exact. When you reduce the word length by 3 or 4 bits through quantization, errpdf generates an excellent approximation. For word length changes that exceed four bits, errpdf provides a less good match to the theoretical mean. For fixed-point quantizers, the word length property defines the precision.

As you change the rounding mode for your quantizer, the pdf changes as well, as shown in this table.

Round Mode	Probability Density Function (f(x) = pdf)	Mean (µ)	Variance (2)	dB = 10log102
ceil	1/;        ;      0 otherwise	/2	2/12	-6.02f - 10.79
convergent	1/;   ;  0 otherwise	0	2/12	-6.02f - 10.79
fix	1/(2);   ;      0 otherwise	0	2/3	-6.02f - 4.77
floor	1/;       ;      0 otherwise	-/2	2/12	-6.02f - 10.79
round	1/;   ;  0 otherwise	0	2/12	-6.02f - 10.79

In the table, represents the quantization level (eps(q)) for your quantizer, x is the uniformly distributed random quantization error, and f is the word length of the quantizer.

Examples

Using a quantizer on a signal, compare the pdf calculated by errpdf to the error generated by a Monte Carlo experiment. Notice that the quantizer uses 4 bits with 3 bits for the fraction length. Signal u in the Monte Carlo experiment is a double array.

• q = quantizer('round',[4 3]);
  [f,x] = errpdf(q);
  subplot(211)
  plot(x,f)
  title('Computed PDF of the quantization error.')
  
  % Compare f to the sample pdf from a Monte Carlo experiment
  r = realmax(q);
  u = 2*r*rand(10000,1)-r;  % Original signal
  y = quantize(q,u);        % Quantized signal
  e = y - u;                % Error
  subplot(212)
  hist(e,20);set(gca,'xlim',[min(x) max(x)])
  title('Estimate of the PDF of the quantization error.')
  

Looking at the plot shown here you see that the computed, or theoretical, and estimated pdfs agree closely.

Algorithm

Here are the methods for calculating the pdf for the five rounding modes. In the equations, x = y-u, where u is the original signal and y is the signal value after quantization. is the minimum quantization step for the quantizer. For all of the following, f(x) denotes the probability density function of the error.

Ceil mode

Convergent mode

Fix mode

Floor mode

Round mode

See Also

quantizer/errmean, quantizer/errvar

References

[1] Schlichthärle, Dietrich, Digital Filter, Springer, 2000, Section 8.3 "Quantization," pp. 233-240

errmean

errvar