Filter Design Toolbox

errvar

Return the variance of the quantization error resulting from quantizing a signal

Syntax

• qvar = errvar(q)
  

Description

qvar = errvar(q) returns the variance of the uniformly distributed random quantization error that results when you use quantizer q to quantize a signal.

The value of errvar does not depend on the signal quantized unless the precision (the value of the least significant bit) of your signal and your quantizer are very nearly the same. Use eps to determine the precision for quantizers or various word lengths.

When the precision of your signal is close to the precision of your quantizer, qvar may not match the theoretical value. When your signal has infinite extent and infinite precision, the value calculated for qvar matches the theoretical value of the variance of the uniformly distributed quantization error.

For most purposes, when the difference in precision between a signal and the quantizers is greater then 16 bits, the result calculated by errvar is exact. When you reduce the word length by 3 or 4 bits through quantization, errvar generates an excellent approximation. For word length changes that exceed 4 bits, errvar provides a less good match to the theoretical mean. For fixed-point quantizers, the word length property defines the precision.

As you change the rounding mode for your quantizer, the variance changes as well, as shown in this table.

Round Mode	Probability Density Function (f(x) = pdf)	Mean (µ)	Variance (2)	dB = 10log102
ceil	1/;        ;      0 otherwise	-/2	2/12	-6.02f - 10.79
convergent	1/;   ;  0 otherwise	0	2/12	-6.02f - 10.79
fix	1/(2);   ;      0 otherwise	0	2/3	-6.02f - 4.77
floor	1/;       ;      0 otherwise	-/2	2/12	-6.02f - 10.79
round	1/;   ;  0 otherwise	0	2/12	-6.02f - 10.79

In the table, represents the quantization level (eps(q)) for your quantizer, x is the uniformly distributed random quantization error, and f is the word length of the quantizer.

Examples

To demonstrate the accuracy of errvar, compare the theoretical variance for the quantization error as determined by Monte Carlo analysis using a signal to the result from errvar:

• q = quantizer;
  v = errvar(q)
  
  % Compare to the sample variance from a Monte Carlo experiment
  r = realmax(q);
  u = 2*r*rand(1000,1)-r;  % Original signal
  y = quantize(q,u);       % Quantized signal
  e = y - u;               % Error
  v_est = var(e)           % Estimate of the error variance
  v =
  
    7.7610e-011 % =eps(q)^2/12 from the table
  
  v_est =
  
    7.5534e-011
  
  v_est-v
  
  ans =
  
   -2.0758e-012
  

Algorithm

The variance depends on the rounding mode of the quantizer. Ceil, convergent, floor, and round share the same variance through different calculations. Fix differs by a factor of four. For the definition and derivation of µ for each mode, refer to errvar. E(x) is the expected value of the random variable; the variance is ². In the equations, x = y-u, where u is the original signal and y is the signal value after quantization. is the minimum quantization step for the quantizer.

Ceil and floor modes

Convergent and round modes

Fix mode

See Also

quantizer/errmean, quantizer/errpdf

References

[1] Schlichthärle, Dietrich, Digital Filter, Springer, 2000, Section 8.3 "Quantization," pp. 233-240

errpdf

euclidfactors