Quantization noise: Roundoff error in digital computation, signal processing, control, and communications

Quantization Noise: Roundoff Error in Digital Computation, Signal Processing, Control, and Communications

Volumn 9780521886710, Issue , 2008, Pages 1-751

  Widrow, Bernard ^a   Kollár, István ^b  

^a STANFORD UNIVERSITY   (United States)

^b BUDAPEST UNIVERSITY OF TECHNOLOGY AND ECONOMICS   (Hungary)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATION THEORY; DIGITAL ARITHMETIC; DIGITAL SIGNAL PROCESSING; SIGNAL PROCESSING;

DIGITAL COMPUTATION; FLOATING POINTS; HEURISTIC EXPLANATION; KEY FEATURE; MATHEMATICAL PROOF; QUANTIZATION NOISE; ROUND-OFF ERRORS; UNIFORM QUANTIZATION;

QUANTIZATION (SIGNAL);

EID: 84925134546     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1017/CBO9780511754661     Document Type: Book

References (128)

1. Abramowitz, M. and Stegun, I. A. (1972). Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables, Dover Publications, New York.
2. Amiantov, I. N. and Tikhonov, V. I. (1956). Impact of Gaussian processes on typical nonlinear components, Izvestiya Akademii Nauk SSSR, Otdelenie Tekhnicheskih Nauk (4): 33-41.
3. Arsac, J. (1966). Fourier Transforms and the Theory of Distributions, Prentice-Hall, Englewood Cliffs, N. J.
4. Bendat, J. S. and Piersol, A. G. (1986). Random Data: Analysis and Measurement Procedures, second, revised and expanded edn, Wiley, New York. First edition: 1971.
5. Bennett, W. R. (1948). Spectra of quantized signals, Bell System Technical Journal 27(4): 446-472.
6. Bertram, J. E. (1958). The effect of quantization in sampled-feedback systems, Transactions of the AIEE, Part 2, 77: 177-82.
7. Blachman, N. M. (1995). Recovering a bandlimited density function from histogram values. Manuscript.
8. Bognar, Mrs., J. Mogyorodi, J. Prekopa, A. Renyi, and Szasz, D. (2001). Valoszini gszamitâsfeladatgyüjtemény (Problems in Probability Theory), Ty-potex, Budapest. In Hungarian.
9. Borel, E. (1897). Sur l’interpolation (On Interpolation), Comptes Rendus des Sciences de l’Academie des Sciences 124: 673-676. In French.
10. Bracewell, R. N. (1986). The Fourier Transform and Its Applications, second, revised edn, McGraw-Hill, New York. First published: 1965.
11. Bremermann, H. (1965). Distributions, Complex Variables and Fourier Transforms, Addison-Wesley, Reading, MA.
12. Campbell, L. L. (1968). Sampling theorem for the Fourier transform of a distribution with bounded support, SIAM Journal of Applied Mathematics 16(3): 626-636.
13. CCITT (1984). Red Book, Vol. III, CCITT. Fascicle III.3. Recommendation G.711, Pulse Code Modulation (PCM) of Voice Frequencies.
14. Churchill, R. V. (1948). Introduction to Complex Variables and Applications, McGraw-Hill, New York.
15. Claasen, T. A. C. M. and Jongepier, A. (1981). Model for the power spectral density of quantization noise, IEEE Transactions on Acoustics, Speech and Signal Processing 29(4): 914-917.
16. Cooley, J. W. and Tukey, J. W. (1965). An algorithm for the machine computation of complex Fourier series, Mathematical Computation 19: 297-301.
17. Cramer, H. (1946). Mathematical Methods of Statistics, Princeton University Press, Princeton.
18. De Moor, B. (1991). Generalizations of the OSVD: Structure, properties and applications, in R. J. Vaccaro (ed.), SVD and Signal Processing, Vol. II, Algorithms, Analysis and Applications, Elsevier Science Publishers B.V., Amsterdam, pp. 83-98.
19. Domokos, G. (1989). Digital modelling of chaotic motion, Studia Scientiarum Math-ematicarum Hungarica 25: 323-41.
20. Domokos, G. and Szasz, D. (2003). Ulam's scheme revisited: digital modeling of chaotic attractors via micro-perturbations, Discrete and Continuous Dynamical Systems, Ser. A. 9(4): 859-876.
21. Dunay, R. and Kollar, I. (1999). Matlab-based analysis of roundoff noise, Periodica Polytechnica Series Electrical Engineering 43(1): 53-64.
22. Dunay, R., Kollar, I. and Widrow, B. (1998). Dithering for floating-point number representation, Proc. 1st International On-line Workshop on Dithering in Measurement, March 1-31, 1998. http://measure.feld.cvut.cz/dithering98/, pp. 79-90.
23. Elderton, W. P. (1938). Correzioni dei momenti quando la curva e simmetrica (Correction of Moments for Symmetrical Curves), Giornale dell’Istituto Italiano degliAttuari 9: 145. In Italian.
24. Erdelyi, A. (ed.) (1954). Tables of Integral Transforms, McGraw-Hill, New York.
25. Feller, W. (1970). An Introduction to Probability Theory and Its Applications, third, rev. edn, Wiley, New York. Original edition: 1950.
26. Forsythe, G. and Moler, C. (1967). Computer Solution of Linear Algebraic Systems, Prentice-Hall, Englewood Cliffs.
27. Franklin, G. F., Powell, J. D. and Workman, M. L. (1992). Digital Control of Dynamic Systems, Addison-Wesley, Reading, MA. Second edition.
28. Fujii, A. and Azegami, K. (1967). Quantization noise for a sine wave and a set of sine waves, Review of the Elecrical Communication Laboratories 15: 145-152.
29. Furman, G. G. (1957). Improvement of Quantized Feedback Systems by Means of External Dither, Master's thesis, MIT.
30. Gersho, A. and Gray, R. (1992). Vector Quantization and Signal Compression, Kluwer Academic Publishers, Boston.
31. Godfrey, K. (ed.) (1993). Perturbation Signals for System Identification, Prentice-Hall, Englewood Cliffs, NJ.
32. Goldberg, D. (1991). What every computer scientist should know about floatingpoint arithmetic, ACM Computing Surveys 23(1): 5-48. URL: http://docs-pdf.sun.com/800-7895/800-7895.pdf.
33. Golub, G. G. and van Loan, C. F. (1989). Matrix Computations, The Johns Hopkins University Press, Baltimore and London.
34. Gradshteyn, I. S. and Ryzhik, I. M. (1994). Table of Integrals, Series, and Products, corrected and enlarged edn, Academic Press, New York.
35. Gray, R. M. (1990). Quantization noise spectra, IEEE Transactions on Information Theory 36(6): 1220-44.
36. Gray, R. M. and Stockham, Jr, T. G. (1993). Dithered quantizers, IEEE Transactions on Information Theory 39(3): 805-12.
37. Hagen, J. B. and Farley, D. T. (1973). Digital-correlation techniques in radio science, Radio Science 8-9(8): 775-84.
38. Higgins, J. R. (1985). Five short stories about the cardinal series, Bulletin of the American Mathematical Society 12(1): 45-89.
39. Horvath, G. 1987. Kvantalasi hibak FFT alapu spektrumbecsleseknel (Quantization errors in FFT based spectrum estimations), Cand. Sci. Thesis, Hungarian Academy of Sciences. In Hungarian.
40. Hui, D. and Neuhoff, D. L. (2001). Asymptotic analysis of optimal fixed-rate uniform scalar quantization, IEEE Transactions on Information Theory 47(3): 957-977.
41. Hungarian Census (1992). 1990 Population Census, Detailed Data, Hungarian Central Statistical Office.
42. IEEE (1985). IEEE standard for binary floating-point arithmetic. ANSI/IEEE Standard 754-1985, New York, NJ.
43. IEEE (1987). IEEE standard for radix-independent floating-point arithmetic. ANSI/IEEE Standard 854-1987, New York, NJ.
44. Inose, H., Yasuda, Y. and Murakami, J. (1962). A telemetering system by code modulation and A-S modulation, IRE Transactions on Space Electronics & Telemetry SET-8(3): 204-9.
45. Jackson, L. B. (1996). Digital Filters and Signal Processing, Kluwer Academic Publishers, Boston.
46. Jensen, R. V. (1987). Classical chaos, American Scientist 75(2): 168-81.
47. Jerri, A. J. (1977). The Shannon sampling theorem - its various extensions and applications: A tutorial review, Proceedings of the IEEE 65(11): 1565-96.
48. Kahan, W. (1998). Matlab's loss is nobody's gain, World-wide web: http://www.cs.berkeley.edurwkahan/MxMulEps.pdf. revised July 31, 2004.
49. Kaneko, T. and Liu, B. (1970). Accumulation of round-off error in fast Fourier transforms, Journal of the Association for Computing Machinery 17(4): 63754.
50. Kaneko, T. and Liu, B. (1973). On local roundoff errors in floating-point arithmetic, Journal of the Association for Computing Machinery 20(3): 391-98.
51. Katzenelson, J. (1962). On errors introduced by combined sampling and quantization, IRE Transactions on Automatic Control AC-7(3): 58-68.
52. Kollar, I. (1983). Steady-state value measurements with quantized data, Proc. 2nd IMEKO TC8 Symposium ‘Theoretical and Practical Limits of Measurement Accuracy,’ Budapest, May 10-12, 1983, pp. 92-101.
53. Kollar, I. (1984). Statistical theory of quantization: Results and limits, Periodica Polytechnica Series Electrical Engineering 28(2-3): 173-190.
54. Kollar, I. (1986). The noise model of quantization, Proc. 1st IMEKO TC4 Symposium on Noise in Electrical Measurements, Como (Italy), June 19-21, 1986
55. OMIKK Technoinform, Budapest, 1987, pp. 125-129.
56. Kollar, I. (1988). Understanding quantization by simple models, Acta IMEKO 1988, Proc. IMEKO XI. World Congress, Houston, Texas, Oct. 16-21, 1988, Vol. 5, ISA, Research Triangle Park, NC, pp. 487-94.
57. Kollar, I. (1989). Merestechnika II. (Jelanalizis) Feladatgyujteminy (Signal Processing in Measurement — Problems and Solutions), 3rd, revised and enlarged edn, Tankonyvkiado, Budapest. Textbook of the Technical University of Budapest, 51441 (J5-1441). In Hungarian.
58. Kollar, I. (1993). Tools and methods of signal analysis, in L. Schnell (ed.), Technology of Electrical Measurement John Wiley and Sons, Chichester, pp. 249380.
59. Kollar, I. (1994). Bias of mean value and mean square value measurements based on quantized data, IEEE Transactions on Instrumentation and Measurement43(5): 733-9.
60. Kollar, I. (1997). Quantization noise, Doct. Sci. Thesis, Hungarian Academy of Sciences, Budapest.
61. Kollar, I. (2006). Digital non-subtractive dither: Necessary and sufficient condition for unbiasedness, with implementation issues, Proceedings of the 23rd IEEE Instrumentation and Measurement Technology Conference, Sorrento, Italy, 2427 April 2006, pp. 140-145.
62. Kollar, I. and Blair, J. J. (2005). Improved determination of the best fitting sine wave in ADC testing, IEEE Transactions on Instrumentation and Measurement54(5): 1978-83.
63. Kontro, J., Kalliojarvi, K. and Neuvo, Y. (1992). Floating-point arithmetic in signal processing, Proc. IEEE International Symposium on Circuits and Systems, San Diego, CA, May 10-13, 1992, Vol. 4, pp. 1784-91. 92CH3139-3.
64. Korn, G. A. (1965). Statistical measurements with quantized data, Simulation 7(4): 229-239.
65. Korn, G. A. and Korn, T. M. (1968). Mathematical Handbook for Scientists and Engineers; Definitions, Theorems, and Formulas for Reference and Review, second edn, McGraw-Hill, New York.
66. Kotel’nikov, V. A. (1933). On the transmission capacity of ‘ether’ and wire in electrocommunications, First All-Union Conference on Questions of Communications, 1933, Izd. Red. Upr. Svyazi RKKA, Moscow, pp. 1-19. In Russian.
67. Leon-Garcia, A. (1994). Probability and Random Processes for Electrical Engineering, second edn, Addison-Wesley, Reading, MA.
68. Lighthill, M. J. (1958). Introduction to Fourier Analysis and Generalized Functions, Cambridge University Press, Cambridge.
69. Linvill, W. K. (1949). Analysis and design of sampled data control systems, DSc. Thesis. MIT Project Whirlwind Report R-170.
70. Linvill, W. K. (1951). Sampled-data control systems studied through comparison of sampling with amplitude modulation, AIEE Transactions 70: 1779-1788.
71. Liu, B. and Kaneko, T. (1969). Error analysis of digital filters realized with floatingpoint arithmetic, Proceedings of the IEEE 57(10): 1735-1747.
72. Loeve, M. (1977). Probability Theory, fourth edn, Springer-Verlag, New York. Original edition: Probability theory: foundations, random sequences, New York, Van Nostrand, 1955.
73. Lukacs, E. (1970). Characteristic Functions, second edn, Griffin, London.
74. Marks, R. J. I. (1991). Introduction to Shannon Sampling and Interpolation Theory, Springer-Verlag, New York.
75. Milne-Thomson, L. M. (1960). The Calculus of Finite Differences, second edn, MacMillan & Co. Ltd., London.
76. Moorer, J. A. (1979). The digital coding of high-quality musical sound, Journal of the Audio Engineering Society 27(9): 657-66.
77. Muller, J.-M. (2005). On the definition of ulp(x), Research Report No. 2005-09, http://www.ens-lyon.fr/LIP/Pub/Rapports/RR/RR2005/RR2005-09.pdf. Ecole Normale Superieure de Lyon, France.
78. Norsworthy, S., Schreier, R. and Temes, G. C. (1997). Delta-Sigma Data Converters, IEEE Press, Piscataway, NJ.
79. Nyquist, H. (1928). Certain topics in telegraph transmission theory, AIEE Transactions 47:617-644.
80. Oppenheim, A. V. and Schafer, R. W. (1989). Discrete-Time Signal Processing, Prentice-Hall, Englewood Cliffs, NJ.
81. Oppenheim, A. V., Schafer, R. W. and Buck, J. R. (1998). Discrete-Time Signal Processing, second edn, Prentice-Hall, Englewood Cliffs, NJ.
82. Oppenheim, A. V. and Weinstein, C. J. (1972). Effects of finite register length in digital filtering and the fast Fourier transform, Proceedings of the IEEE 60(8): 957-76.
83. Oppenheim, A. V., Willsky, A. S. and Young, I. T. (1983). Signals and Systems, Prentice-Hall, Englewood Cliffs, NJ.
84. Pacut, A., Hejn, K. and Kramarski, L. (1997). Generalized model of the quantization error - analytical approach, 2nd International Workshop on ADC Modelling and Testing, Tampere, Finland, June 1-3, 1997, pp. 1-5. CD-ROM, topic 23.
85. Paduart, J., Schoukens, J. and Rolain, Y. (2004). Fast measurement of quantization distortions in DSP algorithms, Proceedings of the 21st IEEE Instrumentation and Measurement Technology Conference, Como, Italy, 18-20 May 2004, Vol. 1, pp. 140-145.
86. Papoulis, A. (1962). The Fourier Integral and Its Applications, McGraw-Hill, New York.
87. Papoulis, A. (1991). Probability, Random Variables and Stochastic Processes, McGraw-Hill, New York. Original edition: 1965.
88. Peebles, P. Z. (1976). Communication System Principles, Addison-Wesley, Reading, MA.
89. Pendergrass, N. A. and Farnbach, J. S. (1978). A high resolution, low frequency spectrum analyzer, Hewlett-Packard Journal 29(13): 2-13.
90. Pintelon, R. and Schoukens, J. (2001). System Identification: A Frequency Domain Approach, IEEE Press, Piscataway.
91. Pohlmann, K. C. (1992). The Compact Disc Handbook, second edn, A-R Editions, Madison, WC.
92. Press, W. H., Flannery, B. P., Teukolsky, S. A. and Vetterling, W. T. (1988). Numerical Recipes, Cambridge University Press, Cambridge.
93. Ragazzini, J. R. and Zadeh, L. A. (1952). The analysis of sampled-data systems, AIEE Transactions 71: 225-234.
94. Roberts, G. L. (1962). Picture coding using pseudo-random noise, IRE Transactions on Information Theory IT-8(2): 145-54.
95. Robertson, G. H. (1969). Computer study of quantizer output spectra, Bell System Technical Journal 48(5): 2391-2403.
96. Schreier, R. and Temes, G. C. (2005). Understanding Delta-Sigma Converters, IEEE Press and Wiley Interscience, Reading, MA. Second edition.
97. Schuchman, L. (1964). Dither signals and their effect on quantization noise, IEEE Transactions on Communication Theory 12: 162-65.
98. Shannon, C. E. (1949). Communication in the presence of noise, Proceedings of the IRE 47(1): 10-21.
99. Sheppard, W. F. (1898). On the calculation of the most probable values of frequency-constants, for data arranged according to equidistant divisions of scale, Proceedings of the London Mathematical Society 29: 353-380.
100. Spang, H. A. and Schultheiss, P. M. (1962). Reduction of quantization noise by use of feedback, IRE Transactions on Communications Systems 10: 373-380.
101. Sripad, A. B. and Snyder, D. L. (1977). A necessary and sufficient condition for quantization errors to be uniform and white, IEEE Transactions on Acoustics, Speech and Signal Processing 25(5): 442-448.
102. Statistical Abstract (1994). Statistical Abstract of the United States: The National Data Book, 114th edn, US Department of Commerce, Economics and Statistics Administration, Bureau of the Census.
103. Stuart, A. and Ord, J. K. (1994). Kendall's Advanced Theory of Statistics, Vol. I. Distribution Theory, sixth edn, Edward Arnold, London and John Wiley & Sons, New York.
104. Szekely, G. J. (1986). Paradoxes in Probability Theory and Mathematical Statistics, Reidel-Kluwer, Dordrecht-Boston, MA. New English edition: Springer 2002.
105. Thong, T. and Liu, B. (1976). Fixed-point fast fourier transform error analysis, IEEE Trans. ASSP ASSP-24: 563-573.
106. Tumfart, S. (1976). Stochastic-ergodic measurement techniques and their contribution towards the solution of metrological problems in science and technology, Acta IMEKO 1976, Proc. IMEKO VII. World Congress, London, 10-14 May, 1976, Vol 1, pp. 263-76.
107. Velichkin, A. I. (1962). Correlation function and spectral density of a quantized process, Telecommunications and Radio Engineering, Part II: Radio Engineering 17(7): 70-77.
108. Vetier, A. (1991). Szemliletes mirtik- 6s valoszinus6gelm6let (Elementary Measure and Probability Theory), Tankonyvkiado, Budapest. In Hungarian.
109. Wannamaker, R. A. (1992). Psychoacoustically optimal noise shaping, Journal of the Audio Engineering Society 40(7/8): 212-223.
110. Wannamaker, R. A. (1994). Subtractive and non-subtractive dithering: A comparative analysis, 97th Convention of the Audio Engineering Society. Preprint 3920 (N-3).
111. Wannamaker, R. A., Lipshitz, S. P, Vanderkooy, J. and Wright, J. N. (2000). A theory of non-subtractive dither, IEEE Transactions on Signal Processing 48(2): 499-516.
112. Wehrmann, W. (1973). Einfuhrung in die stochastisch-ergodische Impulstechnik (Introduction into Stochastic-Ergodic Measurement Technology), R. Oldenburg Verlag, Vienna. In German.
113. Weinstein, C. J. (1969). Roundoff noise in floating point fast Fourier transform computation, IEEE Transactions on Audio and Electroacoustics 17(3): 209-15.
114. Weinstein, C. and Oppenheim, A. V. (1969). A comparison of roundoff noise in floating point and fixed point digital filter realizations, Proceedings of the IEEE57(6): 1181-83.
115. Welch, P. D. (1969). A fixed-point fast Fourier transform error analysis, IEEE Transactions on Audio and Electroacoustics 17(2): 151-57.
116. Whittaker, E. T. (1915). On the functions which are represented by the expansion of interpolating theory, Proceedings of the Royal Society of Edinburgh 35: 181-194.
117. Whittaker, J. M. (1929). The ‘Fourier’ theory of cardinal functions, Proceedings of the Edinburgh Mathematical Society 1: 169-176.
118. Widrow, B. (1956a). A study of rough amplitude quantization by means of Nyquist sampling theory, Sc. D. Thesis. MIT, Department of Electrical Engineering.
119. Widrow, B. (1956b). A study of rough amplitude quantization by means of Nyquist sampling theory, IRE Transactions on Circuit Theory 3(4): 266-276.
120. Widrow, B. (1957). Propagation of statistics in systems, IRE 1957 Wescon Convention Record, Pt. 2, San Francisco, August 1957, pp. 114-21.
121. Widrow, B. (1961). Statistical analysis of amplitude-quantized sampled-data systems, Transactions of the AIEE, Part II.: Applications and Industry79(52): 555-68.
122. Widrow, B., Kollar, I. and Liu, M.-C. (1995). Statistical theory of quantization, IEEE Transactions on Instrumentation and Measurement 45(6): 353-61.
123. Wilkinson, J. H. (1994). Rounding Errors in Algebraic Processes, Dover Publications, New York. Original edition: Prentice Hall, 1963.
124. Wold, H. (1934a). Sheppard's correction formulae in several variables, Skandinavisk Aktuartidskript 17: 248.
125. Wold, H. (1934b). Sulla correzione di Sheppard (On Sheppard's corrections), Giornale dell’Istituto Italiano degli Attuari 5: 104. In Italian.
126. Woodward, P. M. (1964). Probability and Information Theory, with Applications to Radar, second edn, Pergamon Press, Oxford, New York. First edition: 1953.
127. Zakai, M. (1965). Band-limited functions and the sampling theorem, Information and Control 8(2): 143-158.
128. Zemanian, A. H. (1965). Distribution Theory and Transform Analysis; An Introduction to Generalized Functions, with Applications, McGraw-Hill, New York.