Vector Quantization with Complexity Costs

IEEE Transactions on Information Theory

Volumn 39, Issue 4, 1993, Pages 1133-1145

  Kühnel, Hans ^b  

^a UNIVERSITY OF BONN   (Germany)

^b UNIVERSITY OF MUNICH   (Germany)

Author keywords

complexity costs; entropy estimation; image compression; maximum; neural networks; Vector quantization

Indexed keywords

CODING ERRORS; COMPUTATIONAL COMPLEXITY; DATA REDUCTION; ENCODING (SYMBOLS); IMAGE COMPRESSION; IMAGE RECONSTRUCTION; INFORMATION THEORY; NEURAL NETWORKS; PARAMETER ESTIMATION; PROBABILITY; VECTORS;

CODEBOOK; COMPLEXITY COSTS; COMPLEXITY FUNCTION; COST FUNCTION; DISTORTION ERRORS; MAXIMUM ENTROPY ESTIMATION; REFERENCE VECTORS; VECTOR QUANTIZATION;

DATA COMPRESSION;

EID: 0027629328     PISSN: 00189448     EISSN: 15579654     Source Type: Journal    
DOI: 10.1109/18.243432     Document Type: Article

References (50)

1. S. C. Ahalt A. K. Krishnamurthy, P. Chen, and D. E. Melton, “Competitive learning algorithms for vector quantization,” Neural Net., vol. 3, pp. 277–290, 1990.
2. A. G. Andreou, K. A. Boahen, P. O. Pouliquen, A. Pavasović, R. E. Jenkins, and K. Strohbehn, “Current mode subthreshold MOS circuits for analog VLSI neural systems,” IEEE Trans. Neural Net., vol. 2, pp. 205–213, 1991.
3. G. Ball and D. Hall, “A clustering technique for summarizing mUltivariate ate data,” Behavioral Sciences, vol. 12, 153–155, 1967.
4. J. C. Bezdek, “A convergence theorem for the fuzzy ISODATA clustering algorithms,” IEEE Trans. Pattern Anal. Machine Intell., vol. 2, pp. 1–8, Jan. 1980.
5. L. Bobrowski and J. C. Bezdek, “ C-means clustering with the l 1 and l ∞ norms,” IEEE Trans. Syst., Man, Cybern., vol. 21, pp. 545–554, May-June 1991.
6. J. Buhmann and H. Kühnel, “Complexity optimized data clustering by competitive neural networks,” Neural Computat., vol. 5, no. 1, pp. 75–88, 1992.
7. J. Buhmann and H. Kühnel, “Complexity optimized vector quantization: A neural network approach,” in Data Compression Conference ‘92, J. Storer, Ed. Los Alamitos, CA: IEEE Computer Society Press, 1992, pp. 12–21.
8. J. Buhmann and H. Kühnel, “Unsupervised and supervised data clustering with competitive neural networks,” in IEEE LICNN Int. Conf. on Neural Net., Baltimore, MD, 1992, IV-796-1V-801.
9. P. A. Chou, T. Lookabaugh, and R. M. Gray, “Entropy-constrained vector quantization,” IEEE Trans. Acoust., Speech Signal Processing, vol. 37, pp. 31–42, Jan. 1989.
10. P. A. Chou, T. Lookabaugh, and R. M. Gray, “Optimal pruning with applications to tree-structured sourcecoding coding and modeling,” IEEE Trans. Inform. Theory, vol. 35, pp. 299–315, 315, Mar. 1989.
11. T. M. Cover, “Elements of Information Theory. New York: John Wiley, 1991.
12. C. Darken and J. Moody, “Fast adaptive K-means clustering: Some empirical results,” in IEEE Int. Joint Conf Neural Net., San Diego, CA, vol. II, June 17–21, 1990, pp. 233–238.
13. C. Darken and J. Moody, “Towards faster stochastic gradient search,” in Neural Inform. Processing Syst. 4. San Mateo, CA: Morgan Kaufmann, 1992.
14. J. Daugman, “Complete discrete 2-D Gabor transforms by neural networks for image analysis and compression,” IEEE Trans. Acoust., Speech Signal Processing, vol. 36, pp. 1169–1179, July 1988.
15. R. A. DeVore, B. Jawerth, and B. J. Lucier, “Image compression through wavelet transform coding,” IEEE Trans. Inform. Theory, vol. 38, pt. II, pp. 719–746, Mar. 1992.
16. R. O. Duda and P. E. Hart, Pattern Classification and Scene Analysis. New York: Wiley, 1973.
17. J. C. Dunn, “A fuzzy relative of the ISODATA process and its use in detecting compact well-separated clusters,” J. Cybern., vol. 3, no. 3, pp. 32–57, 1974.
18. R. Drubin and D. Willshaw, “An analogue approach to the travelling salesman problem using an elastic net method,” Nature, vol. 326, pp. 689–691, 1987.
19. N. Farvardin, “A study of vector quantization for noisy channels,” IEEE Trans. Inform. Theory, vol. 36, 799–809, July 1990.
20. T. Feder and D. H. Greene, “Optimal algorithms for approximate clustering,” in Proc. 20th Ann. ACM Symp. Theory of Computing (STOC), 1988, pp. 434–444.
21. D. J. Field, “Relations between the statistics of natural images and the response properties of cortical cells,” J. Opt. Soc. Amer. A, vol. 4, no. 12, pp. 2379–2394, 1987.
22. A. Gersho, “Asymptotically optimal block quantization,” IEEE Trans. Inform. Theory, vol. 25, pp. 373–380, July 1979.
23. H. Gish and J. N. Pierce, “Asymptotically efficient quantizing,” IEEE Trans. Inform. Theory, vol. IT-14, pp. 676–683, 1968.
24. R. M. Gray, “Vector quantization,” IEEE Acoust., Speech Signal Processing 4–29, 1984.
25. R. Hanson, J. Stutz, and P. Cheeseman, “Bayesian classification theory,” Tech. Rep. FIA-90-12-7-01, NASA Ames Research Center, 1991.
26. Y. Hussain and N. Farvardin, “Adaptive block transform coding of speech based on LPC vector quantization,” IEEE Trans. Signal Processing, vol. 39, pp. 2611–2620, Dec. 1991.
27. E. T. Jaynes, “Information theory and statistical mechanics,” Physical Rev., vol. 106, pp. 620–630, 1957.
28. E. T. Jaynes, “On the rationale of maximum-entropy methods,” Proc. IEEE, vol. PROC-70, pp. 939–952, 1982.
29. S. Kirkpatrick, C. D. Gelatt, and M. P Vecchi, “Optimization by simulated annealing,” Science, vol. 220, pp. 671–680, 1983.
30. T. Kohonen, Self-Organization and Associative Memory. Berlin: Springer, 1984.
31. J. Lazzaro, R. Ryckebusch, M. A. Mahowald, and C. A. Mead, “Winner-take-all networks of O(n) complexity,” in Neutral Information Processing Systems J. San Mateo, CA: Morgan Kaufmann, 1989, pp. 703–711.
32. Y. Linde, A. Buzo, and R. M. Gray, “An algorithm for vector quantizer design,” IEEE Trans. Commun., vol. COM-28, pp. 84–95, 1980.
33. S. P. Lloyd, “Least squares quantization in PCM,” IEEE Trans. Inform. Theory, 1982.
34. S. P. Luttrell, “Hierarchical vector quantisation,” IEE Proc., vol. 136, pp. 4065–4413, 1989.
35. J. MacQueen, “Some methods for classification and analysis of multivariate variate observations,” in Proc. 5th Berkeley Symp. Mathematical Statist. and Probability, pp. 281–297, 1967.
36. S. G. Mallat, “A theory for multiresolution signal decomposition: The wavelet representation,” IEEE Trans. Pattern Anal. Machine Intell., vol. 11, pp. 674–693, 1989.
37. H. Ritter, T. Martinetz, and K. Schulten, Neural Computation and Self-Organizing Maps. New York: Addison Wesley, 1992.
38. K. Rose, E. Gurewitz, and G. Fox, “A deterministic annealing approach to clustering,” Pattern Recogn. Lett., vol. 11, no. 4, pp. 589–594, Nov. 1990.
39. K. Rose, E. Gurewitz, and G. Fox, “Statistical mechanics and phase transitions in clustering,” Physical Rev. Lett., vol. 65, no. 8, pp. 945–948, 1990.
40. T. Senoo and B. Girod, “Vector quantization for entropy coding of image subbands,” IEEE Trans. Image Processing, in press.
41. P. D. Simic, “Statistical mechanics as the underlying theory of “elastic” and “neural” optimizations,” Network, vol. 1, pp. 89–103, 1990.
42. P. D. Simic, “Constrained nets for graph matching and other quadratic assignment problems,” Neural Computat., vol. 3, pp. 268–281, 1991.
43. Y. Tikochinsky, N. Z. Tishby, and R. D. Levine, “Alternative approach to maximum-entropy inference,” Physical Rev. A, vol. 30, pp. 2638–2644, 1984.
44. B. Wegmann and C. Zetsche, “Statistical dependence between orientation filter outputs used in an human vision based image code,” in SPIE Proceedings of the Visual Communications and Image Processing ‘90, vol. 1360, M. Kunt, Ed. Bellingham: Int. Soc. Opt. Eng., 1990, pp. 909–923.
45. P. H. Westerink, D. E. Boekee, J. Biemond, and J. W. Woods, “Subband coding of images using vector quantization,” IEEE Trans. Commun., vol. 36, pp. 713–719, 1988.
46. Y. Yamada, S. Tazaki, and R. Gray, “Asymptotic performance of block quantizers with difference distortion measures,” IEEE Trans. Inform. Theory, vol. IT-26, 6–14, Jan. 1980.
47. A. L. Yuille, “Generalized deformable models, statistical physics, and matching problems,” Neural Computat., vol. 2, no. 1, pp. 1–24, 1990.
48. L. Zadeh, “Fuzzy sets,” Inform. Contr., vol. 8, pp. 338–353, 1965.
49. P. L. Zador, “Asymptotic quantization error of continuous signals and the quantization dimension,” IEEE Trans. Information Theory, vol. IT-28, 28, pp. 139–149, Mar. 1982.
50. K. Zeger and A. Gersho, “Pseudo-gray coding,” IEEE Trans. Commun., vol. 38, pp. 2147–2158, Dec. 1990.