Learning Compatibility Coefficients for Relaxation Labeling Processes

IEEE Transactions on Pattern Analysis and Machine Intelligence

Volumn 16, Issue 9, 1994, Pages 933-945

  Pelillo, Marcello ^a   Refice, Mario ^b  

^a UNIVERSITY OF BARI   (Italy)

^b POLITECNICO DI BARI   (Italy)

Author keywords

Compatibility coefficients; constraint satisfaction; gradient projection; learning; neural networks; nonlinear programming; part of speech disambiguation; relaxation labeling

Indexed keywords

ALGORITHMS; ARTIFICIAL INTELLIGENCE; IMAGE PROCESSING; LEARNING SYSTEMS; NEURAL NETWORKS; NONLINEAR PROGRAMMING; OPTIMIZATION; PARALLEL PROCESSING SYSTEMS; PATTERN RECOGNITION; PERFORMANCE;

COMPATIBILITY COEFFICIENTS; CONSTRAINT SATISFACTION; GRADIENT PROJECTION; PART OF SPEECH DISAMBIGUATION; REAL NUMBERS; RELAXATION LABELING;

ITERATIVE METHODS;

EID: 0028499183     PISSN: 01628828     EISSN: None     Source Type: Journal    
DOI: 10.1109/34.310691     Document Type: Article

References (69)

1. R. M. Haralick and L. G. Shapiro, “The consistent labeling problem: Part I,” IEEE Trans. Pattern Anal. Machine lutell., vol. PAMI-1, no. 2, pp. 173–184, 1979.
2. R. M. Haralick, L. S. Davis, A. Rosenfeld, and D. L. Milgram, “Reduction operations for constraint satisfaction,” Inform. Sci., vol. 14, pp. 199–219, 1978.
3. A. Rosenfeld, R. A. Hummel, and S. W. Zucker, “Scene labeling by relaxation operations,” IEEE Trans. Syst., Man, Cybern. vol. SMC-6, no. 6, pp. 420–433, 1976.
4. S. Peleg and A. Rosenfeld, “Breaking substitution ciphers using a relaxation algorithm,” Commun. ACM, vol. 22, no. 11, pp. 598–605, 1979.
5. S. Pclcg, “Ambiguity reduction in handwriting with ambiguous segmentation and uncertain interpretation,” Comput, Graph. Image Processing. vol. 10, pp. 235–245, 1979.
6. J. O. Eklundh and A. Rosenfeld, “Some relaxation experiments using triples of pixels,” IEEE Trans. Syst., Man, Cyhern., vol. SMC-10, no. 3, pp. 150–153, 1980.
7. R. A. Hummel and S. W. Zucker, “On the foundations of relaxation labeling processes.” IEEE Trans. Pattern Anal. Machine linell., vol. PAMI-5, no. 3, pp. 267–287, 1983.
8. O. D. Faugeras and M. Berthod, “Improving consistency and reducing ambiguity in stochastic labeling: An optimization approach,” IEEE Trans. Pattern Anal. Machine Intell. vol. PAMI-3, no. 4, pp. 412–424, 1981.
9. J. J. Hopfield, “Neural networks and physical systems with emergent collective computational abilities,” Proc. Nat. Acad. Sci. USA, vol. 79, pp. 2554 2558, 1982.
10. “Neurons with graded response have collective computational properties like those of two-state neurons,” Proc. Nat. Acad. Sci. USA, vol. 81, pp. 3088–3092, 1984.
11. S. E. Fahlmari, G. E. Hinton, and T. J. Sejnowski, “Massively parallel architectures for Al: NETL, Thistle, and Boltzmann machines,” in Proc. Nat. Conf. Artificial Intell. (AAAI-83). Washington, DC, 1983, pp. 109–113.
12. M. Kamada, K. Toraichi, R. Mori, K. Yamamoto, and H. Yamada, “A parallel architecture for relaxation operations,” Pattern Recognit., vol. 21, no. 2, pp. 175–181, 1988.
13. Z. Chen, S. Lin, and Y. Chen, “A parallel architecture for probabilistic relaxation operations on images,” Pattern Recognit., vol. 23, no. 6, pp. 637–645, 1990.
14. R. M. Haralick, J. L. Mohammed, and S. W. Zucker, “Compatibilities and the fixed points of arithmetic relaxation processes,” Comput. Graph. Image Processing, vol. 13, pp. 242–256, 1980.
15. D. P. O'Leary and S. Peleg, “Analysis of relaxation processes: The two-node node two-label case,” IEEE Trans. Syst., Man, Cyhern., vol. SMC-13, no. 4, pp. 618–623, 1983.
16. S. Peleg and A. Rosenfeld, “Determining compatibility coefficients for curve enhancement relaxation processes,” IEEE Trans. Syst., Man, Cyhern., vol. SMC-8, no. 7, pp. 548–555, 1978.
17. H. Yamamoto, “A method of deriving compatibility coefficients for relaxation operators,” Comput. Graph. Image Processing, vol. 10, pp. 256–271, 1979.
18. S. Peleg, “A new probabilistic relaxation scheme,” IEEE Trans. Pattern Anal. Machine Intell. vol. PAMI-2, no. 4, pp. 362–369, 1980.
19. L. S. Davis and A. Rosenfeld, “Cooperating processes for low-level vision: A survey,” Artificial Intell., vol. 17, pp. 245–263, 1981.
20. J. Kittler, “Compatibility and support functions in probabilistic relaxation, in Proc. 8th Int. Conf. Pattern Recognit., Paris, France, 1986, pp. 186–189.
21. R. A. Hummel, “A design method for relaxation labeling applications,” in Proc. Nat. Conf. Artificial Intell. (44AI-83), Washington, DC, 1983, PP. 168-171.
22. S. W. Zucker, R. A. Hummel, and A. Rosenfeld, “An application of relaxation labeling to line and curve enhancement,” IEEE Trans. Comput., vol. C-26, no. 4, pp. 394–403, 1977.
23. L. S. Davis and A. Rosenfeld, “Curve segmentation by relaxation labeling,” IEEE Trans. Comput., vol. C-26, no. 10, pp. 1053–1057, 1977.
24. J. B. Rosen, “The gradient projection method for nonlinear programming-Part Linear constraints,” J. Soc. Indust. Appl. Math., vol. 8, no. 1, pp. 181-217, 1960.
25. D. G. Luenberger. Linear and Nonlinear Programming. Reading, MA: Addison-Wesley, 1984.
26. S. A. Lloyd, “An optimization approach to relaxation labeling algorithms, Image Vision Comput., vol. 1, no. 2, pp. 85–91, 1983.
27. J. Illingworth and J. Kittler, “Optimization algorithms in probabilistic relaxation labeling,” in Pattern Recognition Theory and Applications, P. A. Devijver and J. Kittler, Eds. Berlin: Springer, 1987, pp. 109–117.
28. L. R. Bahl, F. Jelinek, and R. L. Mercer, “A maximum-likelihood approach to continuous speech recognition,” IEEE Trans. Pattern Anal. Machine Intell. vol. PAMI-5, no. 2, pp. 179–190, 1983.
29. J. Hertz, A. Krogh, and R. G. Palmer, Introduction to the Theory of Neural Computation. Redwood City, CA: Addison-Wesley, 1991.
30. J. Kittler and J. Illingworth, “Relaxation labeling algorithms-A review,” Image Vision Comput., vol. 3, no. 4, pp. 206–216, 1985.
31. X. Zhuang, R. M. Haralick, and H. Joo, “A simplex-like algorithm for the relaxation labeling process,” IEEE Trans. Pattern Anal. Machine Intell. vol. 11, no, 12, pp. 1316–1321, 1989.
32. S. W. Zucker, Y. G. Leclerc, and J. L. Mohammed, “Continuous relaxation and local maxima selection: Conditions for equivalence,” IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-3, no. 3, pp. 117–127, 1981.
33. S. Kullback, Information Theory and Statistics. New York: Wiley, 1959.
34. G. E. Hinton and T. J. Sejnowski, “Learning and relearning in Boltzmann machines,” in Parallel Distributed Processing, Vol. 1: Foundations, D. E. Rumelhart and J. L. McClelland, Eds. Cambridge, MA: MIT Press, 1986, pp. 283–317.
35. S. A. Solla, E. Levin, and M. Fleisher, “Accelerated learning in layered neural networks,” Complex Syst., vol. 2. pp. 625–640, 1988.
36. J. Lin, “Divergence measures based on the Shannon entropy,” IEEE Trans. Inform. Theory, vol. IT-37, no. 1, pp. 145–151, 1991.
37. M. Pelillo and M. Refice, “An optimization algorithm for determining the compatibility coefficients of relaxation labeling processes,” in Proc. 11th Int. Conf. Pattern Recognit., The Hague, The Netherlands, 1992, pp. 145–148.
38. “Learning compatibility coefficients for word-class disambiguation relaxation processes,” in Proc. Int. Conf Spoken Language Processing, Banff, Canada, 1992, pp. 389–392.
39. J. L. Mohammed, R. A. Hummel, and S. W. Zucker, “A gradient projection algorithm for relaxation methods,” IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-5, no. 3, pp. 330–332, 1983.
40. P. Parent and S. W. Zucker, “Radial projection: An efficient update rule for relaxation labeling,” IEEE Trans. Pattern Anal. Machine Intell., vol. 11, no. 8, pp. 886–889, 1989.
41. J. J. Hopfield and D. W. Tank, “Neural computation of decisions in optimization problems,” Biol. Cybern., vol. 52, pp. 141–152, 1985.
42. G. V. Wilson and G. S. Pawley, “On the stability of the traveling salesman problem algorithm of Hoptield and Tank,” Biol. Cybern., vol. 58, pp. 63–70, 1988.
43. G. W. Davis, “Sensitivity analysis in neural net solutions,” IEEE Trans. Syst., Man, Cybern., vol. SMC-19, no. 5, pp. 1078–1082, 1989.
44. T. Elfving and J. O. Eklundh, “Some properties of stochastic labeling procedures,” Comput. Graph. Image Processing, vol. 20, pp. 158–170, 1982.
45. D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning internal representations by error propagation,” in Parallel Distributed Processing, Vol. 1: Foundations, D. E. Rumelhart and J. L. McClelland, Eds. Cambridge, MA: MIT Press, 1986, pp. 318–362.
46. F. J. Pineda, “Generalization of back-propagation to recurrent neural networks,” Phys. Rev. Lett., vol. 59, no. 19, pp. 2229–2232, 1987.
47. L. B. Almeida, “A learning rule for asynchronous perceptrons with feedback in a combinatorial environment,” in Proc. Int. Conf. Neural Networks, San Diego, CA, 1987, pp. 609–618.
48. R. Rohwer and B. Forrest, “Training time-dependence in neural networks,” in Proc. Int. Conf. Neural Networks, San Diego, CA, 1987, pp. 701–708.
49. R. J. Williams and D. Zipser, “A learning algorithm for continually running fully recurrent neural networks,” Neural Computat., vol. 1, pp. 270–280, 1989.
50. C. Torras, “Relaxation and neural learning: Points of convergence and divergence,” J. Parallel Distrib. Comput., vol. 6, pp. 217–244, 1989.
51. S. S. Yu and W. H. Tsai, “Relaxation by the Hopfield neural network,” Pattern Recognit., vol. 25, no. 2, pp. 197–209, 1992.
52. A. M. Derouault and B. Mérialdo, “Natural language modeling for phoneme-to-text transcription,” IEEE Trans. Pattern Anal. Machine Intell. vol. PAMI-8, no. 6, pp. 742–749, 1986.
53. K. W. Church, “A stochastic parts program and noun phrase parser for unrestricted text,” in Proc. Int. Conf. Acoust., Speech, Signal Processing, Glasgow, Scotland, 1989, pp. 695–698.
54. M. Pelillo, F. Moro, and M. Refice, “Probabilistic prediction of parts-of-speech of-speech from word spelling using decision trees,” in Proc. Int. Conf. Spoken Language Processing, Banff, Canada, 1992, pp. 1343–1346.
55. M. Pelillo and M. Refice, “Syntactic category disambiguation through relaxation processes,” in Proc. EUROSPEECH 91, Genova, Italy, 1991, pp. 757–760.
56. L. Boves and M. Refice, “The linguistic processor in a multi-lingual text-to-speech and speech-to-text conversion system,” in Proc. Europ. Conf. Speech Technol., Edimburgh, Scotland, 1987, pp. 385–401.
57. S. M. Katz, “Estimation of probabilities from sparse data for the language model component of a speech recognizer,” IEEE Trans. Acoust., Speech, Signal Processing, vol. ASSP-35, no. 3, pp. 400–401, 1987.
58. I. J. Good, “The population frequencies of species and the estimation of population parameters,” Biometrika, vol. 40, no. 3/4, pp. 237–264, 1953.
59. M. J. J. Holt, “Comparison of generalization in multi-layer perceptrons with the log-likelihood and least-squares cost functions” in Proc. 11th Int. Conf. Pattern Recognit., The Hague, The Netherlands, 1992, pp. 17–20
60. J. Benello, A. W. Mackie, and J. A. Anderson, “Syntatic category disambiguation with neural networks,” Comput. Speech Language, vol. 3, pp. 203–217, 1989.
61. T. A. Jamison and R. I. Schalkoff, “Image labeling: A neural network approach,” Image Vision Comput., vol. 6, no. 4, pp. 203–213, 1988.
62. T. J. Sejnowski and C. R. Rosenberg, “Parallel networks that learn to pronounce English text,” Complex Syst., vol. 1, pp. 145–168, 1987.
63. D. E. Goldberg, Genetic Algorithms in Search, Optimization, and Machine Learning. Reading, MA: Addison-Wesley, 1989.
64. P. J. M. van Laarhoven and E. H. L. Aarts, Simulated Annealing: Theory and Applications. Dordrecht: Reidel, 1987.
65. D. H. Ballard, G. E. Hinton, and T. J. Sejnowski, “Parallel visual computation,” Nature, vol. 306, pp. 21–26, 1983.
66. J. A. Anderson, “Cognitive and psychological computation with neural models,” IEEE Trans. Syst., Man, Cybern., vol. SMC-13, no. 5, pp. 799–815, 1983.
67. S. W. Zucker, A. Dobbins, and L. Iverson, “Two stages of curve detection suggest two styles of visual computation,” Neural Computat., vol. 1, pp. 68–81, 1989.
68. G. Mitchinson, “Learning algorithms and networks of neurons,” in The Computing Neuron, R. Durbin, C. Miall, and G. Mitchinson, Eds. Reading, MA: Addison-Wesley, 1989, pp. 35–53.
69. F. Crick, “The recent excitement about neural networks,” Nature. vol. 337, pp. 129–132, 1989.