A general framework for adaptive processing of data structures

IEEE Transactions on Neural Networks

Volumn 9, Issue 5, 1998, Pages 768-786

  Frasconi, Paolo ^a,b   Gori, Marco ^a,c   Sperduti, Alessandro ^d  

^a IEEE   (Italy)

^b UNIVERSITY OF FLORENCE   (Italy)

^c UNIVERSITY OF SIENA   (Italy)

^d UNIVERSITY OF PISA   (Italy)

Author keywords

Graphical models; Graphs; Learning data structures; Problem solving; Recurrent neural networks; Recursive neural networks; Sequences; Syntactic pattern recognition

Indexed keywords

ADAPTIVE SYSTEMS; DATA PROCESSING; GRAPH THEORY; LEARNING SYSTEMS; MATHEMATICAL MODELS; NEURAL NETWORKS; PATTERN RECOGNITION; RECURSIVE FUNCTIONS;

ADAPTIVE PROCESSING;

DATA STRUCTURES;

EID: 0032165969     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/72.712151     Document Type: Article

References (60)

1. P. Frasconi, M. Gori, and G. Soda, "Recurrent neural networks and prior knowledge for sequence processing: A constrained nondeterministic approach," Knowledge-Based Syst., vol. 8, no. 6, pp. 313-332, 1995.
2. T. J. M. Cabe, "A software complexity measure," IEEE Trans. Software Eng., vol. 2, pp. 308-320, 1976.
3. R. Prather, "Design and analysis of hierarchical software metrics," ACM Computing Surveys, vol. 27, no. 4, pp. 497-518, 1995.
4. N. Nilsson, Principles of Artificial Intelligence. Palo Alto, CA: Tioga, 1980.
5. C. Goller, "A Connectionist Approach for Learning Search-Control Heuristics for Automated Deduction Systems," Ph.D. dissertation, Dept. Comput. Sci., Tech. Univ. Munich, 1997.
6. K. S. Fu, Syntactic Pattern Recognition and Applications. Englewood Cliffs, NJ: Prentice-Hall, 1982.
7. T. Pavlidis, Structural Pattern Recognition. New York: Springer-Verlag, 1977.
8. R. C. Gonzalez and M. G. Thomason, Syntactical Pattern Recognition. Reading, MA: Addison-Wesley, 1978.
9. R. J. Schalhoff, Pattern Recognition: Statistical, Structural and Neural Approaches. New York: Wiley, 1992.
10. Y. Bengio, P. Simard, and P. Frasconi, "Learning long-term dependencies with gradient descent is difficult," IEEE Trans. Neural Networks, vol. 5, pp. 157-166, Mar. 1994.
11. G. E. Hinton, "Mapping part-whole hierarchies into connectionist networks," Artificial Intell., vol. 46, pp. 47-75, 1990.
12. J. B. Pollack, "Recursive distributed representations," Artificial Intell., vol. 46, no. 1-2, pp. 77-106, 1990.
13. T. A. Plate, "Holographic reduced representations," IEEE Trans. Neural Networks, vol. 6, pp. 623-641, May 1995.
14. A. Sperduti, "Labeling RAAM," Connection Sci., vol. 6, no. 4, pp. 429-459, 1994.
15. _, "Encoding labeled graphs by labeling RAAM," in Advances in Neural Information Processing Systems, J. D. Cowan, G. Tesauro, and J. Alspector, Eds. San Mateo, CA: Morgan Kaufmann, vol. 6, pp. 1125-1132, 1994.
16. _, "Stability properties of labeling recursive autoassociative memory," IEEE Trans. Neural Networks, vol. 6, pp. 1452-1460, Nov. 1995.
17. A. Sperduti and A. Starita, "A memory model based on lraam for associative access of structures," in IEEE Int. Conf. Neural Networks, 1996, pp. 543-548.
18. V. Cadoret, "Encoding syntactical trees with labeling recursive autoassociative memory," in Proc. ECAI, Amsterdam, The Netherlands, 1994, pp. 555-559.
19. A. Sperduti and A. Starita, "Supervised neural networks for the classification of structures," IEEE Trans. Neural Networks, vol. 8, 1997.
20. T. J. Sejnowski and C. R. Rosenberg, "Parallel networks that learn to pronounce English text," J. Complex Syst., vol. 1, pp. 145-168, Feb. 1987.
21. J. L. Elman, "Finding structure in time," Cognitive Sci., vol. 14, pp. 179-211, 1990.
22. C. L. Giles and C. W. Omlin, "Extraction, insertion, and refinement of symbolic rules in dynamically driven recurrent neural networks," Connection Sci., vol. 5, no. 3, pp. 307-337, 1993.
23. L. R. Rabiner, "A tutorial on hidden Markov models and selected applications in speech recognition," Proc. IEEE, vol. 77, no. 2, pp. 257-286, 1989.
24. Y. Bengio and P. Frasconi, "Input-output HMM's for sequence processing," IEEE Trans. Neural Networks, vol. 7, pp. 1231-1249, Sept. 1996.
25. B. de Vries and J. C. Principe, "The gamma model - A new neural-net model for temporal processing," Neural Networks, vol. 5, pp. 565-576, 1992.
26. A. D. Back and A. C. Tsoi, "A comparison of discrete-time operator models and for nonlinear system identification," in Advances in Neural Information Processing Systems, G. Tesauro, D. Touretzky, and T. Leen, Eds. Cambridge, MA: MIT Press, vol. 7, 1995, pp. 883-890.
27. J. Pearl, Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. San Mateo, CA: Morgan Kaufmann, 1988.
28. J. Whittaker, Graphical Models in Applied Multivariate Statistics. Chichester, U.K.: Wiley, 1990.
29. R. M. Neal, "Asymmetric parallel Boltzmann machines are belief networks," Neural Comput., vol. 4, no. 6, pp. 832-834, 1992.
30. C. M. Bishop, Neural Networks for Pattern Recognition. Oxford, U.K.: Clarendon, 1995.
31. S. Lauritzen, Graphical Models, no. 17 in Statistical Science Series. Oxford, U.K.: Clarendon, 1996.
32. W. L. Buntine, "Operations for learning with graphical models," J. Artificial Intell. Res., vol. 2, pp. 159-225, 1994.
33. M. L. Forcada and R. C. Carrasco, "Learning the initial state of a second-order recurrent neural network during regular-language inference," Neural Comput., vol. 7, no. 5, pp. 923-930, 1995.
34. J. Thacher, "Tree automata: An informal survey," in Currents in the Theory of Computing, A. Aho, Ed. Englewood Cliffs, NJ: Prentice-Hall, 1973, pp. 143-172.
35. F. V. Jensen, S. L. Lauritzen, and K. G. Olosen, "Bayesian updating in recursive graphical models by local computations," Comput. Statist. Quarterly, vol. 4, pp. 269-282, 1990.
36. P. Frasconi, M. Gori, and A. Sperduti, "Hidden recursive models for probabilistic learning of structured information," preprint, Dipartimento di Sistemi e Informatica, Universitá di Firenze, Firenze, Italy, 1997.
37. L. K. Saul, T. Jaakkola, and M. I. Jordan, "Mean field theory for sigmoid belief networks," J. Artificial Intell. Res., vol. 4, pp. 61-76, 1996.
38. K. Lari and S. J. Young, "The estimation of stochastic context-free grammars using the inside-outside algorithm," Comput. Speech and Language, vol. 4, no. 1, pp. 35-56, 1990.
39. P. Frasconi, M. Gori, M. Maggini, and G. Soda, "Representation of finite-state automata in recurrent radial basis function networks," Machine Learning, vol. 23, pp. 5-32, 1996.
40. C. Giles and T. Maxwell, "Learning, invariance, and generalization in high-order neural networks," Appl. Opt., vol. 26, no. 23, p. 4972, 1987.
41. C. B. Miller and C. L. Giles, "Experimental comparison of the effect of order in recurrent neural networks," Int. J. Pattern Recognition Artificial Intell., (Special issue on Applications of Neural Networks to Pattern Recognition), I. Guyon, Ed., 1993.
42. P. Frasconi, M. Gori, and A. Sperduti, "Optimal learning of data structures," in Proc. Int. Joint Conf. Artificial Intell., Nagoya, Japan, August 23-29, 1997, pp. 1066-1071.
43. L. H. Hall and L. B. Kier, "The molecular connectivity chi indexes and kappa shape indexes in structure-property modeling," in Reviews in Computational Chemistry, K. B. Lipkowitz and D. B. Boyd, Eds. New York: VCH, 1991, pp. 367-422.
44. L. Wos, R. Overbeek, E. Lusk, and J. Boyle, Automated Reasoning: Introduction and Applications. Englewood Cliffs, NJ: Prentice-Hall, 1984.
45. R. Letz, J. Schumann, S. Bayerl, and W. Bibel, "Setheo: A high-performance theorem prover," J. Automated Reasoning, vol. 8, no. 2, pp. 183-212, 1992.
46. C. B. Suttner and W. Ertel, "Using backpropagation for guiding the search of a theorem prover," Int. J. Neural Network Res. Applicat., vol. 2, no. 1, pp. 3-16, 1991.
47. C. Goller and A. Küchler, "Learning task-dependent distributed structure-representations by backpropagation through structure," in IEEE Int. Conf. Neural Networks, 1996, pp. 347-352.
48. A. Sperduti, A. Starita, and C. Goller, "Learning distributed representations for the classification of terms," in Proc. Int. Joint Conf. Artificial Intell., 1995, pp. 509-515.
49. A. Sperduti, D. Majidi, and A. Starita, "Extended cascade-correlation for syntactic and structural pattern recognition," in Advances in Structural and Syntactical Pattern Recognition, P. Perner, P. Wang, and A. Rosenfeld, Eds., vol. 1121 of Lecture notes in Computer Science. Berlin, Germany: Springer-Verlag, 1996, pp. 90-99.
50. S. E. Fahlman, "The recurrent cascade-correlation architecture," in Advances in Neural Information Processing Systems, R. P. Lippmann, J. E. Moody, and D. S. Touretzky, Eds. San Mateo, CA: Morgan Kaufmann, vol. 3, 1991, pp. 190-196.
51. D. Angluin and C. H. Smith, "A survey of inductive inference: Theory and methods," ACM Comput. Survey, vol. 15, pp. 237-269, Sept. 1983.
52. C. Omlin and C. L. Giles, "Constructing deterministic finite-state automata in recurrent neural networks," J. ACM, vol. 43, no. 6, pp. 937-972, 1996.
53. H. T. Siegelmann and E. D. Sontag, "On the computational power of neural nets," J. Comput. Syst. Sci., vol. 50, no. 1, pp. 132-150, 1995.
54. E. M. Gold, "Complexity of automaton identification from given data," Inform. Contr., vol. 37, pp. 302-320, 1978.
55. S. C. Kremer, "A Theory of Grammatical Induction in the Connectionist Paradigm," Ph.D. dissertation, Dept Comput. Sci., Univ. Alberta, Canada, 1996.
56. A. Sperduti, "On the computational power of recurrent neural networks for structures," Neural Networks, to appear, 1997.
57. C. L. Giles, C. B. Miller, D. Chen, H. H. Chen, G. Z. Sun, and Y. C. Lee, "Learning and extracting finite state automata with second-order recurrent neural networks," Neural Comput., vol. 4, no. 3, pp. 393-405, 1992.
58. G. G. Towell and J. W. Shavlik, "Extracting refined rules from knowledge-based neural networks," Machine Learning, vol. 13, pp. 71-101, 1993.
59. R. Neco and M. L. Forcada, "Beyond mealy machines: learning translators with recurrent neural," in Proc. WCNN'96 (World Congr. Neural Networks), San Diego, CA, pp. 408-411.
60. Y. Bengio and F. Gingras, "Recurrent neural networks for missing or asynchronous data," in Advances in Neural Information Processing Systems, M. Mozer, D. Touretzky, and M. Perrone, Eds., vol. 8. Cambridge, MA: MIT Press, 1996.