A neural-network architecture for syntax analysis

IEEE Transactions on Neural Networks

Volumn 10, Issue 1, 1999, Pages 94-114

  Chen, Chun Hsien ^b   Honavar, Vasant ^a,c  

^a IEEE

^b INDUSTRIAL TECHNOLOGY RESEARCH INSTITUTE   (Taiwan)

^c Iowa State University   (United States)

Author keywords

Lexical analysis; Modular neural networks; Neural associative processing; Neural associative processor; Neural parser; Neural symbolic processing; Parsing; Syntax analysis

Indexed keywords

ARTIFICIAL INTELLIGENCE; ASSOCIATIVE PROCESSING; COMPUTATIONAL LINGUISTICS; COMPUTER ARCHITECTURE; COMPUTER SYSTEMS PROGRAMMING; PARALLEL PROCESSING SYSTEMS; PROGRAM COMPILERS;

LEXICAL ANALYSIS; MODULAR NEURAL NETWORKS; NEURAL ASSOCIATIVE PROCESSING; NEURAL SYMBOLIC PROCESSING; SYNTAX ANALYSIS;

NEURAL NETWORKS;

EID: 0032786152     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/72.737497     Document Type: Article

References (105)

1. A. V. Aho, R. Sethi, and J. D. Ullman, Compilers: Principles, Techniques, and Tools. Reading, MA: Addison-Wesley, 1986.
2. A. V. Aho and J. D. Ullman, Principles of Compiler Design. Reading, MA: Addison-Wesley, 1977.
3. R. B. Allen, "Connectionist language users," Connection Sci., vol. 2, no. 4, p. 279, 1990.
4. M. Arbib, "Schema theory: Cooperative computation for brain theory and distributed AI," in Artificial Intelligence and Neural Networks: Steps Toward Principled Integration, V. Honavar and L. Uhr, Eds. San Diego, CA: Academic, 1994, pp. 51-74.
5. G. Berg, "A connectionist Parser with recursive sentence structure and lexical disambiguation," in Proc. 10th Nat. Conf. Artificial Intell., 1992, pp. 32-37.
6. L. A. Bookman, "A framework for integrating relational and associational knowledge for comprehension, in Computational Architectures Integrating Neural and Symbolic Processes: A Perspective on the State of the Art, R. Sun and L. Bookman, Eds. Norwell, MA: Kluwer Academic Publishers, 1995, ch. 9, pp. 283-318.
7. N. P. Chapman, LR Parsing: Theory and Practice.Cambridge. U.K.: Cambridge Univ. Press, 1987.
8. C. Chen and V. Honavar, "Neural-network automata," in Proc. World Congr. Neural Networks, San Diego, CA, vol. 4, pp. 470-477, June 1994.
9. C. Chen and V. Honavar, "A neural architecture for content as well as address-based storage and recall: Theory and applications," Connection Sci., vol. 7, nos. 3/4, pp. 281-300, 1995.
10. _, "A neural-network architecture for high-speed database query processing system," Microcomputer Applicat., vol. 15, no. 1, pp. 7-13, 1996.
11. S. Das, C. L. Giles, and G. Z. Sun, "Using prior knowledge in a NNDPA to learn context-free languages," in Advances in Neural Information Processing Systems 5, S. J. Hanson, J. D. Cowan, and C. L. Giles, Eds. San Mateo, CA: Morgan Kaufmann, 1993, pp. 65-72.
12. C. P. Dolan and P. Smolensky, "Tensor product production system: A modular architecture and representation," Connection Sci., vol. 1, pp. 53-58, 1989.
13. M. G. Dyer, "Connectionist natural language processing: A status report," in Computational Architectures Integrating Neural and Symbolic Processes: A Perspective on the State of the Art, R. Sun and L. Bookman, Eds. Norwell, MA: Kluwer, 1995, ch. 12, pp. 389-429.
14. J. L. Elman, "Finding structure in time," Cognitive Sci., vol. 14., pp. 179-211, 1990.
15. M. A. Fanty, "Context-free parsing with connectionist networks," Proc. AIP Neural Networks for Computing, Conf. 151, Snowbird, UT, 1986, pp. 140-145.
16. P. Frasconi, M. Gori, M. Maggini, and G. Soda, "Unified integration of explicit rules and learning by example in recurrent networks," IEEE Trans. Knowledge Data Eng., 1993.
17. C. L. Giles, B. W. Home, and T. Lin, "Learning a class of large finite state machines with a recurrent neural network," Neural Networks, vol. 8, no. 9, pp. 1359-1365, 1995.
18. C. L. Giles, C. G. Miller, D. Chen, G. Z. Sun, and Y. C. Lee, "Learning and extracting finite state automata with second-order recurrent neural networks," Neural Comput., vol. 4., no. 3., p. 380, 1992.
19. L. Goldfarb and S. Nigam, "The unified learning paradigm: A foundation for AI," in Artificial Intelligence and Neural Networks: Steps Toward Principled Integration, V. Honavar and L. Uhr, Eds. San Diego, CA: Academic, 1994, pp. 533-559.
20. S. Goonatilake and S. Khebbal, Eds., Intelligent hybrid systems. London, U.K.: Wiley, 1995.
21. S. M. Gowda et al., "Design and characterization of analog VLSI neural-network modules," IEEE J. Solid-State Circuits, vol. 28, pp. 301-313, Mar. 1993.
22. H. P. Graf and D. Henderson, "A reconfigurable CMOS neural network," in ISSCC Dig. Tech. Papers, San Francisco, CA, 1990, pp. 144-145.
23. D. Grant et al., "Design, implementation and evaluation of a high-speed integrated hamming neural classifier," IEEE J. Solid-State Circuits, vol. 29, pp. 1154-1157, Sept. 1994.
24. K. E. Grosspietsch, "Intelligent systems by means of associative processing," in Fuzzy, Holographic, and Parallel Intelligence, B. Souček and the IRIS Group, Eds. New York: Wiley, 1992, pp. 179-214.
25. A. Gupta, "Parallelism in Production Systems," Ph.D. dissertation, Carnegie-Mellon Univ., Pittsburgh, PA, Mar. 1986.
26. A. Hamilton et al., "Integrated pulse stream neural networks: Results, issues, and pointers," IEEE Trans. Neural Networks, vol. 3, pp. 385-393, May 1992.
27. M. Hassoun, Fundamentals of Artificial Neural Networks. Cambridge, MA: MIT Press, 1995.
28. J. Hendler, "Beyond the fifth generation: Parallel AI research in Japan," IEEE Expert, Feb. 1994, pp. 2-7.
29. K. A. Hester et al., "The predictive RAAM: A RAAM that can learn to distinguish sequences from a continuous input stream," in Proc. World Congr. Neural Networks, San Diego, CA, vol. 4, June 1994 pp. 97-103.
30. G. E. Hinton, "Implementing semantic networks in parallel hardware," in Parallel Models of Associative Memory, G. E. Hinton and J. A. Anderson, Eds. Hillsdale, NJ: Lawrence Erlbaum, 1989.
31. V. Honavar, "Toward learning systems that integrate different strategies and representations," in Artificial Intelligence and Neural Networks: Steps Toward Principled Integration, V. Honavar and L. Uhr, Eds. San Diego, CA: Academic, 1994, pp. 615-644.
32. _, "Symbolic artificial intelligence and numeric artificial neural networks: Toward a resolution of the dichotomy," in Computational Architectures Integrating Symbolic and Neural Processes, R. Sun and L. Bookman, Eds. Norwell, MA: Kluwer, 1995, pp. 351-388.
33. V. Honavar and L. Uhr, "Coordination and control structures and processes: Possibilities for connectionist networks," J. Experimental Theoretical Artificial Intell., vol. 2, pp. 277-302, 1990.
34. _, Eds., Artificial Intelligence and Neural Networks: Steps Toward Principled Integration. San Diego, CA: Academic, 1994.
35. _, "Integrating symbol processing systems and connectionist networks," in Intelligent Hybrid Systems, S. Goonatilake and S. Khebbal, Eds. London, U.K.: Wiley, 1995, pp. 177-208.
36. J. E. Hopcroft and J. D. Ullman, Introduction to Automata Theory, Languages, and Computation. Reading, MA: Addison-Wesley, 1979.
37. B. Horne, D. R. Hush, and C. Abdallah, "The state-space recurrent neural network with application to regular grammatical inference," Dept. Electr. Comput. Eng., Univ. New Mexico, Albuquerque, NM, UNM Tech. Rep. EECE 92-002, 1992.
38. D. B. Howe and K. Asanovic, "SPACE: Symbolic processing in associative computing elements," in VLSI for Neural Networks and Artificial Intelligence, J. G. Delgado-Frias, Ed. New York: Plenum, 1994, pp. 243-252.
39. A. N. Jain, A. Waibel, and D. S. Touretzky, "PARSEC: A structured connectionist parsing system for spoken language," in IEEE Proc. Int. Conf. Acoust., Speech, Signal Processing, San Francisco, CA, Mar. 1992, pp. 205-208.
40. S. C. Kleene, "Representation of events in nerve nets and finite automata," in Automata Studies, C. E. Shannon and J. McCarthy, Eds. Princeton, NJ: Princeton Univ., 1956, pp. 3-42.
41. P. Kogge, J. Oldfield, M. Brule, and C. Stormon, "VLSI and rule-based systems," in VLSI for Artificial Intelligence, J. G. Delgado-Frias and W. R. Moore, Eds. Norwell, MA: Kluwer, 1989, pp. 95-108.
42. R. Kumar, "NCMOS: A high performance CMOS logic," IEEE J. Solid-State Circuits, vol. 29, pp. 631-633, 1994.
43. R. C. Lacher and K. D. Nguyen, "Hierarchical architectures for reasoning," in Computational Architectures Integrating Neural and Symbolic Processes: A Perspective on the State of the Art, R. Sun and L. Bookman, Eds. Norwell, MA: Kluwer, ch. 4, pp. 117-150, 1995.
44. P. Langley, Elements of Machine Learning. Palo Alto, CA: Morgan Kaufmann, 1995.
45. S. H. Lavington, C. J. Wang, N. Kasabov, and S. Lin, "Hardware support for data parallelism in production systems," in VLSI for Neural Networks and Artificial Intelligence, J. G. Delgado-Frias, Ed. New York: Plenum, 1994, pp. 231-242.
46. D. Levine and M. Aparicio IV, Eds., Neural Networks for Knowledge Representation and Inference. Hillsdale, NJ: Lawrence Erlbaum, 1994.
47. J. B. Lont and W. Guggenbühl, "Analog CMOS implementation of a multilayer perceptron with nonlinear synapses," IEEE Trans. Neural Networks, vol. 3, pp. 457-465, 1992.
48. F. Lu and H. Samueli, "A 200-MHz CMOS pipelined multiplier-accumulator using a quasidomino dynamic full-adder cell design," IEEE J. Solid-State Circuits, vol. 28, pp. 123-132, 1993.
49. B. J. MacLennan, Principles of Programming Languages: Design, Evaluation, and Implementation, 2nd ed. New York: CBS College, 1987.
50. P. Masa, K. Hoen, and H. Wallinga, "70 input, 20 nanosecond pattern classifier," IEEE Int. Joint Conf. Neural Networks, Orlando, FL, vol. 3, 1994.
51. L. W. Massengill and D. B. Mundie, "An analog neural-network hardware implementation using charge-injection multipliers and neuron-specific gain control, IEEE Trans. Neural Networks, vol. 3, pp. 354-362, May 1992.
52. W. S. McCulloch and W. Pitts, "A logical calculus of ideas immanent in nervous activity," Bull. Math. Biophys., vol. 5, pp. 115-133, 1943.
53. D. McGregor, S. McInnes, and M. Henning, "An architecture for associative processing of large knowledge bases (LKB's)," Computer J., vol. 30, no. 5, pp. 404-412, Oct. 1987.
54. L. Miclet, Structural Methods in Pattern Recognition. New York: Springer-Verlag, 1986.
55. B. Miikkulainen, "Subsymbolic parsing of embedded structures," in Computational Architectures Integrating Neural and Symbolic Processes: A Perspective on the State of the Art, R. Sun and L. Bookman, Eds. Norwell, MA: Kluwer., ch. 5, pp. 153-186, 1995.
56. M. Minsky, Computation: Finite and Infinite Machines. Englewood Cliffs, NJ: Prentice-Hall, 1967.
57. M. Minsky and S. Papert, Perceptrons: An Introduction to Computational Geometry. Cambridge, MA: MIT Press, 1969.
58. E. Mjolsness, "Connectionist grammars for high-level vision," in Artificial Intelligence and Neural Networks: Steps Toward Principled Integration, V. Honavar and L. Uhr, Eds. San Diego, CA: Academic, 1994, pp. 423-451.
59. G. Moon et al., "VLSI implementation of synaptic weighting and summing in pulse coded neural-type cells," IEEE Trans. Neural Networks, vol. 3, pp. 394-403, May 1992.
60. M. C. Mozer and J. Bachrach, "Discovering the structure of a reactive environment by exploration," Neural Comput., vol. 2, no. 4, p. 447, 1990.
61. M. C. Mozer and S. Das, "A connectionist symbol manipulator that discovers the structure of context-free languages," Advances in Neural Inform. Processing Syst. San Mateo, CA: Morgan Kaufmann, vol. 5, p. 863, 1993.
62. J. Naganuma, T. Ogura, S. I. Yamada, and T. Kimura, "High-speed CAM-based architecture for a prolog machine (ASCA)," IEEE Trans. Comput., vol. 37, pp. 1375-1383, Nov. 1988.
63. Y. H. Ng, R. J. Glover, and C. L. Chng, "Unify with active memory," in VLSI for Artificial Intell., J. G. Delgado-Frias and W. R. Moore, Eds. Norwell, MA: Kluwer, 1989, pp. 109-118.
64. I. Noda and M. Nagao, "A learning method for recurrent neural networks based on minimization of finite automata," in Proc. Int. Joint Conf. Neural Networks. Piscataway, NJ: IEEE Press, vol. 1, 1992, pp. 27-32.
65. D. A. Norman, "Reflections on cognition and parallel distributed processing," in Parallel Distributed Processing, J. McClelland, D. Rumelhart, and the PDP Research Group, Eds. Cambridge, MA: MIT Press, 1986.
66. C. Omlin and C. L. Giles, "Extraction and insertion of symbolic information in recurrent neural networks," in Artificial Intelligence and Neural Networks: Steps Toward Principled Integration, V. Honavar and L. Uhr, Eds. San Diego, CA: Academic, 1994, pp. 271-299.
67. _, "Stable encoding of large finite-state automata in recurrent neural networks with sigmoid discriminants," Neural Computat., vol. 8, no. 4, pp. 675-696, May 1996.
68. R. G. Parekh and V. Honavar, "Learning regular languages from simple examples," Dept. Comput. Sci., Iowa State Univ., Tech. Rep. ISU-CS-TR 97-06, 1997.
69. R. G. Parekh, C. Nichitiu, and V. Honavar, "A polynomial time incremental algorithm for regular grammar inference," Dept. Comput. Sci., Iowa State Univ., Tech. Rep. ISU-CS-TR 97-03, 1997.
70. R. G. Parekh and V. Honavar, "Automata induction, grammar inference, and language acquisition," in Handbook of Natural Language Processing, H. Moisl, R. Dale, and H. Somers, Eds. New York: Marcel Dekker, 1998.
71. B. Péter, Recursive Functions in Computer Theory. New York: Halsted, 1981.
72. G. Pinkas, "A fault-tolerant connectionist architecture for construction of logic proofs, in Artificial Intelligence and Neural Networks: Steps Toward Principled Integration, V. Honavar and L. Uhr, Eds. San Diego, CA: Academic, 1994, pp. 321-340.
73. J. B. Pollack, On Connectionist Models of Language Processing, Ph.D. dissertation, Comput. Sci. Dept., Univ. Illinois, Urbana-Champaign, 1987.
74. _, "Recursive distributed representations, "Artificial Intell., vol. 46, pp. 77-105, 1990.
75. I. Popescu, "Hierarchical neural networks for rules control in knowledge-based expert systems," Neural, Parallel Sci. Comput., vol. 3, pp. 379-392, 1995.
76. B. D. Ripley, Pattern Recognition and Neural Networks. New York: Cambridge Univ., 1996.
77. I. Robinson, "The pattern addressable memory: Hardware for associative processing," in VLSI for Artificial Intelligence, J. G. Delgado-Frias and W. R. Moore, Eds. Norwell, MA: Kluwer Academic Publishers, 1989, pp. 119-129.
78. M. E. Robinson et al., "A modular CMOS design of a Hamming network," IEEE Trans. Neural Networks, vol. 3, pp. 444-456, 1992.
79. D. Rodohan and R. Glover, "A distributed parallel associative processor (DPAP) for the execution of logic programs," in VLSI for Neural Networks and Artificial Intelligence, J. G. Delgado-Frias, Ed. New York: Plenum, 1994, pp. 265-273.
80. H. Rogers, Jr., Theory of Recursive Functions and Effective Computability. Cambridge, MA: MIT Press, 1987.
81. D. E. Rumelhart, J. L. McClelland, and the PDP Research Group, Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Vol. 1: Foundations. Cambridge, MA: MIT Press, 1986.
82. A. Sanfeliu and R. Alquezar, "Understanding neural networks for grammatical inference and recognition," in Advances in Structural and Syntactic Pattern Recognition, H. Bunke, Ed. Singapore: World, 1992.
83. W. Schneider, "Connectionism: Is it a paradigm shift for psychology?," Behavior Res. Methods, Instruments, Comput., vol. 19, pp. 73-83, 1987.
84. D. Schulenburg, "Sentence processing with realistic feedback, in IEEE/INNS Int. Joint Conf. Neural Networks, Baltimore, MD, vol. IV, pp. 661-666, 1992.
85. B. Selman and G. Hirst, "A rule-based connectionist parsing system," in Proc. 7th Annu. Conf. Cognitive Sci. Soc., Irvine, CA, 1985.
86. C. H. Séquin and R. D. Clay, "Fault tolerance in artificial neural networks," in Proc. IJCNN, San Diego, CA, vol. 1, pp. 703-708, 1990.
87. D. Servan-Schreiber, A. Cleeremans, and J. L. McClelland, "Graded state machines: The representation of temporal contingencies in simple recurrent neural networks," in Artificial Intell. Neural Networks: Steps Toward Principled Integration, V. Honavar and L. Uhr, Eds. San Diego, CA: Academic., 1994, pp. 241-269.
88. L. Shastri and V. Ajjanagadde, "Connectionist system for rule based reasoning with multiplace predicates and variables," Comput. Inform. Sci. Dept., Univ. Pennsylvania, Philadelphia, PA, Tech. Rep. MS-CIS-8906, 1989.
89. J. W. Shavlik, "A framework for combining symbolic and neural learning," in Artificial Intelligence and Neural Networks: Steps Toward Principled Integration, V. Honavar and L. Uhr, Eds. San Diego, CA: Academic, 1994, pp. 561-580.
90. H. T. Siegelman and E. D. Sontag, "Turing-computability with neural nets," Appl. Math. Lett., vol. 4, no. 6, pp. 77-80, 1991.
91. S. Sippu and E. Soisalon-Soininen, "Parsing theory, vol. II: LR(k) nad LL(k) parsing." Berlin, Germany: Springer-Verlag, 1990.
92. B. Souček and the IRIS Group, Eds., "Neural and intelligent systems integrations: Fifth and sixth generation integrated reasoning information systems." New York: Wiley, 1991.
93. G. Z. Sun, C. L. Giles, H. H. Chen, and Y. C., The Neural Network Pushdown Automation: Model, Stack and Learning Simulations, UMIA CS-TR-93-77, Univ. Maryland, College Park, MD, Aug. 1993.
94. R. Sun, "Logics and variables in connectionist models: A brief overview," in Artificial Intelligence and Neural Networks: Steps Toward Principled Integration, V. Honavar, and I. Uhr, Eds. San Diego, CA: Academic, 1994, pp. 301-320.
95. R. Sun and L. Bookman, Eds., Computational Architectures Integrating Symbolic and Neural Processes. Norwell, MA: Kluwer, 1995.
96. G. Swaminathan, S. Srinivasan, S. Mitra, J. Minnix, B. Johnson, and R. Inigo, "Fault tolerance of neural networks," Proc. IJCNN, Washington, D.C., vol. 2, pp. 699-702, 1990.
97. K. J. Thurber and L. D. Wald, "Associative and parallel processors," Comput. Surveys, vol. 7, no. 4, pp. 215-255, 1975.
98. K. Uchimura et al., "An 8G connection-per-second 54 mW digital neural network with low-power chain-reaction architecture, ISSCC Dig. Tech. Papers, San Francisco, CA, 1992, pp. 134-135.
99. L. Uhr and V. Honavar, "Artificial intelligence and neural networks: Steps toward principled integration," in Artificial Intelligence and Neural Networks: Steps Toward Principled Integration, V. Honavar and L. Uhr, Eds. San Diego, CA: Academic, 1994, pp. xvii-xxxii.
100. F. Van der Velde, "Symbol manipulation with neural networks: Production of a context-free language using a modifiable working memory," Connection Sci., vol. 7, no. 3/4, pp. 247-280, 1995.
101. 6 synapse neural network," IEEE Trans. Neural Networks, vol. 4, pp. 387-393, May 1993.
102. R. L. Watrous and G. M. Kuhn, "Induction of finite-state languages using second-order recurrent neural networks," Neural Comput., vol. 4, no. 3, p. 406, 1992.
103. R. J. Williams and D. Zipser, "A learning algorithm for continually running fully recurrent neural networks," Neural Comput., vol. 1, pp. 270-280, 1989.
104. D. Wood, Theory of Computation. New York: Wiley, 1987.
105. Z. Zeng, R. M. Goodman, and P. Smyth, "Discrete recurrent neural networks for grammatical inference," IEEE Trans. Neural Networks, vol. 5, pp. 320-330, Mar. 1994.