Learning and representing temporal knowledge in recurrent networks

IEEE Transactions on Neural Networks

Volumn 22, Issue 12 PART 2, 2011, Pages 2409-2421

  Borges, Rafael V ^a   D'Avila Garcez, Artur ^a   Lamb, Luis C ^b  

^a CITY UNIVERSITY   (United Kingdom)

^b FEDERAL UNIVERSITY OF RIO GRANDE DO SUL   (Brazil)

Author keywords

Integrating domain knowledge into nonlinear models; knowledge extraction; model verification; neural symbolic computation; recurrent neural networks; temporal knowledge learning; temporal logic reasoning

Indexed keywords

KNOWLEDGE EXTRACTION; MODEL VERIFICATION; NON-LINEAR MODEL; TEMPORAL KNOWLEDGE LEARNING; TEMPORAL LOGIC REASONING;

ARTIFICIAL INTELLIGENCE; INTEGRATION; KNOWLEDGE ACQUISITION; KNOWLEDGE REPRESENTATION; MATHEMATICAL MODELS; RECURRENT NEURAL NETWORKS;

KNOWLEDGE BASED SYSTEMS;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; COMPUTER SIMULATION; NONLINEAR SYSTEM;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; COMPUTER SIMULATION; NONLINEAR DYNAMICS;

EID: 83655201111     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2011.2170180     Document Type: Article

References (42)

1. A. S. d'Avila Garcez, L. C. Lamb, and D. M. Gabbay, Neural-Symbolic Cognitive Reasoning (Cognitive Technologies). New York: Springer-Verlag, 2009.
2. E. A. Feigenbaum, "Some challenges and grand challenges for computational intelligence," J. ACM, vol. 50, no. 1, pp. 32-40, Jan. 2003.
3. L. G. Valiant, "Three problems in computer science," J. ACM, vol. 50, no. 1, pp. 96-99, Jan. 2003.
4. S. Bader and P. Hitzler, "Dimensions of neural-symbolic integration-a structured survey," in We Will Show Them! Essays in Honour of Dov Gabbay (International Federation for Computational Logic), S. Artëmov, H. Barringer, A. d'Avila Garcez, L. Lamb, and J. Woods, Eds. Tucson, AZ: College Publications, 2005, pp. 167-194.
5. J. Lehmann, S. Bader, and P. Hitzler, "Extracting reduced logic programs from artificial neural networks," Appl. Intell., vol. 32, no. 3, pp. 249-266, 2010.
6. I. Cloete and J. Zurada, Knowledge-Based Neurocomputing. Cambridge, MA: MIT Press, 2000.
7. E. M. Clarke, E. A. Emerson, and J. Sifakis, "Model checking: Algorithmic verification and debugging," Commun. ACM, vol. 52, no. 11, pp. 74-84, Nov. 2009.
8. M. Fisher, D. Gabbay, and L. Vila, Handbook of Temporal Reasoning in Artificial Intelligence. Amsterdam, The Netherlands: Elsevier, 2005.
9. A. Pnueli, "The temporal logic of programs," in Proc. 18th IEEE Symp. Found. Comp. Sci., Providence, RI, Oct.-Nov. 1977, pp. 46-67.
10. D. Zhang and J. Tsai, Machine Learning Applications in Software Engineering. River Edge, NJ: World Scientific, 2005.
11. D. Alrajeh, J. Kramer, A. Russo, and S. Uchitel, "Learning operational requirements from goal models," in Proc. ICSE, May 2009, pp. 265-275.
12. H. T. Siegelmann, B. G. Horne, and C. L. Giles, "Computational capabilities of recurrent NARX neural networks," IEEE Trans. Syst., Man Cybern., Part B, vol. 27, no. 2, pp. 208-215, Apr. 1997. (Pubitemid 127770652)
13. T. Lin, B. Horne, P. Tino, and C. L. Giles, "Learning long-term dependencies in NARX recurrent neural networks," IEEE Trans. Neural Netw., vol. 7, no. 6, pp. 1329-1338, Nov. 1996. (Pubitemid 126780474)
14. L. C. Lamb, R. V. Borges, and A. S. d'Avila Garcez, "A connectionist cognitive model for temporal synchronization and learning," in Proc. 22nd AAAI Conf. Artif. Intell., 2007, pp. 827-832. (Pubitemid 350149672)
15. R. Sun, "Robust reasoning: Integrating rule-based and similarity-based reasoning," Artif. Intell., vol. 75, no. 2, pp. 241-295, Jun. 1995.
16. A. d'Avila Garcez, K. Broda, and D. Gabbay, Neural-Symbolic Learning Systems: Foundations and Applications (Perspectives in Neural Computing). New York: Springer-Verlag, 2002.
17. A. d'Avila Garcez and L. Lamb, "A connectionist computational model for epistemic and temporal reasoning," Neural Comput., vol. 18, no. 7, pp. 1711-1738, Jul. 2006. (Pubitemid 44024735)
18. A. d'Avila Garcez, L. Lamb, and D. Gabbay, "Connectionist computations of intuitionistic reasoning," Theoret. Comput. Sci., vol. 358, no. 1, pp. 34-55, Jul. 2006. (Pubitemid 44067327)
19. A. d'Avila Garcez, M. D. Gabbay, and S. Artur, "Connectionist modal logic: Representing modalities in neural networks," Theoret. Comput. Sci., vol. 371, nos. 1-2, pp. 34-53, 2007.
20. R. Andrews, J. Diederich, and A. B. Tickle, "A survey and critique of techniques for extracting rules from trained artificial neural networks," Knowl.-Based Syst., vol. 8, no. 6, pp. 373-389, Dec. 1995.
21. H. Barringer, M. Fisher, D. Gabbay, G. Gough, and R. Owens, "METATEM: An introduction," Formal Asp. Comput., vol. 7, no. 5, pp. 533-549, 1995.
22. S. Muggleton and L. Raedt, "Inductive logic programming: Theory and methods," J. Logic Program., vol. 19, no. 20, pp. 629-679, 1994.
23. T. M. Mitchell, Machine Learning. New York: McGraw-Hill, 1997.
24. J. L. Elman, "Finding structure in time," Cognit. Sci., vol. 14, no. 2, pp. 179-211, 1990.
25. D. E. Rumelhart, G. E. Hinton, and R. J. Williams, "Learning internal representations by error propagation," Parallel Distrib. Process., vol. 1, pp. 318-362, Oct. 1986.
26. S. Pinker, M. A. Nowak, and J. J. Lee, "The logic of indirect speech," Proc. Nat. Acad. Sci., vol. 105, no. 3, pp. 833-838, Jan. 2008. (Pubitemid 351282044)
27. L. Shastri, "Shruti: A neurally motivated architecture for rapid, scalable inference," in Perspectives of Neural-Symbolic Integration, B. Hammer and P. Hitzler, Eds. New York: Springer-Verlag, 2007, pp. 183-203.
28. R. Sun, "Theoretical status of computational cognitive modeling," Cognit. Syst. Res., vol. 10, no. 2, pp. 124-140, 2009.
29. L. Valiant, "A neuroidal architecture for cognitive computation," J. ACM, vol. 47, no. 5, pp. 854-882, 2000.
30. B. Hammer and P. Hitzler, Perspectives of Neural-Symbolic Integration. New York: Springer-Verlag, 2007.
31. H. Gust and K.-U. Kuehnberger, "Learning symbolic inferences with neural networks," in Proc. 27th Ann. Conf. Cognit. Sci. Soc., 2005, pp. 875-880.
32. S. Bader, P. Hitzler, S. Holldobler, and A. Witzel, "A fully connectionist model generator for covered first-order logic programs," in Proc. Int. Joint Conf. Artif. Intell., 2007, pp. 666-671.
33. L. de Penning, A. S. d'Avila Garcez, L. C. Lamb, and J. J. Meyer, "A neural-symbolic cognitive agent for online learning and reasoning," in Proc. 22nd Int. Joint Conf. Artif. Intell., Barcelona, Spain, Jul. 2011, pp. 1653-1658.
34. P. Hitzler and A. K. Seda, "Characterizations of classes of programs by three-valued operators," in Proc. 5th Int. Conf. Logic Prog. Non-Monotonic Reason., 1999, pp. 357-371.
35. A. S. d'Avila Garcez, K. Broda, and D. M. Gabbay, "Symbolic knowledge extraction from trained neural networks: A sound approach," Artif. Intell., vol. 125, nos. 1-2, pp. 155-207, 2001. (Pubitemid 32130122)
36. R. Setiono, W. Leow, and J. Zurada, "Extraction of rules from artificial neural networks for nonlinear regression," IEEE Trans. Neural Netw., vol. 13, no. 3, pp. 564-577, May 2002. (Pubitemid 34669648)
37. R. Setiono, B. Baesens, and C. Mues, "Recursive neural network rule extraction for data with mixed attributes," IEEE Trans. Neural Netw., vol. 19, no. 2, pp. 299-307, Feb. 2008. (Pubitemid 351359294)
38. A. Cimatti, E. M. Clarke, F. Giunchiglia, and M. Roveri, "NUSMV: A new symbolic model verifier," in Proc. Int. Conf. Comput. Aided Verificat., 1999, pp. 495-499.
39. S. Goedertier, D. Martens, J. Vanthienen, and B. Baesens, "Robust process discovery with artificial negative events," J. Mach. Learn. Res., vol. 10, pp. 1305-1340, Jun. 2009.
40. A. S. d'Avila Garcez, A. Russo, B. Nuseibeh, and J. Kramer, "An analysis-revision cycle to evolve requirements specifications," in Proc. 16th IEEE Int. Conf. Autom. Softw. Eng., San Diego, CA, Nov. 2001, pp. 354-358.
41. D. Alrajeh, O. Ray, A. Russo, and S. Uchitel, "Using abduction and induction for operational requirements elaboration," J. Appl. Logic, vol. 7, no. 3, pp. 275-288, 2009.
42. D. Peled, M. Y. Vardi, and M. Yannakakis, "Black box checking," J. Autom. Lang. Combinator., vol. 7, no. 2, pp. 225-246, 2001.