Fast relational learning using bottom clause propositionalization with artificial neural networks

Machine Learning

Volumn 94, Issue 1, 2014, Pages 81-104

  França, Manoel V M ^a,b   Zaverucha, Gerson ^a   D'Avila Garcez, Artur S ^b  

^a FEDERAL UNIVERSITY OF RIO DE JANEIRO   (Brazil)

^b CITY UNIVERSITY   (United Kingdom)

Author keywords

Artificial neural networks; Inductive logic programming; Neural symbolic integration; Propositionalization; Relational learning

Indexed keywords

BACK-GROUND KNOWLEDGE; MACHINE LEARNING APPLICATIONS; NEURAL-SYMBOLIC INTEGRATION; NEURAL-SYMBOLIC SYSTEMS; PROPOSITIONAL LOGIC; PROPOSITIONALIZATION; RELATIONAL LEARNING; STATISTICAL FEATURES;

C (PROGRAMMING LANGUAGE); FEATURE EXTRACTION; FORMAL LOGIC; NEURAL NETWORKS; SEMANTICS;

INDUCTIVE LOGIC PROGRAMMING (ILP);

EID: 84891373440     PISSN: 08856125     EISSN: 15730565     Source Type: Journal    
DOI: 10.1007/s10994-013-5392-1     Document Type: Article

References (55)

1. Bain, M., & Muggleton, S. (1994). Learning optimal chess strategies. Machine Intelligence, 13, 291-309.
2. Basilio, R., Zaverucha, G., & Barbosa, V. (2001). Learning logic programs with neural networks. In LNAI: Vol. 2157. Proc. ILP (pp. 402-408). Berlin: Springer.
3. Caruana, R., Lawrence, S., & Giles, C. L. (2000). Overfitting in neural nets: backpropagation, conjugate gradient, and early stopping. In Proc. NIPS (Vol. 13, pp. 402-408). Cambridge: MIT Press.
4. Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002). SMOTE: synthetic minority over-sampling technique. Journal of Artificial Intelligence Research, 16(1), 321-357.
5. Clark, P., & Niblett, T. (1989). The CN2 induction algorithm. Machine Learning, 3, 261-283.
6. Copelli, M., Eichhorn, R., Kinouchi, O., Biehl, M., Simonetti, R., Riegler, P., & Caticha, N. (1997). Noise robustness in multilayer neural networks. Europhysics Letters, 37(6), 427-432.
7. Craven, M., & Shavlik, J. W. (1995). Extracting tree-structured representations of trained networks. In Proc. NIPS (Vol. 9, pp. 24-30). Cambridge: MIT Press.
8. Davis, J., Burnside, E. S., Dutra, I. C., Page, D., & Costa, V. S. (2005). An integrated approach to learning Bayesian networks of rules. In LNAI: Vol. 3720. Proc. ECML (pp. 84-95). Berlin: Springer.
9. De Raedt, L. (2008). Logical and relational learning. Berlin: Springer.
10. De Raedt, L., Frasconi, P., Kersting, K., & Muggleton, S. (2008). LNAI: Vol. 4911. Probabilistic inductive logic programming. Berlin: Springer.
11. DiMaio, F., & Shavlik, J. W. (2004). Learning an approximation to inductive logic programming clause evaluation. In LNAI: Vol. 3194. Proc. ILP (pp. 80-97). Berlin: Springer.
12. Ding, C., & Peng, H. (2005). Minimum redundancy feature selection from microarray gene expression data. Journal of Bioinformatics and Computational Biology, 3(2), 185-205.
13. Džeroski, S., & Lavrač, N. (2001). Relational data mining. Berlin: Springer.
14. Garcez, A. S. D., & Zaverucha, G. (1999). The connectionist inductive learning and logic programming system. Applied Intelligence, 11, 59-77.
15. Garcez, A. S. D., & Zaverucha, G. (2012). Multi-instance learning using recurrent neural networks. In Proc. IJCNN (pp. 1-6). New York: IEEE Press.
16. Garcez, A. S. D., Broda, K., & Gabbay, D. M. (2001). Symbolic knowledge extraction from trained neural networks: a sound approach. Artificial Intelligence, 125(1-2), 155-207.
17. Garcez, A. S. D., Broda, K. B., & Gabbay, D. M. (2002). Neural-symbolic learning systems. Berlin: Springer.
18. Garcez, A. S. D., Lamb, L. C., & Gabbay, D. M. (2008). Neural-symbolic cognitive reasoning. Berlin: Springer.
19. Getoor, L., & Taskar, B. (2007). Introduction to statistical relational learning. Cambridge: The MIT Press.
20. Guillame-Bert, M., Broda, K., & Garcez, A. S. D. (2010). First-order logic learning in artificial neural networks. In Proc. IJCNN (pp. 1-8). New York: IEEE Press.
21. Guyon, I., & Elisseeff, A. (2003). An introduction to variable and feature selection. Journal of Machine Learning Research, 3, 1157-1182.
22. Haykin, S. S. (2009). Neural networks and learning machines. Upper Saddle River: Prentice Hall.
23. Jacobs, R. A. (1988). Increased rates of convergence through learning rate adaptation. Neural Networks, 1(4), 295-307.
24. Kijsirikul, B., & Lerdlamnaochai, B. K. (2005). First-order logical neural networks. International Journal of Hybrid Intelligent Systems, 2(4), 253-267.
25. King, R. D., & Srinivasan, A. (1995). Relating chemical activity to structure: an examination of ILP successes. New Generation Computing, 13(3-4), 411-434.
26. King, R. D., Whelan, K. E., Jones, F. M., Reiser, F. G. K., Bryant, C. H., Muggleton, S. H., Kell, D. B., & Oliver, S. G. (2004). Functional genomic hypothesis generation and experimentation by a robot scientist. Nature, 427(6971), 247-252.
27. Koller, D., & Friedman, N. (2009). Probabilistic graphical models: principles and techniques. Cambridge: MIT Press.
28. Kramer, S., Lavrač, N., & Flach, P. (2001). Propositionalization approaches to relational data mining. In S. Džeroski (Ed.), Relational data mining (pp. 262-291). New York: Springer.
29. Krogel, M. A., & Wrobel, S. (2003). Facets of aggregation approaches to propositionalization. In LNAI: Vol. 2835. Proc. ILP (pp. 30-39). Berlin: Springer.
30. Krogel, M. A., Rawles, S., Železný, F., Flach, P., Lavrač, N., & Wrobel, S. (2003). Comparative evaluation of approaches to propositionalization. In LNAI: Vol. 2835. Proc. ILP (pp. 197-214). Berlin: Springer.
31. Kuželka, O., & Železný, F. (2011). Block-wise construction of tree-like relational features with monotone reducibility and redundancy. Machine Learning, 83, 163-192.
32. Landwehr, N., Kersting, K., & De Raedt, L. D. (2007). Integrating naive Bayes and FOIL. Journal of Machine Learning Research, 8, 481-507.
33. Lavrač, N., & Džeroski, S. (1994). Inductive logic programming: techniques and applications. Chichester: Horwood.
34. May, R., Dandy, G., & Maier, H. (2011). Review of input variable selection methods for artificial neural networks. In K. Suzuki (Ed.), Artificial neural networks - methodological advances and biomedical applications (pp. 19-44). New York: InTech. doi: 10.5772/16004.
35. Møller, M. F. (1993). A scaled conjugate gradient algorithm for fast supervised learning. Neural Networks, 6(4), 525-533.
36. Muggleton, S. (1995). Inverse entailment and Progol. New Generation Computing, 13(3-4), 245-286.
37. Muggleton, S., & De Raedt, L. D. (1994). Inductive logic programming: theory and methods. The Journal of Logic Programming, 19/20, 629-679.
38. Muggleton, S., & Tamaddoni-Nezhad, A. (2008). QG/GA: a stochastic search for Progol. Machine Learning, 70, 121-133.
39. Muggleton, S., Paes, A., Costa, V. S., & Zaverucha, G. (2010). Chess revision: acquiring the rules of chess variants through FOL theory revision from examples. In LNAI: Vol. 5989. Proc. ILP (pp. 123-130). Berlin: Springer.
40. Nienhuys-Cheng, S. H., & de Wolf, R. (1997). LNAI: Vol. 1228. Foundations of inductive logic programming. Berlin: Springer.
41. Paes, A., Revoredo, K., Zaverucha, G., & Costa, V. S. (2005). Probabilistic first-order theory revision from examples. In LNAI: Vol. 3625. Proc. ILP (pp. 295-311). Berlin: Springer.
42. Paes, A., Železný, F., Zaverucha, G., Page, D., & Srinivasan, A. (2007). ILP through propositionalization and stochastic k-term DNF learning. In LNAI: Vol. 4455. Proc. ILP (pp. 379-393). Berlin: Springer.
43. Paes, A., Zaverucha, G., & Costa, V. S. (2008). Revising first-order logic theories from examples through stochastic local search. In LNAI: Vol. 4894. Proc. ILP (pp. 200-210). Berlin: Springer.
44. Perlich, C., & Merugu, S. (2005). Gene classification: issues and challenges for relational learning. In Proc. 4th international workshop on multi-relational mining (pp. 61-67). New York: ACM Press.
45. Pitangui, C. G., & Zaverucha, G. (2012). Learning theories using estimation distribution algorithms and (reduced) bottom clauses. In LNAI: Vol. 7207. Proc. ILP (pp. 286-301). Berlin: Springer.
46. Prechelt, L. (1997). Early stopping - but when? In LNAI: Vol. 1524(2). Neural networks: tricks of the trade (pp. 55-69). Berlin: Springer.
47. Quinlan, J. R. (1993). C4.5: programs for machine learning. San Francisco: Morgan Kaufmann.
48. Richardson, M., & Domingos, P. (2006). Markov logic networks. Machine Learning, 62, 107-136.
49. Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986). Learning internal representations by error propagation. In D. E. Rumelhart & J. L. McClelland (Eds.), Parallel distributed processing: explorations in the microstructure of cognition (pp. 318-362). Cambridge: MIT Press.
50. Rumelhart, D. E., Widrow, B., & Lehr, M. A. (1994). The basic ideas in neural networks. Communications of the ACM, 37(3), 87-92.
51. Srinivasan, A. (2007). The Aleph System, version 5. http://www.cs.ox.ac. uk/activities/machlearn/Aleph/aleph.html. Accessed 27 March 2013.
52. Srinivasan, A., & Muggleton, S. H. (1994). Mutagenesis: ILP experiments in a non-determinate biological domain. In LNAI: Vol. 237. Proc. ILP (pp. 217-232). Berlin: Springer.
53. Tamaddoni-Nezhad, A., & Muggleton, S. (2009). The lattice structure and refinement operators for the hypothesis space bounded by a bottom clause. Machine Learning, 76(1), 37-72.
54. Uwents, W., Monfardini, G., Blockeel, H., Gori, M., & Scarselli, F. (2011). Neural networks for relational learning: an experimental comparison. Machine Learning, 82(3), 315-349.
55. Železný, F., & Lavrač, N. (2006). Propositionalization-based relational subgroup discovery with RSD. Machine Learning, 62, 33-63.