An introduction to recursive neural networks and Kernel methods for cheminformatics

Current Pharmaceutical Design

Volumn 13, Issue 14, 2007, Pages 1469-1495

  Micheli, Alessio ^a   Sperduti, Alessandro ^b   Starita, Antonina ^a  

^a UNIVERSITY OF PISA   (Italy)

^b UNIVERSITY OF PADOVA   (Italy)

Author keywords

Kernel for structures; Kernel methods; Learning in structured domains; Machine learning; QSAR QSPR; Recursive neural networks

Indexed keywords

4 AMINOBUTYRIC ACID RECEPTOR; ALIPHATIC HYDROCARBON; ALKENE; BENZODIAZEPINE; BENZODIAZEPINE DERIVATIVE; CHEMICAL COMPOUND; HYDROXYLAMINE;

ARTIFICIAL NEURAL NETWORK; BACK PROPAGATION; BOOTSTRAPPING; CHEMICAL STRUCTURE; COMPUTER ANALYSIS; INFORMATION PROCESSING; KERNEL METHOD; LEARNING ALGORITHM; MACHINE LEARNING; MOLECULAR MODEL; PREDICTION; PRINCIPAL COMPONENT ANALYSIS; PRIORITY JOURNAL; QUANTITATIVE STRUCTURE ACTIVITY RELATION; QUANTITATIVE STRUCTURE PROPERTY RELATION; REGRESSION ANALYSIS; REVIEW; STRUCTURAL BIOINFORMATICS; SUPPORT VECTOR MACHINE;

CHEMISTRY, PHARMACEUTICAL; MATHEMATICS; NEURAL NETWORKS (COMPUTER); NUMERICAL ANALYSIS, COMPUTER-ASSISTED; QUANTITATIVE STRUCTURE-ACTIVITY RELATIONSHIP;

EID: 34249859316     PISSN: 13816128     EISSN: None     Source Type: Journal    
DOI: 10.2174/138161207780765981     Document Type: Review

References (87)

1. Pearl J. Probabilistic reasoning in intelligent systems: Networks of plausible inference. Morgan Kaufmann 1988.
2. Lauritzen SL. Graphical models. Oxford University Press, 1998.
3. Baldi P, Brunak S. Bioinformatics: The machine learning approach. MIT Press 1998.
4. Jordan MI Ed. Learning in graphical models. MIT Press 1999.
5. Muggleton S. Inductive logic programming, Volume 38 of A P I C series. Academic Press Ltd., London 1992.
6. Lavrač N, Džeroski S. Inductive logic programming: Techniques and applications. Ellis Horwood 1994.
7. Džeroski S, Lavrač N. Relational data mining. Springer-Verlag, Berlin, September 2001.
8. Sperduti A, Starita A. Supervised neural networks for the classification of structures. IEEE Trans Neural Netw 1997; 8(3): 714-35.
9. Frasconi P, Gori M, Sperduti A. A general framework for adaptive processing of data structures. IEEE Trans Neural Netw 1998; 9(5): 768-86.
10. Frasconi P, Gori M, Kuchler A, Sperduti A. From sequences to data structures: Theory and applications. In: Kolen JF, Kremer SC Eds, A Field Guide to Dynamical Recurrent Networks, Chapter 19, Wiley IEEE Press 2001.
11. Vapnik VN. Statistical learning theory. Wiley, New York 1998.
12. Cristianini N, Shawe-Taylor J. Kernel methods for pattern analysis. Cambridge University Press 2004.
13. Scholkopf B, Tsuda K, Vert J-P. Kernel methods in computational biology. MIT Press 2004.
14. Hammer B, Saunders C, Sperduti A Eds. Special issue on neural networks and kernel methods for structured domains. Neural Netw 2005; 18(8): 1015-123.
15. Haykin S. Neural networks, A comprehensive foundation. Prentice Hall, 2nd edition 1999.
16. Bartelett PL. The sample complexity of pattern classification with neural networks: the size of weights is more important than the size of the network. IEEE Trans Inform Theory 1998; 44(2): 525-36.
17. Burden FR, Ford MG, Whitley DC, Winkler DA. Use of automatic relevance determination in QSAR studies using bayesian neural networks. J Chem Inf and Comp Sci 2000; 40: 1423-30.
18. Fahlman SE, Lebiere C. The cascade-correlation Learning architecture. In: Touretzky DS Ed, Advances in neural information processing systems 2, Morgan Kaufmann, San Mateo, CA, 1990; 524-32.
19. King RD, Srinivasan A, Sternberg MJE. Relating chemical activity to structure: an examination of ILP successes. New Generation Comput 1995; 13: 411-33.
20. Srinivasan A, Muggleton S, King RD, Sternberg MJE. Theories for mutagenicity: a study of first-order and feature based induction. Artificial Intelligence 1996; 85(1,2): 277-99.
21. Rumelhart DE, Hinton GE, Williams RJ. Learning internal representations by error propagation. In: Rumelhart DE, McClelland JL, the PDP Research Group, Eds, Parallel distributed processing: Explorations in the microstructure of cognition Vol 1: Foundations. MIT Press, 1986.
22. Cristianini N, Shawe-Taylor J. An introduction to support vector machines and other kernel-based learning methods. Cambridge University Press, Cambridge 2000.
23. Bishop CM. Neural networks for pattern recognition. Clarendon Press, Oxford, Oxford 1995.
24. Mitchell T. Machine learning. McGraw Hill, New York 1997.
25. Duda RO, Hart PE, Stork DG. Pattern classification. John Wiley & Sons, Inc., New York, second edition, November 2001.
26. Bianucci AM, Micheli A, Sperduti A, Starita A. Application of cascade correlation networks for structures to chemistry. Applied Intelligence (Kluwer Academic Publishers) 2000; 12: 117-46.
27. IUPAC. Nomenclature of organic chemistry. Pergamon Press, Oxford 1979.
28. Weininger D. Smiles, a chemical language and information system. 1. Introduction to methodology and encoding rules. J Chem Info Comput Sci 1988; 28: 31-6.
29. Bernazzani L, Duce C, Micheli A, Mollica V, Sperduti A, Starita A, et al. Predicting thermodynamic properties from molecular structures by recursive neural networks. Comparison with classical group contributions methods. Technical Report TR-04-16, Università di Pisa, Dipartimento di Informatica, Pisa, October 2004.
30. Bernazzani L, Duce C, Mollica V, Tiné MR, Micheli A, Sperduti A, et al. Recursive neural networks prediction of thermodynamic properties from molecular structures. Application to mono- and poly-functional linear molecules. In 13th International Congress on Thermal Analysis and Calorimetry ICTAC 2004, September 2004; 96.
31. Duce C. Physical chemical methods in the rational design of new materials: QSAR and calorimetric approaches. PhD thesis, Dept. of Chemistry and Industrial Chemistry, University of Pisa, Italy 2005.
32. Duce C, Micheli A, Solaro R, Starita A, Tiné MR. Prediction of chemical-physical properties by neural networks for structures. XVII Convegno Italiano di Scienza e Tecnologia delle Macromolecole - AIM, 11-15 September 2005, Napoli, Italy. Macromolecular Symposia Journal, Wiley-VCH, Weinheim, Germany 2006; 234(1): 13-9.
33. Micheli A. Recursive processing of structured domains in machine learning. Ph.d. thesis: TD-13/03, Department of Computer Science, University of Pisa, Pisa, Italy, December 2003.
34. Micheli A, Sperduti A, Starita A, Bianucci AM. Analysis of the internal representations developed by neural networks for structures applied to quantitative structure-activity relationship studies of benzodiazepines. J Chem Inf Comput Sci 2001; 41(1): 202-18.
35. Micheli A, Sperduti A, Starita A, Bianucci AM. Design of new biologically active molecules by recursive neural networks. In IJCNN'2001 - Proceedings of the INNS-IEEE International Joint Conference on Neural Networks, Washington, DC, July 2001; 2732-7.
36. Cherqaoui D, Villemin D. Use of neural network to determine the boiling point of alkanes. J Chem Soc Faraday Trans 1994; 90(1): 97-102.
37. Hadjipavlou-Litina D, Hansch C. Quantitative structure-activity relationships of the benzodiazepines. a review and reevaluation. Chem Rev 1994; 94(6): 1483-505.
38. Hammer B. Learning with recurrent neural networks, Volume 254 of Springer Lecture Notes in Control and Information Sciences. Springer-Verlag 2000.
39. Chrisman L. Learning recursive distributed representations for holistic computation. Connection Science 1991; 3(4): 345-66.
40. Tsochantaridis I, Hofmann T, Joachims T, Altun Y. Support vector machine learning for interdependent and structured output spaces. In: Brodley CE Ed, ICML '04: Twenty-first international conference on machine learning, New York, NY, USA 2004. ACM Press.
41. Sperduti A, Majidi D, Starita A. Extended cascade-correlation for syntactic and structural pattern recognition. In: Perner P, Wang P, Rosenfeld A Eds, Advances in structural and syntactical pattern recognition, Lecture notes in Computer Science, Springer-Verlag 1996; 1121: 90-9.
42. Fahlman SE. The recurrent cascade-correlation architecture. In: Lippmann RP Moody JE, Touretzky DS Eds, Advances in neural information processing systems 3, Morgan Kaufmann Publishers, San Mateo, CA 1991; 190-96.
43. Gärtner T. A survey of kernels for structured data. SIGKDD Explor Newsl 2003; 5(1): 49-58.
44. Collins M, Duffy N. Convolution kernels for natural language. In: Dietterich TG, Becker S, Ghahramani Z Eds, Advances in neural information processing systems 14, MIT Press, Cambridge, MA 2002; 625-32.
45. Gärtner T. Exponential and geometric kernels for labelled graphs. In Workshop on Unreal Data: Principles of Modeling Nonvectorial Data, NIPS 2002, 2002.
46. Gärtner T, Flach PA, Wrobel S. On graph kernels: Hardness results and efficient alternatives. In Proc of the 16th Annual Conf on Computational Learning Theory and 7th Kernel Workshop, Lecture notes in Computer Science, Springer-Verlag, 2003; 2777: 129-43.
47. Haussler D. Convolution kernels on discrete structures. Technical Report UCS-CRL-99-10, UC Santa Cruz 1999.
48. Horváth T, Gärtner T, Wrobel S. Cyclic pattern kernels for predictive graph mining. In Proc of the 2004 ACM SIGKDD Int Conf on Knowledge Discovery and Data Mining, 2004; 158-67.
49. Horváth T. Cyclic pattern kernels revisited. In Advances in Knowledge Discovery and Data Mining, 9th Pacific-Asia Conference, PAKDD 2005, 2005; 791-801.
50. Kuang R, Ie E, Wang K, Wang K, Siddiqi M, Freund Y, et al. Profile-based string kernels for remote homology detection and motif extraction. In Proc of the IEEE Computational Systems Bioninformatics 2004, 2004; 152-60.
51. Leslie C, Eskin E, Weston J, Noble W. Mismatch string kernels for SVM protein classification. In Adv in Neural Information Processing Systems, MIT Press 2003; Vol. 15; 1417-24.
52. Leslie C, Eskin E, Noble WS. The spectrum kernel: A string kernel for svm protein classification. In Proc of the Pacific Symposium on Biocomputing 2002; 564-75.
53. Lodhi H, Saunders C, Shawe-Taylor J, Cristianini N, Watkins C. Text classification using String Kernels. J Mach Learn Res 2000; 2: 419-44.
54. Menchetti SS, Costa F, Frasconi P. Weighted decomposition kernels. In: Wrobel S, De Raedt L Eds, Proceedings of the 22-nd International Conference on Machine Learning, Bonn, Germany. ACM Press 2005; 585-92.
55. Saigo H, Vert J-P, Ueda N, Akutsu T. Protein homology detection using string alignment kernels. Bioinformatics 2004; 20(11): 1682-9.
56. Swamidass SJ, Chen J, Phung P, Bruand J, Ralaivola L, Baldi P. Kernels for small molecules and the prediction of mutagenicity, toxicity and anti-cancer activity. Bioinformatics 2005; 21(Suppl 1): 359-68.
57. Viswanathan SVN, Smola AJ. Fast kernels for string and tree matching. In: Thrun S, Becker S, Obermayer K Eds, Advances in Neural Information Processing Systems 15, MIT Press, Cambridge, MA 2003; 569-76.
58. Watkins C. Kernels from matching operations. Technical report, Dept. of Computer Science, Royal Holloway, Univ. of London 1999.
59. Cuturi M, Vert J-P. The context-tree kernel for strings. Neural Netw 2005; 18(8): 1111-23.
60. Jaakkola T, Diekhans M, Haussler D. A discriminative framework for detecting remote protein homologies. J Comput Biol 2000; 7(1,2): 95-114.
61. Jaakkola TS, Diekhans M, Haussler D. Using the fisher kernel method to detect remote protein homologies. In Proceedings of the Seventh International Conference on Intelligent Systems for Molecular Biology, AAAI Press 1999; 149-58.
62. Jaakkola TS, Haussler D. Exploiting generative models in discriminative classifiers. In Proc of Tenth Conference on Advances in Neural Information Processing Systems 1999; 487-93.
63. Mahé P, Ueda N, Akutsu T, Perret J-L, Vert J-P. Extension of marginalized graph kernels. In Proc of the 21st Int Conf on Machine Learning, New York, NY, USA 2004; 70.
64. Nicotra L, Micheli A, Starita A. Fisher kernel for tree structured data. In Proceedings of the IEEE International Joint Conference on Neural Networks (IJCNN'2004), IEEE Press 2004; 1917-22.
65. Baldi P, Brunak S, Frasconi P, Pollastri G, Soda G. Exploiting the past and the future in protein secondary structure prediction. Bioinformatics 1999; 15(11): 937-46.
66. Micheli A, Sona D, Sperduti A. Contextual processing of structured data by recursive cascade correlation. IEEE Trans Neural Netw 2004; 15(6): 1396-410.
67. Baldi P, Pollastri G. The principled design of large-scale recursive neural network architectures-DAG-RNNs and the protein structure prediction problem. J Mach Learn Res 2003; 4: 575-602.
68. Micheli A, Sona D, Sperduti A. Bi-causal recurrent cascade correlation. In IJCNN'2000 - Proceedings of the IEEE-INNS-ENNS International Joint Conf on Neural Networks, 2000; 3: 3-8.
69. Micheli A, Sona D, Sperduti A. Recursive cascade correlation for contextual processing of structured data. In Proc of the Int Joint Conf on Neural Networks - WCCI-IJCNN'2002, 2002; 1: 268-73.
70. Hammer B, Micheli A, Sperduti A. Universal approximation capability of cascade correlation for structures. Neural Comput 2005; 17(5): 1109-59.
71. Williams RJ, Zipser D. A learning algorithm for continually running fully recurrent neural networks. Neural Comput 1989; 1: 270-80.
72. Fahlman SE. Fast-Learning variations on back-propagation: An empirical study. In Proceedings of the 1988 Connectionist Models Summer School, 1988; 38-51.
73. Riedmiller M, Braun H. A direct adaptive method for faster backpropagation learning: The rprop algorithm. In Proceedings of the IEEE International Conference on Neural Networks 1993; 586-91.
74. Bianchini M, Gori M, Scarselli F. Recursive processing of cyclic graphs. In Proc of WCCI-IJCNN'2002, 2002; 1: 154-9.
75. Gori M, Monfardini G, Scarselli F. Learning in graphic domains using neural networks. In: Avesani M, Gori P Eds, Proceedings of the Workshop on "Sub-symbolic paradigms for learning in structured domains", 16th European Conference on Machine Learning (ECML) the 9th European Conference on Principles and Practice of Knowledge Discovery in Databases (PKDD), October 2005; 31-45.
76. Micheli A, Sestito AS. Dealing with graphs by neural networks. In: Avesani P, Gori M Eds, Proceedings of the Workshop on "Subsymbolic paradigms for learning in structured domains", 16th European Conference on Machine Learning (ECML) the 9th European Conference on Principles and Practice of Knowledge Discovery in Databases (PKDD), October 2005; 66-75.
77. Micheli A, Sestito AS. A new neural network model for contextual processing of graphs. Proceedings of WIRN 2005, June. Lecture notes in Computer Science, Springer-Verlag 2006; 3931: 10-7.
78. Pollastri G, Baldi P, Vullo A, Frasconi P. Prediction of protein topologies using GIOHMMs GRNNs. In: Becker S, Thrun S, Obermayer K Eds, Advances in Neural Information Processing Systems 15. MIT Press, 2003.
79. Vullo A, Frasconi P. Disulfide connectivity prediction using recursive neural networks and evolutionary information. Bioinformatics 2004; 20(5): 653-9.
80. Ceroni A, Frasconi P, Pollastri G. Learning protein secondary structure from sequential and relational data. Neural Netw 2005; 18(8): 1029-39.
81. Ralaivola L, Swamidass SJ, Saigo H, Baldi P. Graph kernels for chemical informatics. Neural Netw 2005; 18(8): 1093-110.
82. Mahé P, Ueda N, Akutsu T, Perret JL, Vert JP. Extensions of marginalized graph kernels. In Proceedings of the 21 st International Conference on Machine Learning, Banff, Canada 2004; 552-9.
83. Bianucci AM, Micheli A, Sperduti A, Starita A. Quantitative structure-activity relationships of benzodiazepines by recursive cascade correlation. In IEEE, editor, Proceedings of IJCNN '98 - IEEE World Congress on Computational Intelligence, Anchorage, Alaska, May 1998; 117-22.
84. Micheli A, Portera F, Sperduti A. A preliminary empirical comparison of recursive neural networks and tree kernel methods on regression tasks for tree structured domains. Neurocomputing 2005; 64: 73-92.
85. Debnath AK. Application of 3D-QSAR techniques in anti-HIV-1 drug design - an overview. Curr Pharm Des 2005; 11(24): 3091-110.
86. Pozzan A. Molecular descriptors and methods for ligand based virtual high throughput screening in drug discovery. Curr Pharm Des 2006; 12(17): 2099-110.
87. Steinbeck C, Hoppe C, Kuhn S, Floris M, Guha R, Willighagen EL. Recent developments of the chemistry development kit (CDK) - an open-source java library for chemo- and bioinformatics. Curr Pharm Des 2006; 12(17): 2111-20.