From Hopfield nets to recursive networks to graph machines: Numerical machine learning for structured data

Theoretical Computer Science

Volumn 344, Issue 2-3, 2005, Pages 298-334

  Goulon Sigwalt Abram, Aurélie ^a   Duprat, Arthur ^a   Dreyfus, Gérard ^a  

^a CNRS   (France)

Author keywords

Classification; Folding network; Graph; Graph machine; LRAAM; Neural network; QSAR; QSPR; RAAM; Recurrent network; Recursive network; Regression; Structured data; SVM

Indexed keywords

CLASSIFICATION (OF INFORMATION); COMPUTER GRAPHICS; DATA STRUCTURES; MATHEMATICAL MODELS; PROBLEM SOLVING; REGRESSION ANALYSIS; VECTOR QUANTIZATION;

GRAPH MACHINE; LRAAM; QSAR; QSPR; RAAM; STRUCTURED DATA; SVM;

RECURRENT NEURAL NETWORKS;

EID: 27644529846     PISSN: 03043975     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.tcs.2005.08.026     Document Type: Article

References (60)

1. P. Baldi, and G. Pollastri The principled design of large-scale recursive neural network architectures - DAG-RNNs and the protein structure prediction problem J. Mach. Learning Res. 4 2003 576 602
2. A.M. Bianucci, A. Micheli, A. Sperduti, and A. Starita Application of cascade-correlation networks for structures to chemistry Appl. Intelligence 12 2000 117 147
3. E. Bienenstock, and R. Doursat Elastic matching and pattern recognition in neural networks L. Personnaz G. Dreyfus Neural Networks: From Models to Applications 1989 IDSET Paris 472 482
4. E. Bienenstock, and C. von der Malsburg A neural network for invariant pattern recognition Europhys. Lett. 4 1987 121 126
5. C. Bishop Neural Networks for Pattern Recognition 1995 Clarendon Oxford
6. D.S. Blank, L.A. Meeden, and J.B. Marshall Exploring the symbolic/subsymbolic continuum: a case study of RAAM J. Dinsmore The Symbolic and Connectionist Paradigms: Bridging the Gap 1992 LEA Publishers
7. H.H. Chen, Y.C. Lee, T. Maxwell, C. Giles, High-order correlation model for associative memory, in: J.S. Denker (Ed.), Neural Networks for Computing, American Institute of Physics Conference Proceedings, Vol. 151, 1986, pp. 86-99.
8. M. Collins, and N. Duffy Convolution kernels for natural language S.B.T.G. Dietterich Z. Ghahramani Advances in Neural Information Processing System Vol. 14 2002 MIT Press Cambridge, MA 625 632
9. T. Cover Geometrical and statistical properties of systems of linear inequalities with applications in pattern recognition IEEE Trans. Electronic Comput. 14 1965 326 334
10. F. Cuetos, and D.C. Mitchell Cross-linguistic differences in parsing: restrictions on the use of the Late Closure strategy in Spanish Cognition 30 1988 72 105
11. M. Diligenti, M. Gori, M. Maggini, and E. Martinelli Adaptive graphical pattern recognition for the classification of company logos Pattern Recognition 34 2001 2049 2061
12. G. Dreyfus Assessment methods I. Guyon S. Gunn M. Nikravesh L. Zadeh Feature Extraction, Foundations and Applications 2005 Springer Berlin
13. G. Dreyfus, and Y. Idan The canonical form of non-linear discrete-time models Neural Comput. 10 1998 133 164
14. G. Dreyfus, J.M. Martinez, M. Samuelides, M. Gordon, F. Badran, S. Thiria, and L. Hérault Neural Networks, Methodology and Applications 2005 Springer Berlin
15. G. Dreyfus, A. Zippelius, Graph recognition by neural networks, in: L. Personnaz, G. Dreyfus (Eds.), Neural Networks from Models to Applications, IDSET, 1988, pp. 483-492.
16. A. Duprat, T. Huynh, and G. Dreyfus Towards a principled methodology for neural network design and performance evaluation in QSAR; application to the prediction of logP J. Chem. Inform. Comput. Sci. 38 1998 586 594
17. C. Goller, M. Gori, M. Maggini, Feature extraction from data structures with unsupervised recursive neural networks, Internat. Joint Conf. on Neural Networks, IEEE, 1999, pp. 1121-1126.
18. C. Goller, A. Küchler, Learning task-dependent distributed structure-representations by backpropagation through structure, IEEE Internat. Conf. on Neural Networks, 1996, pp. 347-352.
19. M. Gori, M. Maggini, L. Sarti, A recursive neural network model for processing directed acyclic graphs with labeled edges, Internat. Joint Conf. on Neural Networks, 2003, pp. 1351-1355.
20. A. Goulon-Sigwalt-Abram, A. Duprat, G. Dreyfus, Learning numbers from graphs, Appl. Statist. Modeling Data Anal. (2005), available electronically from http://asmda2005.enst-bretagne.fr/IMG/pdf/proceedings/552.pdf.
21. I. Guyon, L. Personnaz, J.P. Nadal, and G. Dreyfus Storage and retrieval of complex sequences in neural networks Phys. Rev. A 38 1988 6365 6372
22. I. Guyon, L. Personnaz, P. Siarry, G. Dreyfus, Engineering applications of spin glass concepts, in: J.L. van Hemmen, I. Morgenstern (Eds.), Heidelberg Colloquium on Glassy Dynamics, Lecture Notes in Physics, Vol. 275, Springer, Berlin, 1987.
23. B. Hammer On the approximation capability of recurrent neural networks Neurocomputing 31 2000 107 123
24. B. Hammer, Learning with recurrent neural networks, Springer Lecture Notes in Control and Information Sciences, Vol. 254, Springer, Berlin, 2000.
25. B. Hammer Recurrent networks for structured data - A unifying approach and its properties Cognitive Systems Res. 3 2002 145 165
26. B. Hammer, M. Alessio, and A. Sperduti Universal approximation capability of cascade correlation for structures Neural Comput. 17 2005 1109 1159
27. D. Haussler, Convolution kernels on discrete structures, Technical Report UCSC-CLR- 99-10, University of Santa Cruz, 1999.
28. J.J. Hopfield Neural networks and physical systems with emergent collective computational abilities Proc. Nat. Acad. Sci. 81 1982 2554 2558
29. J.J. Hopfield, and D. Tank "Neural" computation of decisions in optimization problems Biol. Cybernet. 52 1985 141 152
30. K. Hornik Some new results on neural network approximation Neural Networks 6 1993 1069 1072
31. K. Hornik, M. Stinchcombe, and H. White Multilayer feedforward networks are universal approximators Neural Networks 2 1989 359 366
32. T.S. Jaakkola, and D. Haussler Exploiting generative models in discriminative classifiers S.S.M. Kearns D. Cohn Advances in Neural Information Processing Systems Vol. 11 1998 MIT Press Cambridge, MA 487 493
33. C. Jochum, J. Gasteiger, Canonical numbering and constitutional symmetry, J. Chem. Inform. Comput. Sci.,17 (1977) 113-117.
34. H. Kashima, K. Tsuda, and A. Inokuchi Marginalized kernels between labeled graphs T. Faucett N. Mishra 20th Internat. Conf. on Machine Learning 2003 AAAI Press Merlo Park, CA 321 328
35. R. Kree, and A. Zippelius Recognition of topological features of graphs and images in neural networks J. Phys. A 21 1988 813 818
36. Y. LeCun, B. Boser, J.S. Denker, D. Henderson, R.E. Howard, W. Hubbard, and L.D. Jackel Backpropagation applied to handwritten zip code recognition Neural Comput. 1 1989 541 551
37. C. Leslie, E. Eskin, A. Cohen, J. Weston, and W. Stafford Noble Mismatch string kernels for discriminative protein classification Bioinformatics 20 2004 467 476
38. H. Lodhi, C. Saunders, N. Cristianini, J. Shawe-Taylor, and C. Watkins Text classification using string kernels J. Mach. Learning Res. 2 2002 419 444
39. P. Mahé, N. Ueda, T. Akutsu, J.-L. Perret, J.-P. Vert, Extensions of marginalized graph kernels, 21st Internat. Conf. on Machine learning, ACM Press, New York, 2004, 552-559.
40. S. Marcos, O. Macchi, C. Vignat, G. Dreyfus, L. Personnaz, and P. Roussel-Ragot A unified framework for gradient algorithms used for filter adaptation and neural network training Internat. J. Circuit Theory Appl. 20 1992 1159 1200
41. S. Menchetti, F. Costa, P. Frasconi, M. Pontil, Wide coverage natural language processing using kernel methods and neural networks for structured data, Pattern Recognition Lett. 26 (2005) 1896-1906.
42. A. Micheli, F. Portera, and A. Sperduti A preliminary empirical comparison of recursive neural networks and tree kernel methods on regression tasks for tree structured domains Neurocomputing 64 2005 73 92
43. O. Nerrand, D. Urbani, P. Roussel-Ragot, L. Personnaz, and G. Dreyfus Training recurrent neural networks: why and how? An illustration in process modeling IEEE Trans. Neural Networks 5 1994 178 184
44. Y. Oussar, and G. Dreyfus How to be a gray box: dynamic semi-physical modeling Neural Networks 14 2001 1161 1172
45. L. Personnaz, I. Guyon, and G. Dreyfus Information storage and retrieval in spin glass like neural networks J. Physique Lettres 46 1985 L 359 L 365
46. L. Personnaz, I. Guyon, and G. Dreyfus Collective computational properties of neural networks: new learning mechanisms Phys. Rev. A 34 1986 4217 4228
47. L. Personnaz, I. Guyon, and G. Dreyfus High-order neural networks: information storage without errors Europhys. Lett. 4 1987 863 867
48. J. Pollack Recursive distributed representations Artificial Intelligence 46 1990 77 106
49. W. Press, and S.A. Teukolsky Numerical Recipes, The Art of Scientific Computing 2002 Cambridge University Press Cambridge
50. B. Quenet, S. Sirapian, R. Dubois, G. Dreyfus, and D. Horn Modeling spatiotemporal olfactory data in two steps: from binary to Hodgkin-Huxley neurons Biosystems 67 2002 203 211
51. F. Rosenblatt The Perceptron: a probabilistic model for information storage and organization in the brain Psychol. Rev. 65 1956 386 408
52. C. Saunders, J. Shawe-Taylor, and A. Vinokourov String kernels, Fisher kernels and finite state automata S. Becker S. Thrun K. Obermayer Advances in Neural Information Processing Systems Vol. 15 2003 MIT Press Cambridge, MA 633 640
53. H.T. Siegelmann, and E.D. Sontag On the computational power of neural networks J. Comput. System Sci. 50 1995 132 150
54. E.D. Sontag, Neural networks for control, in: Essays on Control: Perspectives in the Theory and its Applications, Birkhäuser, Basel, 1993, pp. 339-380.
55. A. Sperduti Encoding labeled graphs by labeling RAAM Connection Sci. 6 1994 429 459
56. L.G. Valiant, A theory of the learnable, Commun. ACM (1984) 1134-1142.
57. C. von der Malsburg, and E. Bienenstock A neural network for the retrieval of superimposed patterns Europhys. Lett. 3 1987 1243 1249
58. C. Waibel, T. Hanazawa, G. Hinton, K. Shikano, and K. Lang Phoneme recognition using time-delay neural networks IEEE Trans. Acoust., Speech Signal Process. 37 1989 328 339
59. Y. Yao, and G.L. Marcialis M. Pontil, P. Frasconi, F. Roli, Combining flat and structured representations for fingerprint classification with recursive neural networks and support vector machines Pattern Recognition 36 2003 397 406
60. J. Yao, A. Panaye, J.P. Doucet, R.S. Zhang, H.F. Chen, M.C. Liu, Z.D. Hu, and B.T. Fan Comparative study of QSAR/QSPR correlations using support vector machines, radial basis function neural networks, and multiple linear regression J. Chem. Inform. Comput. Sci. 44 2004 1257 1266