The graph neural network model

IEEE Transactions on Neural Networks

Volumn 20, Issue 1, 2009, Pages 61-80

  Scarselli, Franco ^a   Gori, Marco ^a   Tsoi, Ah Chung ^b   Hagenbuchner, Markus ^c   Monfardini, Gabriele ^a  

^a UNIVERSITY OF SIENA   (Italy)

^b HONG KONG BAPTIST UNIVERSITY   (Hong Kong)

^c UNIVERSITY OF WOLLONGONG   (Australia)

Author keywords

Graph neural networks (GNNs); Graph processing; Graphical domains; Recursive neural networks

Indexed keywords

BIOCHEMISTRY; COMPUTER VISION; DATA PROCESSING; IMAGE PROCESSING; INFORMATION MANAGEMENT; LEARNING ALGORITHMS; LEARNING SYSTEMS; MOLECULAR BIOLOGY; PATTERN RECOGNITION; RECURSIVE FUNCTIONS;

COMPUTATIONAL COSTS; EUCLIDEAN SPACES; GENERALIZATION CAPABILITIES; GRAPH NEURAL NETWORKS (GNNS); GRAPH PROCESSING; GRAPHICAL DOMAINS; NETWORK METHODS; NETWORK MODELLING; RECURSIVE NEURAL NETWORKS;

NEURAL NETWORKS;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; AUTOMATED PATTERN RECOGNITION; COMPARATIVE STUDY; FACTUAL DATABASE; INTERNET; NONLINEAR SYSTEM; REGRESSION ANALYSIS; REPRODUCIBILITY; STATISTICAL MODEL;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; DATABASES, FACTUAL; INTERNET; LINEAR MODELS; NEURAL NETWORKS (COMPUTER); NONLINEAR DYNAMICS; PATTERN RECOGNITION, AUTOMATED; REGRESSION ANALYSIS; REPRODUCIBILITY OF RESULTS;

EID: 58649113008     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2008.2005605     Document Type: Article

References (96)

1. P. Baldi and G. Pollastri, "The principled design of large-scale recursive neural network architectures-dag-RNNs and the protein structure prediction problem," J. Mach. Learn. Res., vol. 4, pp. 575-602, 2003.
2. E. Krahmer, S. Erk, and A. Verleg, "Graph-based generation of referring expressions," Comput. Linguist., vol. 29, no. 1, pp. 53-72, 2003.
3. A. Mason and E. Blake, "A graphical representation of the state spaces of hierarchical level-of-detail scene descriptions," IEEE Trans. Vis. Comput. Graphics, vol. 7, no. 1, pp. 70-75, Jan.-Mar. 2001.
4. L. Baresi and R. Heckel, "Tutorial introduction to graph transformation: A software engineering perspective," in Lecture Notes in Computer Science. Berlin, Germany: Springer-Verlag, 2002, vol. 2505, pp. 402-429.
5. C. Collberg, S.Kobourov, J. Nagra, J. Pitts, and K. Wampler, "A system for graph-based visualization of the evolution of software," in Proc. ACM Symp. Software Vis., 2003, pp. 77-86.
6. A. Bua, M. Gori, and F. Santini, "Recursive neural networks applied to discourse representation theory," in Lecture Notes in Computer Science. Berlin, Germany: Springer-Verlag, 2002, vol. 2415.
7. A. Srinivasan, S. Muggleton, R. King, and M. Sternberg, "Mutagenesis: IIp experiments in a non-determinate biological domain," in Proc. 4th Int. Workshop Inductive Logic Programm., 1994, pp. 217-232.
8. M. Bianchini, M. Maggini, L. Sarti, and F. Scarselli, "Recursive neural networks learn to localize faces," Phys. Rev. Lett., vol. 26, no. 12, pp. 1885-1895, Sep. 2005.
9. P. Frasconi, M. Gori, and A. Sperduti, "A general framework for adaptive processing of data structures," IEEE Trans. Neural Netw. vol. 9, no. 5, pp. 768-786, Sep. 1998.
10. S. Brin and L. Page, "The anatomy of a large-scale hypertextual web search engine," in Proc. 7th World Wide Web Conf., Apr. 1998, pp. 107-117.
11. A. Sperduti and A. Starita, "Supervised neural networks for the classification of structures," IEEE Trans. Neural Netw., vol. 8, no. 2, pp. 429-459, Mar. 1997.
12. M. Hagenbuchner, A. Sperduti, and A. C. Tsoi, "A self-organizing map for adaptive processing of structured data," IEEE Trans. Neural Netw., vol. 14, no. 3, pp. 491-505, May 2003.
13. J. Kleinberg, "Authoritative sources in a hyperlinked environment," J. ACM, vol. 46, no. 5, pp. 604-632, 1999.
14. A. C. Tsoi, G. Morini, F. Scarselli, M. Hagenbuchner, and M. Maggini, "Adaptive ranking of web pages," in Proc. 12th World Wide Web Conf. Budapest, Hungary, May 2003, pp. 356-365.
15. M. Bianchini, P. Mazzoni, L. Sarti, and F. Scarselli, "Face spotting in color images using recursive neural networks," in Proc. 1st Int. Workshop Artif. Neural Netw. Pattern Recognit., Florence, Italy, Sep. 2003, pp. 76-81.
16. M. Bianchini, M. Gori, and F. Scarselli, "Processing directed acyclic graphs with recursive neural networks," IEEE Trans. Neural Netw., vol. 12, no. 6, pp. 1464-1470, Nov. 2001.
17. A. Küchler and C. Goller, "Inductive learning in symbolic domains using structure-driven recurrent neural networks," in Lecture Notes in Computer Science, G. Görz and S. Hölldobler, Eds. Berlin, Germany: Springer-Verlag, Sep. 1996, vol. 1137.
18. T. Schmitt and C. Goller, "Relating chemical structure to activity: An application of the neural folding architecture," in Proc. Workshop Fuzzy-Neuro Syst./Conf. Eng. Appl. Neural Netw., 1998, pp. 170-177.
19. M. Hagenbuchner and A. C. Tsoi, "A supervised training algorithm for self-organizing maps for structures," Pattern Recognit. Lett., vol. 26, no. 12, pp. 1874-1884, 2005.
20. M. Gori, M. Maggini, E. Martinelli, and F. Scarselli, "Learning user profiles in NAUTILUS," in Proc. Int. Conf. Adaptive Hypermedia Adaptive Web-Based Syst., Trento, Italy, Aug. 2000, pp. 323-326.
21. B. Hammer and J. Jain, "Neural methods for non-standard data," in Proc. 12th Eur. Symp. Artif. Neural Netw., M. Verleysen, Ed., 2004, pp. 281-292.
22. T. Gärtner, "Kernel-based learning in multi-relational data mining," ACM SIGKDD Explorations, vol. 5, no. 1, pp. 49-58, 2003.
23. T. Gärtner, J. Lloyd, and P. Flach, "Kernels and distances for structured data," Mach. Learn., vol. 57, no. 3, pp. 205-232, 2004.
24. R. Kondor and J. Lafferty, "Diffusion kernels on graphs and other discrete structures," in Proc. 19th Int. Conf. Mach. Learn., C. Sammut and A. e. Hoffmann, Eds., 2002, pp. 315-322.
25. H. Kashima, K. Tsuda, and A. Inokuchi, "Marginalized kernels between labeled graphs," in Proc. 20th Int. Conf. Mach. Learn., T. Fawcett and N. e. Mishra, Eds., 2003, pp. 321-328.
26. P. Mahé, N. Ueda, T. Akutsu, J.-L Perret, and J.-P. Vert, "Extensions of marginalized graph kernels," in Proc. 21st Int. Conf. Mach. Learn., 2004, pp. 552-559.
27. M. Collins and N. Duffy, "Convolution kernels for natural language," in Advances in Neural Information Processing Systems, T. G. Dietterich, S. Becker, and Z. Ghahramani, Eds. Cambridge, MA: MIT Press, 2002, vol. 14, pp. 625-632.
28. J. Suzuki, H. Isozaki, and E. Maeda, "Convolution kernels with feature selection for natural language processing tasks," in Proc. Annu. Meeting Assoc. Comput. Linguistics, 2004, pp. 119-126.
29. J. Cho, H. Garcia-Molina, and L. Page, "Efficient crawling through url ordering," in Proc. 7th World Wide Web Conf., Brisbane, Australia, Apr. 1998, pp. 161-172.
30. A. C. Tsoi, M. Hagenbuchner, and F. Scarselli, "Computing customized page ranks," ACM Trans. Internet Technol., vol. 6, no. 4, pp. 381-414, Nov. 2006.
31. H. Chang, D. Cohn, and A. K. McCallum, "Learning to create customized authority lists," in Proc. 17th Int. Conf. Mach. Learn., 2000, pp. 127-134.
32. J. Lafferty, A. McCallum, and F. Pereira, "Conditional random fields: Probabilistic models for segmenting and labeling sequence data," in Proc. 18th Int. Conf. Mach. Learn., 2001, pp. 282-289.
33. L. Chua and L. Yang, "Cellular neural networks: Theory," IEEE Trans. Circuits Syst., vol. CAS-35, no. 10, pp. 1257-1272, Oct. 1988.
34. L. Chua and L. Yang, "Cellular neural networks: Applications," IEEE Trans. Circuits Syst., vol. CAS-35, no. 10, pp. 1273-1290, Oct. 1988.
35. P. Kaluzny, "Counting stable equilibria of cellular neural networks-A graph theoretic approach," in Proc. Int. Workshop Cellular Neural Netw. Appl., 1992, pp. 112-116.
36. M. Ogorzatek, C. Merkwirth, and J. Wichard, "Pattern recognition using finite-iteration cellular systems," in Proc. 9th Int. Workshop Cellular Neural Netw. Appl., 2005, pp. 57-60.
37. J. Hopfield, "Neural networks and physical systems with emergent collective computational abilities," Proc. Nat. Acad. Sci., vol. 79, pp. 2554-2558, 1982.
38. F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, and G. Monfardini, "Computation capabilities of graph neural networks," IEEE Trans. Neural Netw., vol. 20, no. 1, Jan. 2009, to be published.
39. F. Scarselli, A. C. Tsoi,M. Gori, M. Hagenbuchner, and G. Monfardini, "Graphical-based learning environments for pattern recognition" in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics),Volume 3138, 2004, Pages 42-56
40. M. J. D. Powell, "An efficient method for finding the minimum of a function of several variables without calculating derivatives," Comput. J., vol. 7, pp. 155-162, 1964.
41. A. C. Tsoi, "Adaptive processing of sequences and data structures," in Lecture Notes in Computer Science, C. L. Giles and M. Gori, Eds. Berlin, Germany: Springer-Verlag, 1998, vol. 1387, pp. 27-62.
42. L. Almeida, "A learning rule for asynchronous perceptrons with feedback in a combinatorial environment," in Proc. IEEE Int. Conf. Neural Netw., M. Caudill and C. Butler, Eds., San Diego, 1987, vol. 2, pp. 609-618.
43. F. Pineda, "Generalization of back-propagation to recurrent neural networks," Phys. Rev. Lett., vol. 59, pp. 2229-2232, 1987.
44. M. Riedmiller and H. Braun, "A direct adaptive method for faster back-propagation learning: The rprop algorithm," in Proc. IEEE Int. Conf. Neural Netw., San Francisco, CA, 1993, pp. 586-591.
45. R. Reed, "Pruning algorithms - A survey," IEEE Trans. Neural Netw. vol. 4, no. 5, pp. 740-747, Sep. 1993.
46. S. Fahlman and C. Lebiere, "The cascade-correlation learning architecture," in Advances in Neural Information Processing Systems D. Touretzky, Ed. Denver, San Mateo: Morgan Kaufmann, 1989, vol. 2, pp. 524-532.
47. T.-Y. Kwok and D.-Y. Yeung, "Constructive algorithms for structure learning in feedforward neural networks for regression problems," IEEE Trans. Neural Netw., vol. 8, no. 3, pp. 630-645, May 1997.
48. G.-B. Huang, L. Chen, and C. K. Siew, "Universal approximation using incremental constructive feedforward networks with random hidden nodes," IEEE Trans. Neural Netw., vol. 17, no. 1, pp. 879-892, Jan. 2006.
49. F. Scarselli and A. C. Tsoi, "Universal approximation using feedforward neural networks: A survey of some existing methods, and some new results," Neural Netw., vol. 11, no. 1, pp. 15-37, 1998.
50. A. M. Bianucci, A. Micheli, A. Sperduti, and A. Starita, "Analysis of the internal representations developed by neural networks for structures applied to quantitative structure-activity relationship studies of benzodiazepines," J. Chem. Inf. Comput. Sci., vol. 41, no. 1, pp. 202-218, 2001.
51. M. Hagenbuchner, A. C. Tsoi, and A. Sperduti, "A supervised self-organising map for structured data," in Advances in Self-Organising Maps, N. Allinson, H. Yin, L. Allinson, and J. Slack, Eds. Berlin, Germany: Springer-Verlag, 2001, pp. 21-28.
52. E. Seneta, Non-Negative Matrices and Markov Chains. New York: Springer-Verlag, 1981, ch. 4, pp. 112-158.
53. D. E. Rumelhart and J. McClelland, Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Cambridge, MA: PDP Research Group, MIT Press, 1986, vol. 1.
54. H. Bunke, "Graph matching: Theoretical foundations, algorithms, and applications," in Proc. Vis. Interface, Montreal, QC, Canada, 2000, pp. 82-88.
55. D. Conte, P. Foggia, C. Sansone, and M. Vento, "Graph matching applications in pattern recognition and image processing," in Proc. Int. Conf. Image Process., Sep. 2003, vol. 2, pp. 21-24.
56. D. Conte, P. Foggia, C. Sansone, and M. Vento, "Thirty years of graph matching in pattern recognition," Int. J. Pattern Recognit. Artif. Intell., vol. 18, no. 3, pp. 265-268, 2004.
57. A. Agarwal, S. Chakrabarti, and S. Aggarwal, "Learning to rank networked entities," in Proc. 12th ACM SIGKDD Int. Conf. Knowl. Disc. Data Mining, New York, 2006, pp. 14-23.
58. V. Di Massa, G. Monfardini, L. Sarti, F. Scarselli, M. Maggini, and M. Gori, "A comparison between recursive neural networks and graph neural networks," in Proc. Int. Joint Conf. Neural Netw., Jul. 2006, pp. 778-785.
59. G. Monfardini, V. Di Massa, F. Scarselli, and M. Gori, "Graph neural networks for object localization," in Proc. 17th Eur. Conf. Artif. Intell., Aug. 2006, pp. 665-670.
60. F. Scarselli, S. Yong, M. Gori, M. Hagenbuchner, A. C. Tsoi, and M. Maggini, "Graph neural networks for ranking web pages," in Proc. IEEE/WIC/ACM Conf. Web Intelligence, 2005, pp. 666-672.
61. W. Uwents, G. Monfardini, H. Blockeel, F. Scarselli, and M. Gori, "Two connectionist models for graph processing: An experimental comparison on relational data," in Proc. Eur. Conf. Mach. Learn., 2006, pp. 213-220.
62. S.Yong, M. Hagenbuchner, F. Scarselli, A. C. Tsoi, and M. Gori, "Document mining using graph neural networks," in Proc. 5th Int. Workshop Initiative Evaluat. XML Retrieval, N. Fuhr, M. Lalmas, and A. Trotman, Eds., 2007, pp. 458-472.
63. A. K. Debnath, R. Lopex de Compandre, G. Debnath, A. Schusterman, and C. Hansch, "Structure-activity relationship of mutagenic aromatic and heteroaromatic nitro compounds. correlation with molecular orbital energies and hydrophobicity," J. Med. Chem., vol. 34, no. 2, pp. 786-797, 1991.
64. S. Kramer and L. De Raedt, "Feature construction with version spaces for biochemical applications," in Proc. 18th Int. Conf. Mach. Learn. 2001, pp. 258-265.
65. J. Quinlan and R. Cameron-Jones, "FOIL: A midterm report," in Proc. Eur. Conf. Mach. Learn., 1993, pp. 3-20.
66. J. Quinlan, "Boosting first-order learning," in Lecture Notes in Computer Science. Berlin, Germany: Springer-Verlag, 1996, vol. 1160, p. 143.
67. M. Krogel, S. Rawles, F. Zelezny, P. Flach, N. Lavrac, and S. Wrobel, "Comparative evaluation of approaches to propositionalization," in Proc. 13th Int. Conf. Inductive Logic Programm., 2003, pp. 197-214.
68. S. Muggleton, "Machine learning for systems biology," in Proc. 15th Int. Conf. Inductive Logic Programm., Bonn, Germany, Aug. 10 - 13, 2005, pp. 416-423.
69. A. Woźnica, A. Kalousis, and M. Hilario, "Matching based kernels for labeled graphs," in Proc. Int. Workshop Mining Learn. Graphs/ECML/PKDD, T. Gärtner, G. Garriga, and T. Meinl, Eds., 2006, pp. 97-108.
70. L. De Raedt and H. Blockeel, "Using logical decision trees for clustering," in Lecture Notes in Artificial Intelligence. Berlin, Germany: Springer-Verlag, 1997, vol. 1297, pp. 133-141.
71. M. Diligenti, M. Gori, and M. Maggini, "Web page scoring systems for horizontal and vertical search," in Proc. 11th World Wide Web Conf. 2002, pp. 508-516.
72. T. H. Haveliwala, "Topic sensitive pagerank," in Proc. 11th World Wide Web Conf., 2002, pp. 517-526.
73. G. Jeh and J. Widom, "Scaling personalized web search," in Proc. 12th World Wide Web Conf., May 20-24, 2003, pp. 271-279.
74. F. Scarselli, A. C. Tsoi, and M. Hagenbuchner, "Computing personalized pageranks," in Proc. 12th World Wide Web Conf., 2004, pp. 282-283.