A hybrid algorithm for Bayesian network structure learning with application to multi-label learning

Expert Systems with Applications

Volumn 41, Issue 15, 2014, Pages 6755-6772

  Gasse, Maxime ^a   Aussem, Alex ^a   Elghazel, Haytham ^a  

^a UNIVERSITÉ LYON 1   (France)

Author keywords

Bayesian networks; Feature subset selection; Markov boundary; Multi label learning

Indexed keywords

ALGORITHMS; BENCHMARKING; CLASSIFICATION (OF INFORMATION); EXPERIMENTS; LEARNING SYSTEMS;

BAYESIAN NETWORK STRUCTURE; CLASSIFICATION ACCURACY; EXPERIMENTAL COMPARISON; FEATURE SUBSET SELECTION; MARKOV BOUNDARY; MULTI-LABEL LEARNING; MULTICLASS CLASSIFICATION PROBLEMS; STATE-OF-THE-ART ALGORITHMS;

BAYESIAN NETWORKS;

EID: 84902687696     PISSN: 09574174     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.eswa.2014.04.032     Document Type: Article

References (75)

1. A. Agresti Categorical data analysis 2nd ed. 2002 Wiley
2. C.F. Aliferis, A.R. Statnikov, I. Tsamardinos, S. Mani, and X.D. Koutsoukos Local causal and markov blanket induction for causal discovery and feature selection for classification part i: Algorithms and empirical evaluation Journal of Machine Learning Research, JMLR 11 2010 171 234
3. E. Alvares-Cherman, J. Metz, and M. Monard Incorporating label dependency into the binary relevance framework for multi-label classification Expert Systems With Applications, ESWA 39 2011 1647 1655
4. Armen, A. P.; & Tsamardinos, I. (2011). A unified approach to estimation and control of the false discovery rate in Bayesian network skeleton identification. In European symposium on artificial neural networks, ESANN.
5. A. Aussem, S. Rodrigues de Morais, and M. Corbex Analysis of nasopharyngeal carcinoma risk factors with Bayesian networks Artificial Intelligence in Medicine 54 2012
6. A. Aussem, A. Tchernof, S. Rodrigues de Morais, and S. Rome Analysis of lifestyle and metabolic predictors of visceral obesity with Bayesian networks BMC Bioinformatics 11 2010 487
7. Badea, A. (2004). Determining the direction of causal influence in large probabilistic networks: A constraint-based approach. In Proceedings of the sixteenth european conference on artificial intelligence (pp. 263-267).
8. Bernard, A.; & Hartemink, A. (2005). Informative structure priors: Joint learning of dynamic regulatory networks from multiple types of data. In Proceedings of the pacific symposium on biocomputing (pp. 459-470).
9. H. Blockeel, L.D. Raedt, and J. Ramon Top-down induction of clustering trees J.W. Shavlik, International conference on machine learning, ICML 1998 Morgan Kaufman 55 63
10. H. Borchani, C. Bielza, C. Toro, and P. Larrañaga Predicting human immunodeficiency virus inhibitors using multi-dimensional Bayesian network classifiers Artificial Intelligence in Medicine 57 2013 219 229
11. L. Breiman Random forests Machine Learning 45 2001 5 32 (Pubitemid 32933532)
12. L.E. Brown, and I. Tsamardinos A strategy for making predictions under manipulation Journal of Machine Learning Research, JMLR 3 2008 35 52
13. W. Buntine Theory refinement on Bayesian networks B. D'Ambrosio, P. Smets, P. Bonissone, Uncertainty in artificial intelligence, UAI 1991 Morgan Kaufman San Mateo, CA, USA 52 60
14. G. Cawley Causal and non-causal feature selection for ridge regression JMLR: Workshop and Conference Proceedings 3 2008 107 128
15. J. Cheng, R. Greiner, J. Kelly, D.A. Bell, and W. Liu Learning Bayesian networks from data: An information-theory based approach Artificial Intelligence 137 2002 43 90 (Pubitemid 34405215)
16. D. Chickering, D. Heckerman, and C. Meek Large-sample learning of Bayesian networks is NP-hard Journal of Machine Learning Research, JMLR 5 2004 1287 1330
17. D.M. Chickering Optimal structure identification with greedy search Journal of Machine Learning Research, JMLR 3 2002 507 554
18. Cussens, J.; & Bartlett, M. (2013). Advances in Bayesian network learning using integer programming. In Uncertainty in artificial intelligence (pp. 182-191).
19. K. Dembczyski, W. Waegeman, W. Cheng, and E. Hüllermeier On label dependence and loss minimization in multi-label classification Machine Learning 88 2012 5 45
20. B. Ellis, and W.H. Wong Learning causal Bayesian network structures from experimental data Journal of the American Statistical Association 103 2008 778 789
21. Friedman, N.; Nachman, I.; & Peer, D. (1999). Learning Bayesian network structure from massive datasets: The sparse candidate algorithm. In Proceedings of the fifteenth conference on uncertainty in artificial intelligence (pp. 206-215).
22. N. Friedman, I. Nachman, and D. Peer Learning Bayesian network structure from massive datasets: The sparse candidate algorithm K.B. Laskey, H. Prade, Uncertainty in artificial intelligence, UAI 1999 Morgan Kaufmann Publishers 21 30
23. M. Gasse, A. Aussem, and H. Elghazel Comparison of hybrid algorithms for Bayesian network structure learning European conference on machine learning and principles and practice of knowledge discovery in databases, ECML-PKDD Lecture notes in computer science Vol. 7523 2012 Springer
24. Q. Gu, Z. Li, and J. Han Correlated multi-label feature selection C. Macdonald, I. Ounis, I. Ruthven, CIKM 2011 ACM 1087 1096
25. Y. Guo, and S. Gu Multi-label classification using conditional dependency networks International joint conference on artificial intelligence, IJCAI 2011 AAAI Press 1300 1305
26. D. Heckerman, D. Geiger, and D. Chickering Learning Bayesian networks: The combination of knowledge and statistical data Machine Learning 20 1995 197 243
27. D. Kocev, C. Vens, J. Struyf, and S. Džeroski Ensembles of multi-objective decision trees J.N. Kok, European conference on machine learning, ECML Lecture notes in artificial intelligence Vol. 4701 2007 Springer 624 631
28. M. Koivisto, and K. Sood Exact Bayesian structure discovery in Bayesian networks Journal of Machine Learning Research, JMLR 5 2004 549 573
29. K. Kojima, E. Perrier, S. Imoto, and S. Miyano Optimal search on clustered structural constraint for learning Bayesian network structure Journal of Machine Learning Research, JMLR 11 2010 285 310
30. D. Koller, and N. Friedman Probabilistic graphical models: Principles and techniques 2009 MIT Press
31. Koller, D.; & Sahami, M. (1996). Toward optimal feature selection. In International conference on machine learning, ICML (pp. 284-292).
32. A. Liaw, and M. Wiener Classification and regression by randomforest R News 2 2002 18 22
33. O. Luaces, J. Díez, J. Barranquero, J.J. del Coz, and A. Bahamonde Binary relevance efficacy for multilabel classification Progress in AI 1 2012 303 313
34. G. Madjarov, D. Kocev, D. Gjorgjevikj, and S. Džeroski An extensive experimental comparison of methods for multi-label learning Pattern Recognition 45 2012 3084 3104
35. Maron, O.; & Ratan, A. L. (1998). Multiple-instance learning for natural scene classification. In International conference on machine learning, ICML (Vol. 7, pp. 341-349). Citeseer.
36. McCallum, A. (1999). Multi-label text classification with a mixture model trained by em. In AAAI workshop on text learning.
37. Moore, A.; & Wong, W. (2003). Optimal reinsertion: A new search operator for accelerated and more accurate Bayesian network structure learning. In T. Fawcett, & N. Mishra (Eds.), International conference on machine learning, ICML.
38. R. Nagarajan, M. Scutari, and S. Lbre Bayesian networks in R: With applications in systems biology (use R!) 2013 Springer
39. R.E. Neapolitan Learning Bayesian networks 2004 Pearson Prentice Hall Upper Saddle River, NJ
40. Ott, S.; Imoto, S.; & Miyano, S. (2004). Finding optimal models for small gene networks. In Proceedings of the pacific symposium on biocomputing (pp. 557-567).
41. J. Peña, R. Nilsson, J. Björkegren, and J. Tegnér Towards scalable and data efficient learning of Markov boundaries International Journal of Approximate Reasoning 45 2007 211 232 (Pubitemid 46880195)
42. J.M. Peña, J. Björkegren, and J. Tegnér Growing Bayesian network models of gene networks from seed genes Bioinformatics 40 2005 224 229
43. J. Pearl Probabilistic reasoning in intelligent systems: Networks of plausible inference 1988 Morgan Kaufman San Francisco, CA, USA
44. Peña, J. M. (2008). Learning gaussian graphical models of gene networks with false discovery rate control. In European conference on evolutionary computation, machine learning and data mining in bioinformatics (Vol. 6, pp. 165-176).
45. J.M. Peña Finding consensus Bayesian network structures Journal of Artificial Intelligence Research 42 2012 661 687
46. E. Perrier, S. Imoto, and S. Miyano Finding optimal Bayesian network given a super-structure Journal of Machine Learning Research, JMLR 9 2008 2251 2286
47. D. Peer, A. Regev, G. Elidan, and N. Friedman Inferring subnetworks from perturbed expression profiles Bioinformatics 17 2001 215 224
48. E. Prestat, S. Rodrigues de Morais, J. Vendrell, A. Thollet, C. Gautier, and P. Cohen et al. Learning the local Bayesian network structure around the ZNF217 oncogene in breast tumours Computers in Biology and Medicine 4 2013 334 341
49. R Core Team (2013). R: A language and environment for statistical computing. R Foundation for Statistical Computing Vienna, Austria. .
50. J. Read, B. Pfahringer, G. Holmes, and E. Frank Classifier chains for multi-label classification European conference on machine learning and principles and practice of knowledge discovery in databases, ECML-PKDD Lecture notes in computer science Vol. 5782 2009 Springer 254 269
51. Rodrigues de Morais, S.; & Aussem, A. (2010). An efficient learning algorithm for local Bayesian network structure discovery. In European conference on machine learning and principles and practice of knowledge discovery in databases, ECML-PKDD (pp. 164-169).
52. S. Rodrigues de Morais, and A. Aussem A novel Markov boundary based feature subset selection algorithm Neurocomputing 73 2010 578 584
53. V. Roth, and B. Fischer Improved functional prediction of proteins by learning kernel combinations in multilabel settings BMC Bioinformatics 8 2007 S12
54. G.E. Schwarz Estimating the dimension of a model Journal of Biomedical Informatics 6 1978 461 464
55. M. Scutari Learning Bayesian networks with the bnlearn R package Journal of Statistical Software 35 2010 1 22
56. Scutari, M. (2011). Measures of variability for graphical models (Ph.D. thesis). School in Statistical Sciences, University of Padova.
57. M. Scutari, and A. Brogini Bayesian network structure learning with permutation tests Communications in Statistics Theory and Methods 41 2012 3233 3243
58. M. Scutari, and R. Nagarajan Identifying significant edges in graphical models of molecular networks Artificial Intelligence in Medicine 57 2013 207 217
59. Silander, T.; & Myllymäki, P. (2006). A simple approach for finding the globally optimal Bayesian network structure. In Uncertainty in artificial intelligence, UAI (pp. 445-452).
60. C. Snoek, M. Worring, J.V. Gemert, J. Geusebroek, and A. Smeulders The challenge problem for automated detection of 101 semantic concepts in multimedia ACM international conference on multimedia 2006 ACM Press 421 430 (Pubitemid 47113558)
61. P. Spirtes, C. Glymour, and R. Scheines Causation, prediction, and search 2nd ed. 2000 The MIT Press
62. N. Spolaõr, E.A. Cherman, M.C. Monard, and H.D. Lee A comparison of multi-label feature selection methods using the problem transformation approach Electronic Notes in Theoretical Computer Science 292 2013 135 151
63. M. Studený, and D. Haws Learning Bayesian network structure: Towards the essential graph by integer linear programming tools International Journal of Approximate Reasoning 55 2014 1043 1071
64. Trohidis, K.; Tsoumakas, G.; Kalliris, G.; & Vlahavas, I. (2008). Multi-label classification of music into emotions. In ISMIR (pp. 325-330).
65. Tsamardinos, I.; Aliferis, C.; & Statnikov, A. (2003). Algorithms for large scale Markov blanket discovery. In Florida artificial intelligence research society conference FLAIRS'03 (pp. 376-381).
66. Tsamardinos, I.; & Borboudakis, G. (2010). Permutation testing improves Bayesian network learning. In European conference on machine learning and knowledge discovery in databases, ECML-PKDD (pp. 322-337).
67. I. Tsamardinos, L. Brown, and C. Aliferis The max-min hill-climbing Bayesian network structure learning algorithm Machine Learning 65 2006 31 78 (Pubitemid 44451193)
68. Tsamardinos, I.; & Brown, L. E. (2008). Bounding the false discovery rate in local Bayesian network learning. In AAAI conference on artificial intelligence (pp. 1100-1105).
69. G. Tsoumakas, I. Katakis, and I. Vlahavas Mining multi-label data Transformation 135 2010 1 20
70. G. Tsoumakas, I. Katakis, and I. Vlahavas Random k-labelsets for multi-label classification IEEE Transactions on Knowledge and Data Engineering, TKDE 23 2010 1 12
71. Tsoumakas, G.; & Vlahavas, I. (2007). Random k-labelsets: An ensemble method for multilabel classification. In Proceedings of the 18th european conference on machine learning (Vol. 4701, pp. 406-417).
72. Villanueva, E.; & Maciel, C. (2012). Optimized algorithm for learning Bayesian network superstructures. In International conference on pattern recognition applications and methods, ICPRAM.
73. E. Villanueva, and C.D. Maciel Efficient methods for learning Bayesian network super-structures Neurocomputing 2014 3 12
74. M.L. Zhang, and K. Zhang Multi-label learning by exploiting label dependency European conference on machine learning and principles and practice of knowledge discovery in databases, ECML PKDD Knowledge discovery in databases, KDD Vol. 16 2010 ACM Press 999
75. M.-L. Zhang, and Z.-H. Zhou Multilabel neural networks with applications to functional genomics and text categorization IEEE Transactions on Knowledge and Data Engineering 18 2006 1338 1351 (Pubitemid 44335199)