A Scalable Data Science Workflow Approach for Big Data Bayesian Network Learning

Proceedings - 2014 International Symposium on Big Data Computing, BDC 2014

Volumn , Issue , 2015, Pages 16-25

  Wang, Jianwu ^a   Tang, Yan ^b   Nguyen, Mai ^a   Altintas, Ilkay ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b HOHAI UNIVERSITY   (China)

Author keywords

Bayesian network; Big Data; Distributed computing; Ensemble learning; Hadoop; Kepler; Scientific workflow

Indexed keywords

ARTIFICIAL INTELLIGENCE; BAYESIAN NETWORKS; COMPUTER AIDED INSTRUCTION; DATA HANDLING; DISTRIBUTED COMPUTER SYSTEMS; LEARNING ALGORITHMS; LEARNING SYSTEMS;

BAYESIAN NETWORK LEARNING; DISTRIBUTED ENVIRONMENTS; ENSEMBLE ALGORITHMS; ENSEMBLE LEARNING; HADOOP; KEPLER; MACHINE LEARNING TECHNIQUES; SCIENTIFIC WORKFLOWS;

BIG DATA;

EID: 84962852768     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/BDC.2014.10     Document Type: Conference Paper

References (29)

1. R. Lu, H. Zhu, X. Liu, J. K. Liu, J. Shao. "Toward efficient and privacypreserving computing in big data era". Network, IEEE, Vol. 28, Issue 4, pp. 46-50, 2014.
2. Y. Zhang, Y. Zhang, E. Swears, N. Larios, Z. Wang, Q. Ji, "Modeling Temporal Interactions with Interval Temporal Bayesian Networks for Complex Activity Recognition", IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 35, Issue 10, pp. 2468-2483, 2013
3. M. Neil, C. Xiaoli, N. Fenton, "Optimizing the Calculation of Conditional Probability Tables in Hybrid Bayesian Networks Using Binary Factorization", IEEE Transactions on Knowledge and Data Engineering, Vol. 24, Issue 7, pp.1306-1312, 2012
4. M. Gui, A. Pahwa, S. Das, "Bayesian Network Model With Monte Carlo Simulations for Analysis of Animal-Related Outages in Overhead Distribution Systems", IEEE Transactions on Power Systems, Vol. 26, Issue 3, pp. 1618-1624, 2011
5. N. E. Fenton, M. Neil, "A critique of software defect prediction models", IEEE Transactions on Software Engineering, Vol. 25, Issue 5, pp. 675-689, 1999.
6. S. Sun, C. Zhang, G. Yu, "A bayesian network approach to traffic flow forecasting", IEEE Transactions on Intelligent Transportation Systems, Vol 7, Issue 1, pp. 124-132, 2006.
7. K. Dejaeger, T. Verbraken, B. Baesens, "Toward Comprehensible Software Fault Prediction Models Using Bayesian Network Classifiers", IEEE Transactions on Software Engineering, Vol. 39, Issue 2, pp. 237-248, 2013.
8. M. Neil and N. Fenton, "Using Bayesian Networks to Model the Operational Risk to Information Technology Infrastructure in Financial Institutions", Journal of Financial Transformation, Vol. 22, pp. 131-138, 2008.
9. Y. Tang, K. Cooper, C. Cangussu, "Bayesian Belief Network Structure Learning Algorithms", Technical Report. University of Texas at Dallas. UTDCS-25-09. 2009.
10. I. Tsamardinos, L. E. Brown, C. F. Aliferis, "The max-min hill-climbing Bayesian network structure learning algorithm", Machine Learning, Vol. 65, Issue 1, pp. 31-78, 2006.
11. J. Cheng, R. Greiner, J. Kelly, D. A, Bell, W. Liu, "Learning Bayesian networks from data: An information-theory based approach", Artificial Intelligence, Vol.137, pp. 43-90, 2002.
12. X. Xie, Z. Geng, "A Recursive Method for Structural Learning of Directed Acyclic Graphs", Journal of Machine Learning Research, Vol.9, pp. 459-483, 2008.
13. R. Chen, K. Sivakumar , H. Kargupta, Learning bayesian network structure from distributed data, In Proceedings of the 3rd SIAM International Data Mining Conference, pp. 284-288, 2003.
14. Y. Na , J. Yang, "Distributed Bayesian network structure learning", In Proceedings of 2010 IEEE International Symposium on Industrial Electronics (ISIE), pp. 1607-1611, 2010.
15. Y. Tamada, S. Imoto, S. Miyano, "Parallel Algorithm for Learning Optimal Bayesian Network Structure", Journal of Machine Learning Research, Vol.12, pp. 2437-2459, 2011.
16. D. M. Chickering, D. Geiger, D. Heckerman, "Learning Bayesian networks is NP-hard". Vol. 196, Technical Report MSR-TR-94-17, Microsoft Research, 1994.
17. D. Opitz, and R. Maclin, "Popular ensemble methods: An empirical study", Journal of Artificial Intelligence Research, Vol. 11, pp. 169-198, 1999.
18. B. Zenko, "A comparison of stacking with meta decision trees to bagging, boosting, and stacking with other methods", In Proceedings IEEE International Conference on Data Mining (ICDM 2001), pp. 669-670, 2001.
19. K. Monteith, J. L. Carroll, K. Seppi, T. Martinez., "Turning Bayesian Model Averaging into Bayesian Model Combination", In Proceedings of the International Joint Conference on Neural Networks (IJCNN'11), pp. 2657-2663, 2011.
20. J. A. Hoeting, D. Madigan, A. E. Raftery, C. T. Volinsky, "Bayesian Model Averaging: A Tutorial". Statistical Science, Vol. 14 , Issue 4 , pp. 382-401, 1999.
21. M. Scutari, Bayesian Network Repository, http://www.bnlearn.com/bnrepository, 2011
22. I. Beinlich, Suermondt, H.R. Chavez, G. Cooper, "The ALARM monitoring system: a case study with two probabilistic inference techniques for belief networks", In Proceedings of Artificial Intelligence in Medical Care, pp. 247-256, 1989.
23. S. Das, Y. Sismanis, K. S. Beyer, R. Gemulla, P. J. Haas, and J. McPherson, "Ricardo: Integrating R and Hadoop, " In Proceedings of ACM SIGMOD International Conference on Management Data (SIGMOD10), pp. 987-998, 2010.
24. A. Ghoting and E. Pednault, "Hadoop-ML: An Infrastructure for the Rapid Implementation of Parallel Reusable Analytics, " In Proceedings of Large-Scale Machine Learning: Parallelism and Massive Data Sets Workshop (NIPS '09), 2009.
25. D. Wegener, M. Mock, D. Adranale, S. Wrobel, "Toolkit-Based High-Performance Data Mining of Large Data on MapReduce Clusters, " In Proceedings of International Conference on Data Mining Workshops (ICDMW '09), pp. 296-301, 2009.
26. C. Chu, S. K. Kim, Y. Lin, Y. Yu, G. R. Bradski, A. Y. Ng, K. Olukotun, "Map-Reduce for machine learning on multicore, " in Advances in neural information processing systems 19, pp. 281-288, 2007.
27. D. Battre, S. Ewen, F. Hueske, O. Kao, V. Markl, D. Warneke, "Nephele/PACTs: A programming model and execution framework for web-scale analytical processing", In Proceedings of the 1st ACM symposium on Cloud computing (SoCC'10), ACM, pp. 119-130, 2010.
28. B. Ludaescher, I. Altintas, C. Berkley, D. Higgins, E. Jaeger-Frank, M. Jones, E. A. Lee, J. Tao, Y. Zhao, "Scientific workflow management and the Kepler system", Concurrency and Computation: Practice & Experience, Special Issue on Scientific Workflows, Vol. 18, Issue 10, pp. 1039-1065, 2006.
29. A. Goderis, C. Brooks, I. Altintas, E. Lee, C. Goble, "Heterogeneous composition of models of computation", Future Generation Computer Systems, Vol. 25, Issue 5, pp. 552-560, 2009.