Learning distributed discrete Bayesian Network Classifiers under MapReduce with Apache Spark

Knowledge-Based Systems

Volumn 117, Issue , 2017, Pages 16-26

  Arias, Jacinto ^a   Gamez, Jose A ^a   Puerta, Jose M ^a  

^a UNIVERSITY OF CASTILLA LA MANCHA   (Spain)

Author keywords

Apache Hadoop; Apache Spark; Bayesian Network Classifiers; Big Data; High dimensionality; MapReduce

Indexed keywords

BAYESIAN NETWORKS; BIG DATA; CLUSTER COMPUTING; CLUSTERING ALGORITHMS; COMPUTER SOFTWARE; LARGE DATASET; SCALABILITY; SUPERVISED LEARNING;

APACHE HADOOP; BAYESIAN NETWORK CLASSIFIERS; DISTRIBUTED PROGRAMMING; EXPERIMENTAL EVALUATION; HIGH DIMENSIONAL DATA; HIGH DIMENSIONALITY; MAP-REDUCE; SUPERVISED CLASSIFICATION;

LEARNING SYSTEMS;

EID: 84977631582     PISSN: 09507051     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.knosys.2016.06.013     Document Type: Article

References (30)

1. [1] Manyika, J., Chui, M., Brown, B., Bughin, J., Big Data: The Next Frontier for Innovation, Competition, and Productivity. Technical Report, 2011 June.
2. [2] McAfee, A., Brynjolfsson, E., Davenport, T.H., Patil, D., Barton, D., Big data, the management revolution. Harvard Bus. Rev. 90:10 (2012), 61–67.
3. [3] Chen, M., Mao, S., Zhang, Y., Leung, V.C., Big Data: Related Technologies, Challenges and Future Prospects. 2014, Springer.
4. [4] Dean, J., Ghemawat, S., MapReduce: simplified data processing on large clusters. Commun. ACM, 2008, 1–13.
5. [5] S. Owen, R. Anil, T. Dunning, E. Friedman, Mahout in Action (2011).
6. [6] White, T., Hadoop: The Definitive Guide. 2012, O'Reilly Media, Inc.
7. [7] Zaharia, M., Chowdhury, M., Franklin, M.J., Shenker, S., Stoica, I., Spark: cluster computing with working sets. HotCloud, 10, 2010.
8. [8] X. Meng, J. Bradley, B. Yavuz, E. Sparks, S. Venkataraman, D. Liu, J. Freeman, D. Tsai, M. Amde, S. Owen, et al., Mllib: Machine Learning in Apache Spark, arXiv:1505.06807(2015).
9. [9] Bielza, C., Larrañaga, P., Discrete Bayesian network classifiers. ACM Comput. Surv. 47:1 (2014), 1–43, 10.1145/2576868.
10. [10] Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.H., The weka data mining software: an update. ACM SIGKDD Explor. Newsl. 11:1 (2009), 10–18.
11. [11] Zhai, Y., Ong, Y.-S., Tsang, I.W., The emerging ”big dimensionality”. Comput. Intell. Mag., IEEE 9:3 (2014), 14–26.
12. [12] Laney, D., 3D data management: controlling data volume, velocity and variety. META Group Res. Note, 6, 2001, 70.
13. [13] Pearl, J., Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. 2014, Elsevier Science.
14. [14] Nielsen, T.D., Jenses, F.V., Bayesian Networks and Decision Graphs. 2009, Springer Science & Business Media.
15. [15] Friedman, N., Geiger, D., Goldszmidt, M., Bayesian network classifiers. Mach. Learn. 29:2-3 (1997), 131–163.
16. [16] Domingos, P., Pazzani, M., On the optimality of the simple Bayesian classifier under zero-one loss. Mach. Learn. 29:2-3 (1997), 103–130.
17. [17] Rish, I., An empirical study of the naive Bayes classifier. IJCAI Workshop on Empirical Methods in Artificial Intelligence, 3, 2001, 41–46.
18. [18] Sahami, M., Learning limited dependence Bayesian classifiers. Second International Conference on Knowledge Discovery and Data Mining, 1996, 91–94.
19. [19] Webb, G.I., Boughton, J.R., Wang, Z., Not so naive Bayes: aggregating one-dependence estimators. Mach. Learn. 58:1 (2005), 5–24.
20. [20] Webb, G.I., Boughton, J.R., Zheng, F., Ting, K.M., Salem, H., Learning by extrapolation from marginal to full-multivariate probability distributions: decreasingly naive Bayesian classification. Mach. Learn. 86:2 (2011), 233–272, 10.1007/s10994-011-5263-6.
21. [21] Rajaraman, A., Ullman, J.D., Ullman, J.D., Ullman, J.D., Mining of Massive Datasets. 1, 2012, Cambridge University Press Cambridge.
22. [22] Shvachko, K., Kuang, H., Radia, S., Chansler, R., The hadoop distributed file system. Mass Storage Systems and Technologies (MSST), 2010 IEEE 26th Symposium on, 2010, IEEE, 1–10.
23. [23] Zaharia, M., Chowdhury, M., Das, T., Dave, A., Ma, J., McCauley, M., Franklin, M.J., Shenker, S., Stoica, I., Resilient distributed datasets: a fault-tolerant abstraction for in-memory cluster computing. Proceedings of the 9th USENIX conference on Networked Systems Design and Implementation, 2012, USENIX Association.
24. [24] Arias, J., Gamez, J.A., Puerta, J.M., Scalable learning of k-dependence Bayesian classifiers under mapreduce. Trustcom/BigDataSE/ISPA, 2015 IEEE, 2, 2015, IEEE, 25–32.
25. [25] Madsen, A.L., Jensen, F., Salmerón, A., Karlsen, M., Langseth, H., Nielsen, T.D., A new method for vertical parallelisation of TAN learning based on balanced incomplete block designs. Lect. Notes in Artif. Intell. 8754 (2014), 302–317.
26. [26] Alonso-Barba, J.I., delaOssa, L., Gámez, J.A., Puerta, J.M., Scaling up the greedy equivalence search algorithm by constraining the search space of equivalence classes. Int. J. Approx. Reas. 54:4 (2013), 429–451.
27. [27] Sonnenburg, S., Franc, V., COFFIN: a computational framework for linear SVMs. Proceedings of the 27th International Conference on Machine Learning (ICML-10), 2010, 999–1006.
28. [28] Bacardit, J., Widera, P., MÃ!‘rquez-Chamorro, A., Divina, F., Aguilar-Ruiz, J.S., Krasnogor, N., Contact map prediction using a large-scale ensemble of rule sets and the fusion of multiple predicted structural features. Bioinformatics 28:19 (2012), 2441–2448, 10.1093/bioinformatics/bts472.
29. [29] Yuan, G.-X., Ho, C.-H., Lin, C.-J., An improved glmnet for l1-regularized logistic regression. J. Mach. Learn. Res. 13:1 (2012), 1999–2030.
30. [30] Bifet, A., Holmes, G., Kirkby, R., Pfahringer, B., MOA: massive online analysis. J. Mach. Learn. Res. 11 (2010), 1601–1604.