Hierarchical attribute reduction algorithms for big data using MapReduce

Knowledge-Based Systems

Volumn 73, Issue , 2015, Pages 18-31

  Qian, Jin ^a   Lv, Ping ^a   Yue, Xiaodong ^b   Liu, Caihui ^c   Jing, Zhengjun ^a  

^a JIANGSU UNIVERSITY OF TECHNOLOGY   (China)

^b SHANGHAI UNIVERSITY   (China)

^c GANNAN NORMAL UNIVERSITY   (China)

Author keywords

Big data; Data and task parallelism; Granular computing; Hierarchical attribute reduction; MapReduce

Indexed keywords

ALGORITHMS; COMPUTATION THEORY; DATA REDUCTION; DECISION TABLES; EQUIVALENCE CLASSES; GRANULAR COMPUTING; ROUGH SET THEORY;

ATTRIBUTE REDUCTION; ATTRIBUTE REDUCTION ALGORITHM; ATTRIBUTE SIGNIFICANCE; HIERARCHICAL ATTRIBUTES; HIERARCHICAL DECISIONS; MAP-REDUCE; PARALLEL COMPUTATION; TASK PARALLELISM;

BIG DATA;

EID: 84915822707     PISSN: 09507051     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.knosys.2014.09.001     Document Type: Article

References (48)

1. V. Abhishek, L. Xavier, E. David, and H. Roy Scaling genetic algorithms using MapReduce Proceedings of the 2009 Ninth International Conference on Intelligent Systems Design and Applications 2009 IEEE 13 18
2. A. Bargiela, and W. Pedrycz Toward a theory of granular computing for human centered information processing IEEE Trans. Fuzzy Syst. 16 2 2008 320 330
3. C.T. Chu, S. Kim, Y.A. Lin, Y.Y. Yu, G. Bradskl, A.Y. Ng, K. Olukotun, MapReduce for machine learning on multicore, In: Proceedings of the 20th Conference on Advances in Neural Information Processing Systems (NIPS2006), vol. 6, 2006, pp. 281-288.
4. J.H. Dai, W.T. Wang, H.W. Tian, and L. Liu Attribute selection based on a new conditional entropy for incomplete decision systems Knowl.-Based Syst. 39 2013 207 213
5. J. Dean, and S. Ghemawat MapReduce: simplified data processing on large clusters Commun. ACM 51 1 2008 107 114
6. D.Y. Deng, D.X. Yan, and J.Y. Wang Parallel reducts based on attribute significance J. Yu, S. Greco, P. Lingras, Rough Set and Knowledge Technology Lecture Notes in Computer Science vol. 6401 2010 Springer Berlin/Heidelberg 336 343
7. I. Duntsch, and G. Gediga Simple data filtering in rough set systems Int. J. Approximate Reasoning 18 1 1998 93 106
8. Q.R. Feng, D.Q. Miao, and Y. Cheng Hierarchical decision rules mining Expert Syst. Appl. 37 3 2010 2081 2091
9. A. Frank, A. Asuncion, UCI Machine Learning Repository, University of California, School of Information and Computer Science, Irvine, CA, 2010. < http://archive.ics.uci.edu/ml >.
10. S. Ghemawat, H. Gobioff, and S.T. Leung The Google file system SIGOPS - Operating Syst. Rev. 37 5 2003 29 43
11. I. Guyan, and A. Elisseeff An introduction to variable and feature selection J. Mach. Learn. Res. 3 2003 1157 1182
12. J. Han, Y. Cai, and N. Cercone Data-driven discovery of quantitative rules in relational databases IEEE Trans. Knowl. Data Eng. 5 1 1993 29 40
13. J. Han, and Y. Fu Mining multiple-level association rules in large database IEEE Trans. Knowl. Data Eng. 11 5 1999 798 805
14. L.X. Han, C.S. Liew, J.V. Hemert, and M. Atkinson A generic parallel processing model for facilitating data mining and integration Parallel Comput. 37 2011 157 171
15. T.P. Hong, C.E. Lin, J.H. Lin, and S.L. Wang Learning cross-level certain and possible rules by rough sets Expert Syst. Appl. 34 3 2008 1698 1706
16. X.H. Hu, and N. Cercone Discovering maximal generalized decision rules through horizontal and vertical data reduction Comput. Intell. 17 4 2001 685 702
17. Q.H. Hu, W. Pedrycz, D.R. Yu, and J. Lang Selecting discrete and continuous features based on neighborhood decision error minimization IEEE Trans. Syst. Man Cybern.-Part B: Cybern. 40 1 2010 137 150
18. J.Y. Liang, F. Wang, C.Y. Dang, and Y.H. Qian An efficient rough feature selection algorithm with a multi-granulation view Int. J. Approximate Reasoning 53 6 2012 912 926
19. J.H. Li, C.L. Mei, and Y.J. Lv A heuristic knowledge-reduction method for decision formal contexts Comput. Math. Appl. 61 2011 1096 1106
20. Y.J. Lu Concept hierarchy in data mining: specification generation and implementation Master Degree Dissertation 1997 Simon Fraser University Cnanda
21. D.Q. Miao, G.Y. Wang, and Q. Liu Granular Computing Past Nowday and Future 2007 Science Publisher Beijing
22. D.Q. Miao, Y. Zhao, Y.Y. Yao, F.F. Xu, and H.X. Li Relative reducts in consistent and inconsistent decision tables of the Pawlak rough set model Inf. Sci. 179 24 2009 4140 4150
23. Z. Pawlak Rough sets Int. J. Comput. Inform. Sci. 11 5 1982 341 356
24. Z. Pawlak Rough Sets: Theoretical Aspects of Reasoning About Data 1991 Kluwer Academic Publishers Boston
25. Y.H. Qian, J.Y. Liang, W. Pedrycz, and C.Y. Dang Positive approximation: an accelerator for attribute reduction in rough set theory Artif. Intell. 174 9 2010 597 618
26. J. Qian, D.Q. Miao, Z.H. Zhang, and W. Li Hybrid approaches to attribute reduction based on indiscernibility and discernibility relation Int. J. Approximate Reasoning 52 2 2011 212 230
27. J. Qian, D.Q. Miao, and Z.H. Zhang Knowledge reduction algorithms in cloud computing Chin. J. Comput. 34 12 2011 2332 2343 (in Chinese)
28. A. Skowron, and C. Rauszer The discernibility matrices and functions in information systems R. Slowiński, Intelligent Decision Support Handbook of Applications and Advances of the Rough Sets Theory 1992 Kluwer Dordrecht
29. A. Srinivasan, T.A. Faruquie, and Sachindra Joshi Data and task parallelism in ILP using MapReduce Mach. Learn. 86 1 2012 141 168
30. R. Susmaga Tree-like parallelization of reduct and construct computation S. Tsumoto, RSCTC 2004 LNAI vol. 3066 2004 Springer Berlin Heidelberg 455 464
31. G.Y. Wang, H. Yu, and D.C. Yang Decision table reduction based on conditional information entropy Chin. J. Comput. 25 7 2002 760 766 (in Chinese)
32. L.H. Wang, and G.F. Wu Attribute reduction based on parallel symbiotic evolution Chin. J. Comput. 26 5 2003 630 635 (in Chinese)
33. X.Z. Wang, T.T. Wang, and J.H. Zhai An attribute reduction algorithm based on instance selection J. Comput. Res. Dev. 49 11 2012 2305 2310 (in Chinese)
34. W.Z. Wu, and Y. Leung Theory and applications of granular labelled partitions in multi-scale decision tables Inf. Sci. 181 18 2011 3878 3897
35. X. Xu, J. Jager, and H.P. Kriegel A fast parallel clustering algorithm for large spatial databases Data Min. Knowl. Disc. 3 1999 263 290
36. 2 | U / C |)) Chin. J. Comput. 29 3 2006 611 615 (in Chinese)
37. Y. Yang, Z. Chen, Z. Liang, and G. Wang Attribute reduction for massive data based on rough set theory and MapReduce J. Yu, S. Greco, P. Lingras, Rough Set and Knowldge Technology Lecture Notes in Computer Science vol. 6401 2010 Springer Berlin/Heidelberg 672 678
38. X.B. Yang, M. Zhang, H.L. Dou, and J.Y. Yang Neighborhood systems-based rough sets in incomplete information system Knowl.-Based Syst. 24 6 2011 858 867
39. Y.Y. Yao Stratified rough sets and granular computing R.N. Dave, T. Sudkamp, Proceedings of the 18th International Conference of the North American Fuzzy Information Processing Society 1999 IEEE Press New York, USA 800 804
40. Y.Y. Yao, and Y. Zhao Discernibility matrix simplification for constructing attribute reducts Inf. Sci. 7 2009 867 882
41. M.Q. Ye, X.D. Wu, X.G. Hu, and D.H. Hu Knowledge reduction for decision tables with attribute value taxonomies Knowl.-Based Syst. 56 2014 68 78
42. J.P. Yuan, and D.H. Zhu A hierarchical reduction algorithm for concept hierarchy Proceedings of the Sixth International Conference on Intelligent Systems Design and Applications (ISDA'06) vol. 1 2006 IEEE Computer Society 724 729
43. L.A. Zadeh Fuzzy sets and information granularity M. Gupta, R. Ragade, R. Yager, Advances in Fuzzy Set Theory and Applications 1979 North-Holland Publishing Amsterdam 3 18
44. J.B. Zhang, T.R. Li, and D. Ruan A parallel method for computing rough set approximations Inf. Sci. 194 2012 209 223
45. W.Z. Zhao, H.F. Ma, and Q. He Parallel K-Means clustering based on MapReduce M.G. Jaatun, G. Zhao, C. Rong, Cloud Computing (CloudCom2009) 2009 Springer Berlin Heidelberg 674 679
46. X. Zhang, C.L. Mei, D.G. Chen, and J.H. Li Multi-confidence rule acquisition oriented attribute reduction of covering decision systems via combinatorial optimization Knowl.-Based Syst. 50 2013 187 197
47. W. Ziarko Acquisition of hierarchy-structured probabilistic decision tables and rules from data Expert Syst. 20 5 2003 305 310
48. D. Zinn, S. Bowers, S. Köhler, and B. Ludäscher Parallelizing XML data-streaming workflows via MapReduce J. Comput. Syst. Sci. 76 6 2010 447 463