FiDoop: Parallel Mining of Frequent Itemsets Using MapReduce

IEEE Transactions on Systems, Man, and Cybernetics: Systems

Volumn 46, Issue 3, 2016, Pages 313-325

  Xun, Yaling ^a   Zhang, Jifu ^a   Qin, Xiao ^b  

^a TAIYUAN UNIVERSITY OF SCIENCE AND TECHNOLOGY   (China)

^b AUBURN UNIVERSITY   (United States)

Author keywords

frequent items ultrametric tree (FIU tree); Frequent itemsets; Hadoop cluster; load balance; MapReduce

Indexed keywords

CLUSTER COMPUTING; CLUSTERING ALGORITHMS; DATA MINING; DIGITAL STORAGE; FAULT TOLERANCE;

HADOOP CLUSTER; ITEM SETS; LOAD BALANCE; MAP-REDUCE; ULTRAMETRIC TREES;

FORESTRY;

EID: 84962799392     PISSN: 21682216     EISSN: 21682232     Source Type: Journal    
DOI: 10.1109/TSMC.2015.2437327     Document Type: Article

References (38)

1. M. J. Zaki, "Parallel and distributed association mining: A survey, " IEEE Concurrency, vol. 7, no. 4, pp. 14-25, Oct./Dec. 1999.
2. I. Pramudiono and M. Kitsuregawa, "FP-tax: Tree structure based generalized association rule mining, " in Proc. 9th ACM SIGMOD Workshop Res. Issues Data Min. Knowl. Disc., Paris, France, 2004, pp. 60-63.
3. R. Agrawal, T. Imieliński, and A. Swami, "Mining association rules between sets of items in large databases, " ACM SIGMOD Rec., vol. 22, no. 2, pp. 207-216, 1993.
4. J. Han, J. Pei, Y. Yin, and R. Mao, "Mining frequent patterns without candidate generation: A frequent-pattern tree approach, " Data Min. Knowl. Disc., vol. 8, no. 1, pp. 53-87, 2004.
5. S. Kotsiantis and D. Kanellopoulos, "Association rules mining: A recent overview, " GESTS Int. Trans. Comput. Sci. Eng., vol. 32, no. 1, pp. 71-82, 2006.
6. R. Agrawal and J. C. Shafer, "Parallel mining of association rules, " IEEE Trans. Knowl. Data Eng., vol. 8, no. 6, pp. 962-969, Dec. 1996.
7. A. Schuster and R. Wolff, "Communication-efficient distributed mining of association rules, " Data Min. Knowl. Disc., vol. 8, no. 2, pp. 171-196, 2004.
8. M.-Y. Lin, P.-Y. Lee, and S.-C. Hsueh, "Apriori-based frequent itemset mining algorithms on MapReduce, " in Proc. 6th Int. Conf. Ubiquit. Inf. Manage. Commun. (ICUIMC), Danang, Vietnam, 2012, pp. 76:1-76:8. [Online]. Available: http://doi.acm.org/10.1145/2184751.2184842
9. D. Chen et al., "Tree partition based parallel frequent pattern mining on shared memory systems, " in Proc. 20th IEEE Int. Parallel Distrib. Process. Symp. (IPDPS), Rhodes Island, Greece, 2006, pp. 1-8.
10. L. Liu, E. Li, Y. Zhang, and Z. Tang, "Optimization of frequent itemset mining on multiple-core processor, " in Proc. 33rd Int. Conf. Very Large Data Bases, Vienna, Austria, 2007, pp. 1275-1285.
11. A. Javed and A. Khokhar, "Frequent pattern mining on message passing multiprocessor systems, " Distrib. Parallel Databases, vol. 16, no. 3, pp. 321-334, 2004.
12. J. Neerbek, "Message-driven FP-growth, " in Proc. WICSA/ECSA Compan. Vol., Helsinki, Finland, 2012, pp. 29-36.
13. Y.-J. Tsay, T.-J. Hsu, and J.-R. Yu, "FIUT: A new method for mining frequent itemsets, " Inf. Sci., vol. 179, no. 11, pp. 1724-1737, 2009.
14. J. Dean and S. Ghemawat, "MapReduce: Simplified data processing on large clusters, " Commun. ACM, vol. 51, no. 1, pp. 107-113, Jan. 2008.
15. J. Dean and S. Ghemawat, "MapReduce: A flexible data processing tool, " Commun. ACM, vol. 53, no. 1, pp. 72-77, Jan. 2010.
16. W. Lu, Y. Shen, S. Chen, and B. C. Ooi, "Efficient processing of k nearest neighbor joins using MapReduce, " Proc. VLDB Endow., vol. 5, no. 10, pp. 1016-1027, 2012.
17. D. W. Cheung, S. D. Lee, and Y. Xiao, "Effect of data skewness and workload balance in parallel data mining, " IEEE Trans. Knowl. Data Eng., vol. 14, no. 3, pp. 498-514, May/Jun. 2002.
18. H. Li, Y. Wang, D. Zhang, M. Zhang, and E. Y. Chang, "PFP: Parallel FP-growth for query recommendation, " in Proc. ACM Conf. Recommend. Syst., Lausanne, Switzerland, 2008, pp. 107-114.
19. L. Cristofor. (2001). Artool Project[J]. [Online]. Available: http://www.cs.umb.edu/laur/ARtool/, accessed Oct. 19, 2012.
20. J. S. Park, M.-S. Chen, and P. S. Yu, "Using a hash-based method with transaction trimming for mining association rules, " IEEE Trans. Knowl. Data Eng., vol. 9, no. 5, pp. 813-825, Sep./Oct. 1997.
21. J. D. Holt and S. M. Chung, "Mining association rules using inverted hashing and pruning, " Inf. Process. Lett., vol. 83, no. 4, pp. 211-220, 2002.
22. F. Berzal, J.-C. Cubero, N. Marín, and J.-M. Serrano, "TBAR: An efficient method for association rule mining in relational databases, " Data Knowl. Eng., vol. 37, no. 1, pp. 47-64, 2001.
23. J. Zhang, X. Zhao, S. Zhang, S. Yin, and X. Qin, "Interrelation analysis of celestial spectra data using constrained frequent pattern trees, " Knowl.-Based Syst., vol. 41, pp. 77-88, Mar. 2013.
24. D. W. Cheung and Y. Xiao, "Effect of data skewness in parallel mining of association rules, " in Research and Development in Knowledge Discovery and Data Mining. Berlin, Germany: Springer, 1998, pp. 48-60.
25. T. Shintani and M. Kitsuregawa, "Hash based parallel algorithms for mining association rules, " in Proc. 4th Int. Conf. Parallel Distrib. Inf. Syst., Miami Beach, FL, USA, 1996, pp. 19-30.
26. E.-H. Han, G. Karypis, and V. Kumar, "Scalable parallel data mining for association rules, " IEEE Trans. Knowl. Data Eng., vol. 12, no. 3, pp. 337-352, May/Jun. 2000.
27. I. Pramudiono and M. Kitsuregawa, "Parallel FP-growth on PC cluster, " in Advances in Knowledge Discovery and Data Mining. Berlin, Germany: Springer, 2003, pp. 467-473.
28. P. Tang and M. P. Turkia, "Parallelizing frequent itemset mining with FP-trees, " in Proc. 21st Int. Conf. Comput. Appl., Seattle, WA, USA, 2006, pp. 30-35.
29. K. Yu and J. Zhou, "Parallel TID-based frequent pattern mining algorithm on a PC cluster and grid computing system, " Expert Syst. Appl., vol. 37, no. 3, pp. 2486-2494, 2010.
30. S. Cong, J. Han, J. Hoeflinger, and D. Padua, "A sampling-based framework for parallel data mining, " in Proc. 10th ACM SIGPLAN Symp. Prin. Pract. Parallel Program., Chicago, IL, USA, 2005, pp. 255-265.
31. K.-M. Yu, J. Zhou, T.-P. Hong, and J.-L. Zhou, "A load-balanced distributed parallel mining algorithm, " Expert Syst. Appl., vol. 37, no. 3, pp. 2459-2464, 2010.
32. L. Zhou et al., "Balanced parallel FP-growth with MapReduce, " in Proc. IEEE Youth Conf. Inf. Comput. Telecommun. (YC-ICT), Beijing, China, 2010, pp. 243-246.
33. K. W. Lin, P.-L. Chen, and W.-L. Chang, "A novel frequent pattern mining algorithm for very large databases in cloud computing environments, " in Proc. IEEE Int. Conf. Granular Comput. (GrC), Kaohsiung, Taiwan, 2011, pp. 399-403.
34. L. Yang, Z. Shi, L. D. Xu, F. Liang, and I. Kirsh, "DH-TRIE frequent pattern mining on Hadoop using JPA, " in Proc. IEEE Int. Conf. Granular Comput. (GrC), Kaohsiung, Taiwan, 2011, pp. 875-878.
35. M. Riondato, J. A. DeBrabant, R. Fonseca, and E. Upfal, "PARMA: A parallel randomized algorithm for approximate association rules mining in MapReduce, " in Proc. 21st ACM Int. Conf. Inf. Knowl. Manage., Maui, HI, USA, 2012, pp. 85-94.
36. S. Hong, Z. Huaxuan, C. Shiping, and H. Chunyan, "The study of improved FP-growth algorithm in MapReduce, " in Proc. 1st Int. Workshop Cloud Comput. Inf. Security, Shanghai, China, 2013, pp. 250-253.
37. J. Choi, C. Choi, K. Yim, J. Kim, and P. Kim, "Intelligent reconfigurable method of cloud computing resources for multimedia data delivery, " Informatica, vol. 24, no. 3, pp. 381-394, 2013.
38. J. Xie and X. Qin, "The 19th heterogeneity in computing workshop (HCW 2010), " in Proc. IEEE Int. Symp. Parallel Distrib. Process. Workshops Ph.D. Forum (IPDPSW), Atlanta, GA, USA, Apr. 2010, pp. 1-5.