A Parallel Distributed Weka Framework for Big Data Mining Using Spark

Proceedings - 2015 IEEE International Congress on Big Data, BigData Congress 2015

Volumn , Issue , 2015, Pages 9-16

  Koliopoulos, Aris Kyriakos ^a   Yiapanis, Paraskevas ^a   Tekiner, Firat ^a   Nenadic, Goran ^a   Keane, John ^a  

^a UNIVERSITY OF MANCHESTER   (United Kingdom)

Author keywords

Big Data; Data Mining; Distributed Systems; Machine Learning; Spark; Weka

Indexed keywords

ARTIFICIAL INTELLIGENCE; BIG DATA; ELECTRIC SPARKS; LEARNING SYSTEMS; USER INTERFACES;

DISTRIBUTED FRAMEWORK; DISTRIBUTED SYSTEMS; INTUITIVE INTERFACES; ITERATIVE COMPUTATION; KNOWLEDGE EXTRACTION; PROCESSING CAPABILITY; SEQUENTIAL EXECUTION; WEKA;

DATA MINING;

EID: 84953864174     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/BigDataCongress.2015.12     Document Type: Conference Paper

References (29)

1. M. Beyer and D. Laney, "The importance of big data: A definition, " Stamford, CT:Gartner.
2. "Apache Hadoop, " http://hadoop.apache.org/.
3. J. Dean and S. Ghemawat, "MapReduce: Simplified Data Processing on Large Clusters, " Com ACM, pp. 107-113, 2008.
4. "Apache Spark, " https://spark.apache.org/.
5. S. Sakr, A. Liu, and A. G. Fayoumi, "The Family of Mapreduce and Large-scale Data Processing Systems, " ACM Comput. Surv., vol. 46, no. 1, pp. 11:1-11:44, 2013.
6. "SparkR, " http://amplab-extras.github.io/SparkR-pkg/.
7. A. Ghoting, R. Krishnamurthy, E. Pednault, B. Reinwald, V. Sindhwani, S. Tatikonda, Y. Tian, and S. Vaithyanathan, "SystemML: Declarative Machine Learning on MapReduce, " in Interntl Conf. on Data Engineering, 2011, pp. 231-242.
8. M. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann, and I. H. Witten, "The WEKA Data Mining Software: An Update, " SIGKDD Explor. Newsl., pp. 10-18, 2009.
9. R. A. Muenchen, "The Popularity of Data Analysis Software, " http://r4stats.com/articles/popularity//, accessed: 2015-02-02.
10. D. Wegener, M. Mock, D. Adranale, and S. Wrobel, "Toolkit-Based High-Performance Data Mining of Large Data on MapReduce Clusters, " in ICDM, 2009, pp. 296-301.
11. M. Zaharia, M. Chowdhury, T. Das, A. Dave, J. Ma, M. Mc-Cauley, M. J. Franklin, S. Shenker, and I. Stoica, "Resilient Distributed Datasets: A Fault-Tolerant Abstraction for Inmemory Cluster Computing, " in NSDI, 2012.
12. U. M. Fayyad, G. Piatetsky-Shapiro, and P. Smyth, "From Data Mining to Knowledge Discovery: An Overview, " in Advances in KDDM, 1996, pp. 1-34.
13. I. H. Witten and E. Frank, Data Mining: Practical Machine Learning Tools and Techniques, 3rd Edition, 2011.
14. M. Hall, "Weka and Hadoop, " http://markahall.blogspot.co.uk/2013/10/weka-And-hadooppart-1.html/.
15. R. Agrawal and J. C. Shafer, "Parallel Mining of Association Rules, " Knowl. and Data Eng., pp. 962-969, 1996.
16. G. Ananthanarayanan, A. Ghodsi, S. Shenker, and I. Stoica, "Disk-locality in Datacenter Computing Considered Irrelevant, " in Conf. on Hot Topics in Operating Systems, 2011.
17. R. Appuswamy, C. Gkantsidis, D. Narayanan, O. Hodson, and A. Rowstron, "Scale-up vs Scale-out for Hadoop: Time to Rethink?" in Cloud Computing, 2013, pp. 20:1-20:13.
18. C. C. Aggarwal, Ed., Data Classification: Algorithms and Applications. Chapman and Hall/CRC, 2014.
19. "Apache Mahout, " http://mahout.apache.org/.
20. E. R. Sparks, A. Talwalkar, V. Smith, J. Kottalam, X. Pan, J. E. Gonzalez, M. J. Franklin, M. I. Jordan, and T. Kraska, "MLI: An API for distributed machine learning, " ICDM, 2013.
21. Radoop: Analyzing big data with rapidminer and hadoop, 2011.
22. S. Das, Y. Sismanis, K. S. Beyer, R. Gemulla, P. J. Haas, and J. McPherson, "Ricardo: Integrating R and Hadoop, " in Intl Conf. on Management of Data, 2010, pp. 987-998.
23. "RHIPE, " https://www.datadr.org/, accessed: 2015-03-03.
24. H. Lin, S. Yang, and S. Midkiff, "RABID: A distributed parallel R for large datasets, " in Congress on Big Data, 2014, pp. 725-732.
25. "MLib, " https://spark.apache.org/mllib/.
26. M. Prez, A. Snchez, P. Herrero, V. Robles, and J. Pea, "Adapting the Weka Data Mining Toolkit to a Grid Based Environment, " Web Intelligence, pp. 492-497, 2005.
27. S. Celis and D. Musicant, "Weka-parallel: machine learning in parallel, " Carleton College, Tech. Rep., 2002.
28. D. Talia, P. Trunfio, and O. Verta, "Weka4WS: A WSRFEnabled Weka Toolkit for Distributed Data Mining on Grids, " Knowledge Discovery in Databases, pp. 309-320, 2005.
29. A. McCallum, K. Nigam, and L. H. Ungar, "Efficient Clustering of High-dimensional Data Sets with Application to Reference Matching, " in KDD, 2000, pp. 169-178.