Challenges for MapReduce in big data

Proceedings - 2014 IEEE 10th World Congress on Services, SERVICES 2014

Volumn , Issue , 2014, Pages 182-189

  Grolinger, Katarina ^a   Hayes, Michael ^a   Higashino, Wilson A ^a,b   L'Heureux, Alexandra ^a   Allison, David S ^a,c,d,e   Capretz, Miriam A M ^a  

^a UNIVERSITY OF WESTERN ONTARIO   (Canada)

^b UNIVERSITY OF CAMPINAS   (Brazil)

^c LAAS CNRS   (France)

^d UNIVERSITÉ DE TOULOUSE   (France)

^e UNIVERSITY OF TOULOUSE CAPITOLE   (France)

Author keywords

Big data; Big data analytics; Interactive analytics; Machine learning; Map reduce; NoSQL; Online processing; Privacy; Security

Indexed keywords

ADVANCED ANALYTICS; BIG DATA; DATA ANALYTICS; DIGITAL STORAGE;

COMPUTING RESOURCE; HIGH SCALABILITIES; ISSUES AND CHALLENGES; MASSIVE DATA SETS; MASSIVELY PARALLELS; ONLINE PROCESSING; RELATIONAL DATABASE; SECURITY AND PRIVACY;

DATA HANDLING;

EID: 85041437840     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/SERVICES.2014.41     Document Type: Conference Paper

References (51)

1. P. Zadrozny and R. Kodali, Big Data Analytics using Splunk, Berkeley, CA, USA: Apress, 2013.
2. F. Ohlhorst, Big Data Analytics: Turning Big Data into Big Money, Hoboken, N.J, USA: Wiley, 2013.
3. J. Dean and S. Ghemawat, "MapReduce: Simplified data processing on large clusters," Commun ACM, 51(1), pp. 107-113, 2008.
4. Apache Hadoop, http://hadoop.apache.org.
5. F. Li, B. C. Ooi, M. T. Özsu and S. Wu, "Distributed data management using MapReduce," ACM Computing Surveys, 46(3), pp. 1-42, 2014.
6. C. Doulkeridis and K. Nørvåg, "A survey of large-scale analytical query processing in MapReduce," The VLDB Journal, pp. 1-26, 2013.
7. S. Sakr, A. Liu and A. Fayoumi, "The family of mapreduce and large-scale data processing systems," ACM Computing Surveys, 46(1), pp. 1-44, 2013.
8. K. Grolinger, W. A. Higashino, A. Tiwari and M. A. Capretz, "Data management in cloud environments: NoSQL and NewSQL data stores," Journal of Cloud Computing: Advances, Systems and Application, 2, 2013.
9. X. Su and G. Swart, "Oracle in-database hadoop: When MapReduce meets RDBMS," Proc. of the 2012 ACM SIGMOD International Conference on Management of Data, 2012.
10. J. Cohen, B. Dolan, M. Dunlap, J. M. Hellerstein and C. Welton, "MAD skills: New analysis practices for Big Data," VLDB Endowment, 2(2), pp. 1481-1492, 2009.
11. Apache Cassandra, http://www.datastax.com/docs.
12. J. C. Anderson, J. Lehnardt and N. Slater, CouchDB: The Definitive Guide, Sebastopol, CA, USA: O'Reilly Media, 2010.
13. A. Thusoo, J. S. Sarma, N. Jain, Z. Shao, P. Chakka, S. Anthony, H. Liu, P. Wyckoff and R. Murthy, "Hive: A warehousing solution over a map-reduce framework," Proc. of the VLDB Endowment, 2(2), pp. 1626-1629, 2009.
14. Apache Mahout, https://mahout.apache.org/.
15. Oracle Big Data connectors, http://www.oracle.com/us/products/database/big-data-connectors/overview/index.html.
16. K. A. Kumar, J. Gluck, A. Deshpande and J. Lin, "Hone: Scaling down hadoop on shared-memory systems," Proc. of the VLDB Endowment, 6(12), pp. 1354-1357, 2013.
17. C. Parker, "Unexpected challenges in large scale machine learning," Proc. of the 1st International Workshop on Big Data, Streams and Heterogeneous Source Mining: Algorithms, Systems, Programming Models and Applications, 2012.
18. J. Lin, "Mapreduce is good enough? if all you have is a hammer, throw away everything that's not a nail!" Big Data, 1(1), pp. 28-37, 2013.
19. G. Malewicz, M. H. Austern, A. J. C. Bik, J. C. Dehnert, I. Horn, N. Leiser and G. Czajkowski, "Pregel: A system for large-scale graph processing," Proc. of the 2010 ACM SIGMOD International Conference on Management of Data, 2010.
20. Apache Giraph, https://giraph.apache.org/.
21. Apache Spark, https://spark.incubator.apache.org/.
22. Y. Bu, B. Howe, M. Balazinska and M. D. Ernst, "HaLoop: Efficient iterative data processing on large clusters," Proc.VLDB Endow., 3(1-2), pp. 285-296, 2010.
23. J. Ekanayake, H. Li, B. Zhang, T. Gunarathne, S. Bae, J. Qiu and G. Fox, "Twister: A runtime for iterative MapReduce," Proc. of the 19th ACM International Symposium on High Performance Distributed Computing, 2010.
24. S. B. Kotsiantis, D. Kanellopoulos and P. Pintelas, "Data preprocessing for supervised learning," International Journal of Computer Science, 1(2), pp. 111, 2006.
25. J. Fan, F. Han and H. Liu, "Challenges of Big Data analysis," National Science Review, in press, 2014.
26. J. Fan and Y. Liao, "Endogeneity in high dimensions," The Annals of Statistics, in press, 2014.
27. B. Dalessandro, "Bring the noise: Embracing randomness is the key to scaling up machine learning algorithms," Big Data, 1(2), pp. 110-112, 2013.
28. G. Ananthanarayanan, S. Kandula, A. Greenberg, I. Stoica, Y. Lu, B. Saha and E. Harris, "Reining in the outliers in map-reduce clusters using Mantri," Proc. of the 9th USENIX Conference on Operating Systems Design and Implementation, 2010.
29. Q. He, Q. Tan, X. Ma and Z. Shi, "The high-activity parallel implementation of data preprocessing based on MapReduce," Proc. of the 5th International Conference on Rough Set and Knowledge Technology, 2010.
30. H. Yang and S. Fong, "Countering the concept-drift problem in Big Data using iOVFDT," IEEE International Congress on Big Data, 2013.
31. T. Hill and P. Lewicki, STATISTICS: Methods and Applications, Tulsa, OK: StatSoft, 2007.
32. J. Heer and S. Kandel, "Interactive analysis of Big Data," XRDS: Crossroads, the ACM Magazine for Students, 19(1), pp. 50-54, 2012.
33. P. Bhatotia, A. Wieder, R. Rodrigues, U. A. Acar and R. Pasquin, "Incoop: MapReduce for incremental computations," Proc. of the 2nd ACM Symposium on Cloud Computing, 2011.
34. Y. Chen, S. Alspaugh and R. Katz, "Interactive analytical processing in Big Data systems: A cross-industry study of MapReduce workloads," Proc. of the VLDB Endowment, 5(12), pp. 1802-1813, 2012.
35. S. Melnik, A. Gubarev, J. J. Long, G. Romer, S. Shivakumar, M. Tolton and T. Vassilakis, "Dremel: Interactive analysis of Web-scale datasets," Proc. of the VLDB Endowment, 3(1-2), pp. 330-339, 2010.
36. A. Hall, O. Bachmann, R. Büssow, S. Gănceanu and M. Nunkesser, "Processing a trillion cells per mouse click," Proc. of the VLDB Endowment, 5(11), pp. 1436-1446, 2012.
37. S. Agarwal, B. Mozafari, A. Panda, H. Milner, S. Madden and I. Stoica, "BlinkDB: Queries with bounded errors and bounded response times on very large data," Proc. of the Eight ACM European Conference on Computer Systems, 2013.
38. H. T. Vo, J. Bronson, B. Summa, J. L. Comba, J. Freire, B. Howe, V. Pascucci and C. T. Silva, "Parallel visualization on large clusters using MapReduce," IEEE Symposium on Large Data Analysis and Visualization, 2011.
39. W. Lam, L. Liu, S. Prasad, A. Rajaraman, Z. Vacheri and A. Doan, "Muppet: MapReduce-style processing of fast data," Proc.VLDB Endow., 5(12), pp. 1814-1825, 2012.
40. S. Ghemawat, H. Gobioff and S. Leung, "The Google file system," ACM SIGOPS Operating Systems Review, 2003.
41. T. Condie, N. Conway, P. Alvaro, J. M. Hellerstein, K. Elmeleegy and R. Sears, "MapReduce online," Proc. of the 7th USENIX Conference on Networked Systems Design and Implementation, 2010.
42. D. Logothetis and K. Yocum, "Ad-hoc data processing in the cloud," Proc. of the VLDB Endowment, 1(2), pp. 1472-1475, 2008.
43. A. Brito, A. Martin, T. Knauth, S. Creutz, D. Becker, S. Weigert and C. Fetzer, "Scalable and low-latency data processing with stream MapReduce," IEEE Third International Conference on Cloud Computing Technology and Science, pp. 48-58, 2011.
44. Storm, distributed and fault-tolerant realtime computation, http://storm-project.net/.
45. L. Neumeyer, B. Robbins, A. Nair and A. Kesari, "S4: Distributed stream computing platform," Proc. of IEEE International Conference on Data Mining Workshops, 2010.
46. M. Zaharia, T. Das, H. Li, S. Shenker and I. Stoica, "Discretized streams: An efficient and fault-tolerant model for stream processing on large clusters," Proc. of the 4th USENIX Conference on Hot Topics in Cloud Computing, 2012.
47. Z. Xiao and Y. Xiao, "Achieving accountable MapReduce in cloud computing," Future Generation Computer Systems, 30,pp. 1-13, 2014.
48. W. Zeng, Y. Yang and B. Luo, "Access control for Big Data using data content," IEEE International Conference on Big Data, 2013.
49. The Personal Information Protection and Electronic Documents Act (PIPEDA), http://www.priv.gc.ca/leg_c/r_o_p_e.asp.
50. Protection of Personal Data, http://ec.europa.eu/justice/dataprotection.
51. I. Roy, S. T. Setty, A. Kilzer, V. Shmatikov and E. Witchel, "Airavat: Security and privacy for MapReduce." Proc. of the 7th Usenix Symposium on Networked Systems Design and Implementation, 2010.