Big data: Survey, technologies, opportunities, and challenges

Scientific World Journal

Volumn 2014, Issue , 2014, Pages

  Khan, Nawsher ^a,b   Yaqoob, Ibrar ^a   Hashem, Ibrahim Abaker Targio ^a   Inayat, Zakira ^a,c   Mahmoud Ali, Waleed Kamaleldin ^a   Alam, Muhammad ^d,e   Shiraz, Muhammad ^a   Gani, Abdullah ^a  

^a UNIVERSITY OF MALAYA   (Malaysia)

^b ABDUL WALI KHAN UNIVERSITY   (Pakistan)

^c UNIVERSITY OF ENGINEERING AND TECHNOLOGY   (Pakistan)

^d SAUDI ELECTRONIC UNIVERSITY   (Saudi Arabia)

^e UNIVERSITI KUALA LUMPUR   (Malaysia)

Author keywords

[No Author keywords available]

Indexed keywords

ADVERTIZING; ALGORITHM; BIG DATA; CLUSTER ANALYSIS; COMPUTER; COMPUTER PROGRAM; CONCEPTUAL FRAMEWORK; CORRELATION ANALYSIS; DATA ANALYSIS; DATA BASE; DATA MINING; ENVIRONMENT; GROWTH RATE; HADOOP; HARD DISK DRIVE; HEALTH SURVEY; HIVE STRUCTURE; HUMAN; INDUSTRY; INFORMATION PROCESSING; INFORMATION SYSTEM; INFORMATION TECHNOLOGY; LIFE CYCLE; MACHINE LEARNING; MAHOUT; NOMENCLATURE; OPTICAL DISK; PIG FRAMEWORK; PUBLISHING; REVIEW; STATISTICAL ANALYSIS; ACCESS TO INFORMATION; INFORMATION DISSEMINATION; INFORMATION RETRIEVAL; INTERNET;

ACCESS TO INFORMATION; AUTOMATIC DATA PROCESSING; INFORMATION DISSEMINATION; INFORMATION STORAGE AND RETRIEVAL; INTERNET;

EID: 84928207325     PISSN: 23566140     EISSN: 1537744X     Source Type: Journal    
DOI: 10.1155/2014/712826     Document Type: Review

References (111)

1. Worldometers, Real time world statistics, " 2014, http://www .worldometers.info/world-population/.
2. D. Che, M. Safran, and Z. Peng, "From Big Data to Big Data Mining: challenges, issues, and opportunities, " in Database Systems for Advanced Applications, pp. 1-15, Springer, Berlin, Germany, 2013.
3. M. Chen, S. Mao, and Y. Liu, "Big data: a survey, " Mobile Networks and Applications, vol. 19, no. 2, pp. 171-209, 2014.
4. S. Kaisler, F. Armour, J. A. Espinosa, and W. Money, "Big data: issues and challenges moving forward, " in Proceedings of the IEEE 46th Annual Hawaii International Conference on System Sciences (HICSS 13), pp. 995-1004, January 2013.
5. R. Cumbley and P. Church, "Is "Big Data" creepy?" Computer Law and Security Review, vol. 29, no. 5, pp. 601-609, 2013.
6. S. Hendrickson, Getting Started with Hadoop with Amazons Elastic MapReduce, EMR, 2010.
7. M. Hilbert and P. López, "The worlds technological capacity to store, communicate, and compute information, " Science, vol. 332, no. 6025, pp. 60-65, 2011.
8. J. Manyika, C. Michael, B. Brown et al. , "Big data: The next frontier for innovation, competition, and productivity, " Tech. Rep. , Mc Kinsey, May 2011.
9. J. M. Wing, "Computational thinking and thinking about computing, " Philosophical Transactions of the Royal Society of London A:Mathematical, Physical and Engineering Sciences, vol. 366, no. 1881, pp. 3717-3725, 2008.
10. J. Mervis, "Agencies rally to tackle big data, " Science, vol. 336, no. 6077, p. 22, 2012.
11. K. Douglas, "Infographic: big data brings marketing big numbers, " 2012, http://www.marketingtechblog.com/ibm-big-datamarketing/.
12. S. Sagiroglu and D. Sinanc, "Big data: a review, " in Proceedings of the International Conference on Collaboration Technologies and Systems (CTS 13), pp. 42-47, IEEE, San Diego, Calif, USA, May 2013.
13. Intel, Big Data Analaytics, " 2012, http://www.intel.com/content/dam/www/public/us/en/documents/reports/data-insightspeer-research-report.pdf
14. A. Holzinger, C. Stocker, B. Ofner, G. Prohaska, A. Brabenetz, and R. Hofmann-Wellenhof, "Combining HCI, natural language processing, and knowledge discovery-potential of IBM content analytics as an assistive technology in the biomedical field, " in Human-Computer Interaction and Knowledge Discovery in Complex, Unstructured, Big Data, vol. 7947 of Lecture Notes in Computer Science, pp. 13-24, Springer, Berlin, Germany, 2013.
15. YouTube, YouTube statistics, 2014, http://www.youtube.com/yt/press/statistics.html
16. Facebook, Facebook Statistics, 2014, http://www.statisticbrain.com/facebook-statistics/.
17. Twitter, Twitter statistics, " 2014, http://www.statisticbrain.com/twitter-statistics/
18. Foursquare, Foursquare statistics, 2014, https://foursquare.com/about
19. Jeff Bullas, "SocialMedia Facts and Statistics You Should Know in 2014, " 2014, http://www.jeffbullas.com/2014/01/17/20-socialmedia-facts-and-statistics-you-should-know-in-2014/.
20. Marcia, Data on Big Data, 2012, http://marciaconner.com/blog/data-on-big-data/
21. IDC, Analyze the futere, " 2014 http://www.idc.com/
22. K. Goda and M. Kitsuregawa, "The history of storage systems, " Proceedings of the IEEE, vol. 100, no. 13, pp. 1433-1440, 2012.
23. Coughlin Associates, "The 2012-2016 capital equipment and technology report for the hard disk drive industry, " 2013, http://www.tomcoughlin.com/Techpapers/2012%20Capital%20Equipment%20Report%20Brochure%20021112.pdf.
24. Freescale Semiconductor Inc. , "Integrated portable system processor, " 1995, http://pdf.datasheetcatalog.com/datasheets2/19/1997441.pdf.
25. Byte.com, "Most important chips, " 1995, http://web.archive.org/web/20080401091547/http:/http://www.byte.com/art/9509/sec7/art9.htm.
26. Motorola, Integrated portable system processor, " 2001, http://ic.laogu.com/datasheet/31/MC68EZ328 MOTOROLA105738.pdf
27. Motorola, Integrated Portable System Processor, 2001, http://www.freescale.com/files/32bit/doc/prod brief/MC68VZ328P.pdf.
28. A. Mandelli and V. Bossi, "World Internet Project, " 2002, http://www.worldinternetproject.net/files/Published/oldis/wip2002-rel-15-luglio.pdf.
29. Helsingin Sanomat, "Continued growth in mobile phone sales, " 2000, http://www2.hs.fi/english/archive/.
30. J. K. Belk, "Insight into the Future of 3G Devices and Services, " 2007, http://www.cdg.org/news/events/webcast/070228webcast/Qualcomm.pdf.
31. eTForecasts, Worldwide PDA & smartphone forecast, Executive summary, 2006, http://www.etforecasts.com/products/ESpdas2003.htm.
32. R. Zackon, Ground Breaking Study of Video Viewing Finds Younger Boomers Consume more Video Media than Any Other Group, 2009, http://www.research-excellence.com/news/032609 vcm.php.
33. NielsenWire, Media consumption and multi-tasking continue to increase across TV, Internet and Mobile, 2009, http://blog.nielsen.com/nielsenwire/mediaentertainment/three-screen-report-mediaconsumption-and-multitasking-continue-to-increase
34. Freescale Semiconductors, "Integrated Cold Fire version 2 microprocessor, " 2005, http://www.freescale.com/webapp/sps/site/prodsummary. jsp?code=SCF5250&lnodeId= 0162468rH3YTLC00M91752.
35. G. A. Quirk, "Find out whats really inside the iPod, " EE Times, 2006, http://www.eetimes.com/design/audio-design/4015931/Findout-what-s-really-inside-the-iPod.
36. PortalPlayer, " Digital media Management system-on-chip, " 2007, http://microblog.routed.net/wp-content/uploads/2007/11/pp5020e.pdf.
37. NVIDIA, "Digital media management system-on-chip, " 2009, http://www.nvidia.com/page/pp5024.html.
38. PortalPlayer, " Digital media Management system-on-chip, " 2007, http://microblog.routed.net/wp-content/uploads/2007/11/pp5020e.pdf.
39. B. Jeff, "The evolution of DSP processors, " 1997, http://www.cs.berkeley.edu/∼pattrsn/152F97/slides/slides.evolution.ps.
40. U. S. Sec urities and Exchange Commission, "Annual report, " 1998, http://www.sec.gov/Archives/.
41. Morgan Stanley, "Global technology data book, " 2006, http://www.morganstanley.com/index.html.
42. S. Ethier, Worldwide Demand Remains Strong for MP3 and Portable Media Players, 2007, http://www.instat.com/.
43. S. Ethier, "The worldwide PMP/MP3 player market: shipment growth to slow considerably, " 2008, http://www.instat.com/.
44. B. Buxton, V. Hayward, I. Pearson et al. , "Big data: the next Google. Interview by Duncan Graham-Rowe, " Nature, vol. 455, no. 7209, pp. 8-9, 2008.
45. Wikibon, "The Big List of Big Data Infographics, " 2012, http://wikibon.org/blog/big-data-infographics/.
46. P. Russom, "Big data analytics, " TDWI Best Practices Report, Fourth Quarter, 2011.
47. S. Radicati and Q. Hoang, Email Statistics Report, 2012-2016, The Radicati Group, London, UK, 2012.
48. J. Manyika, M. Chui, B. Brown et al. , "Big data: the next frontier for innovation, competition, and productivity, " McKinsey Global Institute, 2011.
49. A. Hadoop, "Hadoop, " 2009, http://hadoop.apache.org/.
50. D. Wang, "An efficient cloud storage model for heterogeneous cloud infrastructures, " Procedia Engineering, vol. 23, pp. 510-515, 2011.
51. K. Bakshi, "Considerations for big data: architecture and approach, " in Proceedings of the IEEE Aerospace Conference, pp. 1-7, Big Sky, Mont, USA, March 2012.
52. A. V. Aho, "Computation and computational thinking, " The Computer Journal, vol. 55, no. 7, pp. 833-835, 2012.
53. S. S. V. Bhatnagar and S. Srinivasa, Big Data Analytics, 2012.
54. M. Pastorelli, A. Barbuzzi, D. Carra, M. DellAmico, and P. Michiardi, "HFSP: size-based scheduling for Hadoop, " in Proceedings of the IEEE International Congress on BigData (BigData '13), pp. 51-59, IEEE, 2013.
55. A. Katal, M. Wazid, and R. H. Goudar, "Big data: issues, challenges, tools and good practices, " in Proceedings of the 6th International Conference on Contemporary Computing (IC3 '13), pp. 404-409, IEEE, 2013.
56. A. Azzini and P. Ceravolo, "Consistent process mining over big data triple stores, " in Proceeding of the International Congress on Big Data (BigData '13), pp. 54-61, 2013.
57. A. ODriscoll, J. Daugelaite, and R. D. Sleator, "Big data, Hadoop and cloud computing in genomics, " Journal of Biomedical Informatics, vol. 46, no. 5, pp. 774-781, 2013.
58. Y. Demchenko, P. Grosso, C. de Laat, and P. Membrey, "Addressing big data issues in scientific data infrastructure, " in Proceedings of the IEEE International Conference on Collaboration Technologies and Systems (CTS 13), pp. 48-55, May 2013.
59. Y. Demchenko, C. Ngo, and P. Membrey, "Architecture Framework and Components for the Big Data Ecosystem, " Journal of System and Network Engineering, pp. 1-31, 2013.
60. M. Loukides, "What is data science? The future belongs to the companies and people that turn data into products, " AnOReilly Radar Report, 2010.
61. A. Wahab, M. Helmy, H. Mohd, M. Norzali, H. F. Hanafi, and M. F. M. Mohsin, "Data pre-processing on web server logs for generalized association rules mining algorithm, " Proceedings of World Academy of Science: Engineering&Technology, pp. 48-53.
62. A. Nanopoulos, M. Zakrzewicz, T. Morzy, and Y. Manolopoulos, "Indexing web access-logs for pattern queries, " in Proceedings of the 4th International Workshop on Web Information and Data Management (WIDM 02), pp. 63-68, ACM, November 2002.
63. K. P. Joshi, A. Joshi, and Y. Yesha, "On using a warehouse to analyzeweb logs, "Distributed and ParallelDatabases, vol. 13, no. 2, pp. 161-180, 2003.
64. V. Chandramohan and K. Christensen, "A first look at wired sensor networks for video surveillance systems, " in Proceedings of the 27th Annual IEEE Conference on Local Computer Networks (LCN '02), pp. 728-729, November 2002.
65. L. Selavo, A. Wood, Q. Cao et al. , "LUSTER: wireless sensor network for environmental research, " in Proceedings of the 5th ACM International Conference on Embedded Networked Sensor Systems (SenSys 07), pp. 103-116, ACM, November 2007.
66. G. Barrenetxea, F. Ingelrest, G. Schaefer, M. Vetterli, O. Couach, and M. Parlange, "Sensorscope: out-of-the-box environmental monitoring, " in Proceedings of the IEEE International Conference on Information Processing in Sensor Networks (IPSN 08), pp. 332-343, St. Louis, Mo, USA, April 2008.
67. Y. Kim, T. Schmid, Z. M. Charbiwala, J. Friedman, and M. B. Srivastava, "NAWMS: nonintrusive autonomous water monitoring system, " in Proceedings of the 6th ACM Conference on Embedded Networked Sensor Systems (SenSys '08), pp. 309-321, Raleigh, NC, USA, November 2008.
68. S. Kim, S. Pakzad, D. Culler et al. , "Health monitoring of civil infrastructures using wireless sensor networks, " in Proceedings of the 6th International Symposium on Information Processing in Sensor Networks (IPSN 07), pp. 254-263, IEEE, April 2007.
69. M. Ceriotti, L. Mottola, G. P. Picco et al. , "Monitoring heritage buildings with wireless sensor networks: the Torre Aquila deployment, " in Proceedings of the International Conference on Information Processing in Sensor Networks (IPSN 09), pp. 277-288, IEEE Computer Society, April 2009.
70. G. Tolle, J. Polastre, R. Szewczyk et al. , "A macroscope in the redwoods, " in Proceedings of the 3rd International Conference on Embedded Networked Sensor Systems, pp. 51-63, ACM, 2005.
71. F. Wang and J. Liu, "Networked wireless sensor data collection: issues, challenges, and approaches, " IEEE Communications Surveys & Tutorials, vol. 13, no. 4, pp. 673-687, 2011.
72. J. Cho and H. Garcia-Molina, "Parallel crawlers, " in Proceedings of the 11th International Conference onWorldWideWeb, pp. 124-135, ACM, May 2002.
73. S. Choudhary, M. E. Dincturk, S. M. Mirtaheri et al. , "Crawling rich internet applications: the state of the art, " in Proceedings of the Conference of the Center for Advanced Studies on Collaborative Research (CASCON '12), pp. 146-160, 2012.
74. A. Labrinidis and H. Jagadish, "Challenges and opportunities with big data, " Proceedings of the VLDB Endowment, vol. 5, no. 12, pp. 2032-2033, 2012.
75. J. Fan and H. Liu, "Statistical analysis of big data on pharmacogenomics, " Advanced Drug Delivery Reviews, vol. 65, no. 7, pp. 987-1000, 2013.
76. D. E. OLeary, "Artificial intelligence and big data, " IEEE Intelligent Systems, vol. 28, no. 2, pp. 96-99, 2013.
77. K. Michael and K. W. Miller, "Big data: new opportunities and new challenges, " Editorial: IEEE Computer, vol. 46, no. 6, pp. 22-24, 2013.
78. E. Begoli and J. Horey, "Design principles for effective knowledge discovery frombig data, " in Proceedings of the 10thWorking IEEE/IFIP Conference on Software Architecture (ECSA 12), pp. 215-218, August 2012.
79. D. Talia, "Clouds for scalable big data analytics, " Computer, vol. 46, no. 5, Article ID 6515548, pp. 98-101, 2013.
80. C. H. Lee and T. F. Chien, "Leveraging microblogging big data with a modified density-based clustering approach for event awareness and topic ranking, " Journal of Information Science, vol. 39, no. 4, pp. 523-543, 2013.
81. C. A. Steed, D. M. Ricciuto, G. Shipman et al. , "Big data visual analytics for exploratory earth system simulation analysis, " Computers and Geosciences, vol. 61, pp. 71-82, 2013.
82. J. Lin and D. Ryaboy, "Scaling big data mining infrastructure: the twitter experience, " ACM SIGKDD Explorations Newsletter, vol. 14, no. 2, pp. 6-19, 2013.
83. X. Wu, V. Kumar, J. R. Quinlan et al. , "Top 10 algorithms in data mining, " Knowledge and Information Systems, vol. 14, no. 1, pp. 1-37, 2008.
84. R. A. Johnson and D. W. Wichern, AppliedMultivariate Statistical Analysis, vol. 5, Prentice Hall, Upper Saddle River, NJ, USA, 2002.
85. A. Bifet, G. Holmes, R. Kirkby, and B. Pfahringer, "Moa:Massive online analysis, " The Journal of Machine Learning Research, vol. 11, pp. 1601-1604, 2010.
86. D. Che, M. Safran, and Z. Peng, "From big data to big data mining: challenges, issues, and opportunities, " in Database Systems for Advanced Applications, B. Hong, X. Meng, L. Chen, W. Winiwarter, and W. Song, Eds. , vol. 7827 of Lecture Notes in Computer Science, pp. 1-15, Springer, Berlin, Germany, 2013.
87. Z. Sebepou and K. Magoutis, "Scalable storage support for data stream processing, " in Proceedings of the IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST 10), pp. 1-6, Incline Village, Nev, USA, May 2010.
88. P. Zikopoulos and C. Eaton, Understanding Big Data: Analytics for Enterprise Class Hadoop and Streaming Data, McGraw-Hill Osborne Media, 2011.
89. S. Mohanty, M. Jagadeesh, and H. Srivatsa, BigData Imperatives: Enterprise "Big Data" Warehouse, "BI" Implementations and Analytics, Apress, 2013.
90. S. Madden, "From databases to big data, " IEEE Internet Computing, vol. 16, no. 3, pp. 4-6, 2012.
91. N. Elmqvist and P. Irani, "Ubiquitous analytics: interacting with big data anywhere, anytime, " Computer, vol. 46, no. 4, pp. 86-89, 2013.
92. G. Greenwald, "NSA collecting phone records of millions of Verizon customers daily, " The Guardian, 2013, http://www .theguardian.com/world/2013/jun/06/nsa-phone-recordsverizon-court-order.
93. G. Greenwald and E. MacAskill, "NSA Prism Program Taps in to User Data of Apple, Google and Others, " Guardian, 2013, http://www.guardian.co.uk/world/2013/jun/06/us-tech-giantsnsa-data.
94. J. Polonetsky and O. Tene, "Privacy and big data: making ends meet, " Stanford Law Review Online, vol. 66, 25, 2013.
95. I. Rubinstein, "Big data: the end of privacy or a new beginning?" NYU School of Law, Public Law Research Paper, pp. 12-56, 2012.
96. R. Clarke, "Privacy impact assessment: its origins and development, " Computer Law and Security Review, vol. 25, no. 2, pp. 123-135, 2009.
97. V. Mayer-Schönberger and K. Cukier, Big Data: A Revolution That Will Transform How We Live, Work, and Think, Eamon Dolan/Houghton Mifflin Harcourt, 2013.
98. C. Directive, "Council Directive 96/23/EC of 29 April 1996 on measures to monitor certain substances and residues, " Repealing Directives 85/358/EEC and 86/469/EEC and Decisions 89/187/EEC and 91/664/EEC, OJ EC L 125, pp. 10-31, 1996.
99. Q. Xu and G. Liu, "Configuring Clark-Wilson integrity model to enforce flexible protection, " in Proceedings of the International Conference on Computational Intelligence and Security (CIS '09), vol. 2, pp. 15-20, Beijing, China, December 2009.
100. M. Zhang, "Strict integrity policy of Biba model with dynamic characteristics and its correctness, " in Proceedings of the International Conference on Computational Intelligence and Security (CIS 09), vol. 1, pp. 521-525, December 2009.
101. B. Priyadharshini and P. Parvathi, "Data integrity in cloud storage, " in Proceedings of the 1st International Conference on Advances in Engineering, Science and Management (ICAESM 12), pp. 261-265, March 2012.
102. Z. Xiao and Y. Xiao, "Security and privacy in cloud computing, " IEEE Communications Surveys and Tutorials, vol. 15, no. 2, pp. 843-859, 2013.
103. M. Jensen, J. Schwenk, N. Gruschka, and L. L. Iacono, "On technical security issues in cloud computing, " in Proceedings of the IEEE International Conference on Cloud Computing (CLOUD 09), pp. 109-116, Bangalore, India, September 2009.
104. L. Xu, D. Sun, and D. Liu, "Study onmethods for data confidentiality and data integrity in relational database, " in Proceedings of the 3rd IEEE International Conference on Computer Science and Information Technology (ICCSIT 10), vol. 1, pp. 292-295, July 2010.
105. T. White, Hadoop, The Definitive Guide, OReilly Media, 2010.
106. J. Hurwitz, A. Nugent, F. Halper, and M. Kaufman, Big Data for Dummies, JohnWiley & Sons, 2013, http://www.wiley.com/.
107. J. Ahrens, B. Hendrickson, G. Long, S. Miller, R. Ross, and D. Williams, "Data-intensive science in the US DOE: case studies and future challenges, " Computing in Science and Engineering, vol. 13, no. 6, pp. 14-23, 2011.
108. T. Hey, S. Tansley, and K. Tolle, The Fourth Paradigm: Data-Intensive Scientific Discovery, 2009.
109. N. S. Kim, T. Austin, D. Blaauw et al. , "Leakage current:Moores law meets static power, " Computer, vol. 36, no. 12, pp. 68-75, 2003.
110. R. T. Kouzes, G. A. Anderson, S. T. Elbert, I. Gorton, and D. K. Gracio, "The changing paradigm of data-intensive computing, " IEEE Computer, vol. 42, no. 1, pp. 26-34, 2009.
111. J. Han, M. Kamber, and J. Pei, Data Mining: Concepts and Techniques, Morgan Kaufmann, 2006.