The anatomy of big data computing

Software - Practice and Experience

Volumn 46, Issue 1, 2016, Pages 79-105

  Kune, Raghavendra ^a   Konugurthi, Pramod Kumar ^a   Agarwal, Arun ^b   Chillarige, Raghavendra Rao ^b   Buyya, Rajkumar ^c  

^a Advanced Data Processing Research Institute Secunderabad   (India)

^b UNIVERSITY OF HYDERABAD   (India)

^c UNIVERSITY OF MELBOURNE   (Australia)

Author keywords

big data; big data analytics as a service; big data cloud architecture; big data computing

Indexed keywords

COMPUTER ARCHITECTURE; DATA HANDLING; DATA MINING; DATA WAREHOUSES; DECISION MAKING; DIGITAL STORAGE;

COMPUTING DEMANDS; DATA ANALYTICS; DATA CLOUDS; DATA ORGANIZATION; INTEGRATED PLATFORM; LAYERED ARCHITECTURE; SCIENTIFIC STUDIES; TECHNICAL CHALLENGES;

BIG DATA;

EID: 84955749528     PISSN: 00380644     EISSN: 1097024X     Source Type: Journal    
DOI: 10.1002/spe.2374     Document Type: Article

References (72)

1. Dean J, Ghemawat S,. MapReduce: simplified data processing on large cluster. Communications of the ACM 2008; 51 (1): 107-113.
2. Chervenak A, Foster I, Kesselman C, Salisbury C, Tuecke S,. The data grid: towards an architecture for the distributed management and analysis of large scientific datasets. Journal of Network and Computer Applications 2000; 23 (3): 187-200.
3. Janaki A, Kubach T, Loffer M, Schmid U., Data driven management: bringing more science into management, White Paper, McKinsey Technology Initiative Perspective, 2008. http://www.mckinsey.com/insights/business-technology/big-data-the-next-frontier-for-innovation [last accessed 30 November 2014].
4. CRA (Computing Research Association), Challenges and opportunities with big data, http://www.cra.org/ccc/files/docs/init/bigdatawhitepaper.pdf [last accessed 12 November 2014].
5. Advancing discovery in science and engineering, the role of basic computing research, http://www.cra.org/ccc/files/docs/Natl-Priorities/web-data-spring.pdf [last accessed 10 August 2014].
6. IDC, The digital universe in 2020: big data, bigger digital shadows, and biggest growth in the Far East, www.emc.com/leadership/digital-universe/index.htm [last accessed 20 November 2014].
7. Apache Hadoop, http://hadoop.apache.org [last accessed 15 October 2014].
8. Chen C.L.P, Zhang C Y., Data-intensive applications, challenges, techniques and technologies: a survey on big data, Inform.Sci, DOI: 10.1016/j.ins.2014.01.015.
9. Chen M, Mao S, Liu Y,. Big data survey. Mobile Networks and Applications 2014; 19 (2): 171-209.
10. Chen J, Chen Y, Du X, Li C, Lu J, Zhao S, Zhou X,. Big data challenge: a data management perspective. Frontiers of Computer Science 2013; 7 (2): 157-164.
11. Wu X, Zhu X, Wu GQ, Ding W,. Data mining with big data. IEEE Transactions on Knowledge and Data Engineering 2014; 26 (1): 97-107.
12. Kaisler S, Armour F, Espinosa J.A, and, Money W., Big data: issues and challenges moving forward, in: Proceedings of the 46th IEEE Annual Hawaii international Conference on System Sciences (HICC 2013), Grand Wailea, Maui, Hawaii, January 2013, pp. 995-1004.
13. Assuncao MD, Calheiros RN, Bianchi S, Netto M, Buyya R,. Big data computing and clouds: trends and future directions. Journal of Parallel and Distributed Computing (JPDC) 2015; 79 (5): 3-15.
14. Survey of big data architectures and framework from the industry, NIST big data public working group, 2014. http://jtc1bigdatasg.nist.gov/-workshop2/07-NBD-PWD-Big-Data-Architectures-Survey.pdf, 2014 [last accessed 30 April 2014].
15. Mayer VV, Cukier K,. Big Data: A Revolution That Will Transform How We Live, Work and Think. John Murray Press: UK, 2013.
16. Ginsberg J,. Detecting influenza epidemics using search engine query data. Nature 2009; 457: 1012-1014.
17. Buyya R, Shin Yeo C, Venugopal S, Brobergand J, Brandic I,. Cloud computing and emerging IT platforms: vision, hype, and reality for delivering computing as the 5th utility. Future Generation Computer Systems 2009; 25 (6): 599-616.
18. Broberg J, Venugopal S, Buyya R,. Market oriented grids and utility computing: the state-of-the art and future directions. Journal of Grid Computing 2008; 6 (3): 255-276.
19. Inmon WH,. Building the Data Warehouse 4th edition. Wiley publishing Inc.: Indianapolis, 2005.
20. th ACM Symposium on Principles of Distributed Computing (PODC 2000), Portland, Oregon, July 2000.
21. Gray J., The transaction concept: virtues and limitations, in: Proceedings of the 7th International Conference on Very Large Databases (VLDB' 81) 7 (1981) 144-154.
22. Pritchett D,. BASE: an ACID alternative. Queue Object Relational Mapping 2008; 6 (3): 48-55.
23. Harder T, Reuter A,. Principles of transaction-oriented database recovery. ACM Computing Surveys 1983; 15 (4): 287-317.
24. Cooper M, Mell P., Tackling big data, NIST information technology laboratory computer security division, http://csrc.nist.gov/groups/SMA/forum/documents/june2012presentations/fcsm-june2012-cooper-mell.pdf [last accessed 15 September 2014].
25. A very short history of data science, http://whatsthebigdata.com/2012/04/26/a-very-short-history-of-data-science [last accessed 15 September 2014].
26. In-memory analytics, leveraging emerging technologies for business intelligence, Gartner Report, 2009.
27. Apache Spark, https://spark.incubator.apache.org [last accessed 03 April 2014].
28. Apache storm, http://storm.incubator.apache.org [last accessed 03 April 2014].
29. S4: distributed stream computing platform, http://incubator.apache.org/s4 [last accessed 20 November 2014].
30. Google big query, https://cloud.google.com/bigquery-tour [last accessed 15 January 2015].
31. Chang F, Dean J, Ghemawat S, Heish W. C., Wallach D. A, Burrows M, Chandra T, Fikes A, Gruber R. E., Big table: a distributed storage system for structured data, in: Proceedings of the 7th USENIX Symposium on Operating Systems Design and Implementation (OSDI 2006), Seattle, WA, Nov 2006.
32. Decandia G, Hastorun D, Jampani M, Kakulapati G, Lakshman A, Pilchin A, Sivasubramanian S, Vosshall P, Vogels W., Dynamo: Amazon's highly available key-value store, in: Proceedings of the 21st ACM Symposium on Operating Systems Principles (SOSP 2007), Stevenson, Washington, USA, Oct 2007.
33. Amazon elastic MapReduce, developer guide, 2015, http://docs.aws.amazon.com/ElasticMapReduce/latest/DeveloperGuide/emr-dg.pdf [last accessed 1 November 2014].
34. Chauhan A, Fontama V, Hart M, Hyong W, Woody B,. Introducing Microsoft Azure HDInsight, Technical Overview. Microsoft press: One Microsoft Way, Redmond, Washington, 2014.
35. Rack space, www.rackspace.com [last accessed 22 August 2014].
36. Horton Hadoop, http://hortonworks.com [last accessed 22 August 2014].
37. Cloudera Hadoop, http://www.cloudera.com [last accessed 03 September 2014].
38. Oracle Berkeley DB, Oracle data sheet, http://www.oracle.com/technetwork/products/berkeleydb/berkeley-db-datasheet-132390.pdf [last accessed 03 September 2014].
39. Buyya R, Vecchiola C, Selvi T,. Mastering in Cloud Computing-Foundations and Applications Programming. Morgan Kaufman: USA, 2013.
40. Apache couch DB, a database for the web, www.couchdb.apache.org [last accessed 10 September 2014].
41. MongoDB operations best practices, http://info.10gen.com/rs/10gen/images/10gen-MongoDB-Operations-Best-Practices.pdf.
42. Apache HBase, http://hbase.apache.org [last accessed 20 December 2014].
43. InfiniteGraph: the distributed graph database, a performance and distributed performance benchmark of InfiniteGraph and a leading open source graph database using synthetic data, infinite graph, white paper from objectivity, http://www.objectivity.com/wp-content/uploads/Objectivity-WP-IG-Distr-Benchmark.pdf, 2012 [last accessed 20 December 2014].
44. Neo4j graph database, http://www.neo4j.org [last accessed 20 December 2014].
45. Aggarwal CC, Zhai C,. Probabilistic Models for Text Mining: In Mining Text Data. Kluwer Academic Publishers: Netherlands, 2012: 257-294.
46. Nyce C., Predictive analytics white paper, American Institute for CPCU/Insurance Institute of America, http://www.theinstitutes.org/doc/predictivemodelingwhitepaper.pdf, 2007 [last accessed 30 January 2015].
47. Apache MapReduce, http://hadoop.apache.org/docs/stable/mapred-tutorial.html [last accessed 20 February 2015].
48. Apache Mahout, scalable machine learning library, http://mahout.apache.org [last accessed 20 February 2015].
49. Capacity scheduler, http://hadoop.apache.org/docs/r1.2.1/capacity-scheduler.pdf [last accessed 20 February 2015].
50. Fair scheduler, http://hadoop.apache.org/docs/r1.2.1/fair-scheduler.pdf [last accessed 20 February 2015].
51. Google-gdata,.NET library for the Google data API, http://code.google.com/p/google-gdata [last accessed 20 February 2015].
52. Cloud infrastructure management interface (CIMI) model and RESTful HTTP based protocol-an interface for managing cloud infrastructure, http://dmtf.sites/default/files/standards/documents/DSP0263-1.0.1.pdf [last accessed 13 February 2015].
53. Big data replication, www.tervela.com/big-data-replication [last accessed 20 March 2015].
54. Condie T, Conway N, Alvaro P, Hellerstein J, Elmeleegy K, Sears R,. MapReduce online, in: Proceedings of the 7th USENIX on Networked systems design and implementation (NSDI 2010), San Jose, California, April 2010.
55. Karlof H, Suri S, Vassilvitskii S,. A model of computation for MapReduce, in: Proceedings of the 21st ACM-SIAM Symposium on Discrete Algorithms (SODA 2010), Austin, Texas, January 2010.
56. Afrati F, Sarma A, Salihoglu S, Ullman J,. Vision paper: towards an understanding of the limits of Map-Reduce computation, arxiv.org/abs/1204.1754 [last accessed 09 December 2014].
57. th ACM International Symposium on High Performance Distributed Computing (HPDC 2010), Chicago, Illinois, June 2010.
58. Bhatotia P, Wieder A, Akkus I, Rodrigues R, Acar U., Large-scale incremental data processing with change propagation, in: Proceedings of the 3rd USENIX Workshop on Hot topics in Cloud computing (HotCloud 2011), Portland, June 2011.
59. Bu Y, Howe B, Balazinska M, Ernst M,. HaLoop: efficient iterative data processing on large cluster, in: Proceedings of the 36th International Conference on Very Large Databases (VLDB 2010), Singapore, September 2010.
60. nd ACM Symposium on Cloud Computing (SoCC 2011), Cascais, Portugal, October 2011.
61. Lin J., MapReduce is good enough? If all you have is a hammer, throw away everything that's not a nail!, http://arxiv.org/pdf/1209.2191v1.pdf [last accessed 02 March 2015].
62. st International Conference on Advances in Cloud Computing (ACC 2012), Bangalore, India, July 2012.
63. th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid 2010), Melbourne, Australia, May 2010.
64. Radhadevi PV, Solanki SS,. In-flight geometric calibration of different cameras of IRS-P6 using a physical sensor model. The Photogrammetric Record 2008; 23 (121): 69-89.
65. Deepak P, Kumar K. P, Varadan G,. A service oriented utility grid for data parallel remote sensing applications, in: Proceedings of 2009 High Performance Computing & Simulation (HPCS 2009) in conjunction with International Wireless Communications and Mobile Computing Conference (IWCMC 2009), Leipzig, Germany, June 2009, pp. 131-137.
66. Deepak P, Kumar K. P, Varadan G., Service oriented utility grid for 3-dimensional topographic visualization from satellite images, in: Proceedings of the 4th International Conference on eScience, Indianapolis, Indiana, USA, Dec 2008, pp. 47-54.
67. Lee JG, Kang M,. Geospatial big data: challenges and opportunities. Big Data Research 2015; 2 (2): 74-81.
68. Jardak C, Mahonen P, Riihiajariv J,. Spatial big data and wireless networks: experiences, applications, and research challenges. IEEE Network 2014; 28 (4): 26-31.
69. How big data impacts health care, Harvard business review, https://hbr.org/resources/pdfs/comm/sap/18826-HBR-SAP-Healthcare-Aug-2014.pdf, 2014. [last accessed 03 March 2015].
70. st International Conference on Web Intelligence, Mining and Semantics (WIMS'11), Sogndal, Norway, May 2011.
71. th annual conference on Information and Knowledge Engineering (IKE' 13), Las Vegas, Nevada, USA, July 2013.
72. John A,. R.A. Fisher and the making of maximum likelihood 1912-1922. Statistical Science 1997; 12 (3): 162-176.