Data-intensive text processing with MapReduce

Synthesis Lectures on Human Language Technologies

Volumn 3, Issue 1, 2010, Pages 1-177

  Lin, Jimmy ^a   Dyer, Chris ^a  

^a UNIVERSITY OF MARYLAND   (United States)

Author keywords

Algorithm design; Hadoop; Information retrieval; Machine learning; Natural language processing; Parallel and distributed programming; Text processing

Indexed keywords

ARTIFICIAL INTELLIGENCE; DISTRIBUTED COMPUTER SYSTEMS; FAULT TOLERANCE; INFORMATION RETRIEVAL; LEARNING ALGORITHMS; LEARNING SYSTEMS; NATURAL LANGUAGE PROCESSING SYSTEMS; TEXT PROCESSING;

ALGORITHM DESIGN; COMPUTING APPLICATIONS; DISTRIBUTED COMPUTATIONS; HADOOP; LARGE AMOUNTS OF DATA; LARGE-SCALE DATA PROCESSING; PARALLEL AND DISTRIBUTED PROGRAMMING; SCALABLE ALGORITHMS;

DATA HANDLING;

EID: 84964816728     PISSN: None     EISSN: 19474040     Source Type: Book Series    
DOI: 10.2200/S00274ED1V01Y201006HLT007     Document Type: Article

References (162)

1. Azza Abouzeid, Kamil Bajda-Pawlikowski, Daniel Abadi, Avi Silberschatz, and Alexander Rasin. HadoopDB: An architectural hybrid of MapReduce and DBMS technologies for analytical workloads. In Proceedings of the 35th International Conference on Very Large Data Base (VLDB 2009), pages 922-933, Lyon, France, 2009. 59
2. Réka Albert and Albert-László Barabási. Statistical mechanics of complex networks. Reviews of Modern Physics, 74:47-97, 2002. DOI: 10.1103/RevModPhys.74.47 92
3. Noga Alon, Yossi Matias, and Mario Szegedy. The space complexity of approximating the frequencymoments. In Proceedings of the 28th Annual ACMSymposium onTheory of Computing (STOC '96), pages 20-29, Philadelphia, Pennsylvania, 1996. DOI: 10.1145/237814.237823 145
4. Peter Alvaro, Tyson Condie, Neil Conway, Khaled Elmeleegy, Joseph M. Hellerstein, and Russell C. Sears. BOOM: Data-centric programming in the datacenter. Technical Report UCB/EECS-2009-98, Electrical Engineering and Computer Sciences, University of California at Berkeley, 2009. 32
5. Gene Amdahl. Validity of the single processor approach to achieving large-scale computing capabilities. In Proceedings of the AFIPS Spring Joint Computer Conference, pages 483-485, 1967. DOI: 10.1145/1465482.1465560 17
6. Rajagopal Ananthanarayanan, Karan Gupta, Prashant Pandey, Himabindu Pucha, Prasenjit Sarkar, Mansi Shah, and Renu Tewari. Cloud analytics: Do we really need to reinvent the storage stack? In Proceedings of the 2009Workshop onHotTopics in Cloud Computing (HotCloud 09), San Diego, California, 2009. 29
7. Thomas Anderson, Michael Dahlin, Jeanna Neefe, David Patterson,Drew Roselli, and RandolphWang. Serverless network file systems. In Proceedings of the 15th ACM Symposium on Operating Systems Principles (SOSP 1995), pages 109-126, Copper Mountain Resort, Colorado, 1995. DOI: 10.1145/224056.224066 29
8. Vo Ngoc Anh and Alistair Moffat. Inverted index compression using word-aligned binary codes. Information Retrieval, 8(1):151-166, 2005. DOI: 10.1023/B:INRT.0000048490.99518.5c 76
9. Michael Armbrust,ArmandoFox,Rean Griffith,AnthonyD. Joseph,Randy H.Katz,Andrew Konwinski, Gunho Lee, David A. Patterson, Ariel Rabkin, Ion Stoica, and Matei Zaharia. Above the clouds:ABerkeley viewof cloud computing.Technical Report UCB/EECS-2009-28, Electrical Engineering and Computer Sciences, University of California at Berkeley, 2009. 6
10. Arthur Asuncion, Padhraic Smyth, and MaxWelling. Asynchronous distributed learning of topic models. In Advances in Neural Information Processing Systems 21 (NIPS 2008), pages 81-88, Vancouver, British Columbia, Canada, 2008. 144
11. Ricardo Baeza-Yates, Carlos Castillo, Flavio Junqueira, Vassilis Plachouras, and Fabrizio Silvestri. Challenges on distributed web retrieval. In Proceedings of the IEEE 23rd International Conference on Data Engineering (ICDE 2007), pages 6-20, Istanbul, Turkey, 2007. DOI: 10.1109/ICDE.2007.367846 83
12. Ricardo Baeza-Yates, Carlos Castillo, and Vicente López. PageRank increase under different collusion topologies. In Proceedings of the First International Workshop on Adversarial Information Retrieval on theWeb (AIRWeb 2005), pages 17-24, Chiba, Japan, 2005. 96
13. Ricardo Baeza-Yates, Aristides Gionis, Flavio Junqueira, Vanessa Murdock, Vassilis Plachouras, and Fabrizio Silvestri. The impact of caching on search engines. In Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2007), pages 183-190, Amsterdam, The Netherlands, 2007. DOI: 10.1145/1277741.1277775 82
14. Michele Banko and Eric Brill. Scaling to very very large corpora for natural language disambiguation. In Proceedings of the 39th Annual Meeting of the Association for Computational Linguistics (ACL 2001), pages 26-33, Toulouse, France, 2001. DOI: 10.3115/1073012.1073017 4
15. Paul Barham, Boris Dragovic, Keir Fraser, Steven Hand,Tim Harris, Alex Ho, Rolf Neugebauer, Ian Pratt, and AndrewWarfield. Xen and the art of virtualization. In Proceedings of the 19th ACM Symposium on Operating Systems Principles (SOSP 2003), pages 164-177, Bolton Landing, New York, 2003. DOI: 10.1145/945445.945462 6
16. Luiz André Barroso, Jeffrey Dean, and Urs Hölzle. Web search for a planet: The Google cluster architecture. IEEE Micro, 23(2):22-28, 2003. DOI: 10.1109/MM.2003.1196112 82
17. Luiz André Barroso and Urs Hölzle. The case for energy-proportional computing. Computer, 40(12):33-37, 2007. DOI: 10.1109/MC.2007.443 9
18. Luiz André Barroso and Urs Hölzle. The Datacenter as a Computer: An Introduction to the Design of Warehouse-Scale Machines. Morgan & Claypool Publishers, 2009. DOI: 10.2200/S00193ED1V01Y200905CAC006 8, 9, 10, 14
19. Jacek Becla, Andrew Hanushevsky, Sergei Nikolaev, Ghaleb Abdulla, Alex Szalay, Maria Nieto-Santisteban, Ani Thakar, and Jim Gray. Designing a multi-petabyte database for LSST. SLAC Publications SLAC-PUB-12292, Stanford Linear Accelerator Center, May 2006. 2
20. Jacek Becla and Daniel L.Wang. Lessons learned from managing a petabyte. In Proceedings of the Second Biennial Conference on Innovative Data Systems Research (CIDR 2005), Asilomar, California, 2005. 2
21. Gordon Bell,Tony Hey, and Alex Szalay. Beyond the data deluge. Science, 323(5919):1297-1298, 2009. 2
22. Monica Bianchini, Marco Gori, and Franco Scarselli. Inside PageRank. ACMTransactions on Internet Technology, 5(1):92-128, 2005. DOI: 10.1145/1052934.1052938 98, 100
23. Jorge Luis Borges. Collected Fictions (translated by Andrew Hurley). Penguin, 1999. 5
24. Léon Bottou. Stochastic learning. In Olivier Bousquet and Ulrike von Luxburg, editors, Advanced Lectures on Machine Learning, Lecture Notes in Artificial Intelligence, LNAI 3176, pages 146-168. Springer Verlag, Berlin, 2004. DOI: 10.1007/b100712 144
25. Thorsten Brants, Ashok C. Popat, Peng Xu, Franz J. Och, and Jeffrey Dean. Large language models in machine translation. In Proceedings of the 2007 Joint Conference on Empirical Methods inNatural Language Processing and ComputationalNatural Language Learning, pages 858-867, Prague, Czech Republic, 2007. 4, 5, 133
26. Thorsten Brants and Peng Xu. Distributed Language Models. Morgan&Claypool Publishers, 2010. 134
27. Eric Brill, Jimmy Lin, Michele Banko, Susan Dumais, and Andrew Ng. Data-intensive question answering. In Proceedings of the Tenth Text REtrieval Conference (TREC 2001), pages 393-400, Gaithersburg, Maryland, 2001. 4
28. Frederick P. Brooks. The Mythical Man-Month: Essays on Software Engineering, Anniversary Edition. Addison-Wesley, Reading, Massachusetts, 1995. 12
29. Peter F. Brown, Vincent J. Della Pietra, Stephen A. Della Pietra, and Robert L. Mercer. The mathematics of statistical machine translation: Parameter estimation. Computational Linguistics, 19(2):263-311, 1993. 130
30. Stefan Büttcher, Charles L. A. Clarke, and Gordon V. Cormack. Information Retrieval: Implementing and Evaluating Search Engines. MIT Press, Cambridge, Massachusetts, 2010. 78, 83
31. Rajkumar Buyya, Chee Shin Yeo, Srikumar Venugopal, James Broberg, and Ivona Brandic. Cloud computing and emerging IT platforms: Vision, hype, and reality for delivering computing as the 5th utility. Future Generation Computer Systems, 25(6):599-616, 2009. DOI: 10.1016/j.future.2008.12.001 6
32. Luis-Felipe Cabrera and DarrellD.E.Long. Swift:Using distributed disk striping to provide high I/O data rates. Computer Systems, 4(4):405-436, 1991. 29
33. Chris Callison-Burch, Philipp Koehn, Christof Monz, and Josh Schroeder. Findings of the 2009 workshop on statistical machine translation. In Proceedings of the Fourth Workshop on Statistical Machine Translation (StatMT '09), pages 1-28, Athens, Greece, 2009. 136
34. Fay Chang, Jeffrey Dean, Sanjay Ghemawat,WilsonC. Hsieh, Deborah A.Wallach, Michael Burrows,Tushar Chandra, Andrew Fikes, and RobertGruber. Bigtable:A distributed storage system for structured data. In Proceedings of the 7th Symposium on Operating System Design and Implementation (OSDI 2006), pages 205-218, Seattle,Washington, 2006. 24, 145
35. Stanley F. Chen and Joshua Goodman. An empirical study of smoothing techniques for language modeling. In Proceedings of the 34th Annual Meeting of the Association for Computational Linguistics (ACL 1996), pages 310-318, Santa Cruz, California, 1996. DOI: 10.3115/981863.981904 5, 133
36. Hung chih Yang, Ali Dasdan, Ruey-Lung Hsiao, and D. Stott Parker. Map-Reduce-Merge: Simplified relational data processing on large clusters. In Proceedings of the 2007 ACM SIGMOD International Conference on Management of Data, pages 1029-1040, Beijing, China, 2007. DOI: 10.1145/1247480.1247602 146
37. Cheng-Tao Chu, Sang Kyun Kim,Yi - An Lin,YuanYuan Yu, Gary Bradski, Andrew Ng, and Kunle Olukotun. Map-Reduce for machine learning on multicore. In Advances in Neural Information Processing Systems 19 (NIPS 2006), pages 281-288,Vancouver, British Columbia, Canada, 2006. 141
38. KennethW. Church and Patrick Hanks. Word association norms, mutual information, and lexicography. Computational Linguistics, 16(1):22-29, 1990. 48
39. Jonathan Cohen. Graph twiddling in a MapReduce world. Computing in Science and Engineering, 11(4):29-41, 2009. DOI: 10.1109/MCSE.2009.120 103
40. Brian F. Cooper, Adam Silberstein, Erwin Tam, Raghu Ramakrishnan, and Russell Sears. Benchmarking cloud serving systems with YCSB. In Proceedings of the First ACMSymposium on Cloud Computing (ACM SOCC 2010), Indianapolis, Indiana, 2010. 145
41. Thomas H. Cormen, Charles E. Leiserson, and Ronald L. Rivest. Introduction to Algorithms. MIT Press, Cambridge, Massachusetts, 1990. 88
42. W. Bruce Croft, Donald Meztler, andTrevor Strohman. Search Engines: Information Retrieval in Practice. Addison-Wesley, Reading, Massachusetts, 2009. 83
43. David Culler,RichardKarp,DavidPatterson, Abhijit Sahay,Klaus Erik Schauser,Eunice Santos, Ramesh Subramonian, and Thorsten von Eicken. LogP:Towards a realistic model of parallel computation. ACMSIGPLANNotices, 28(7):1-12, 1993.DOI: 10.1145/173284.155333 15
44. Doug Cutting, Julian Kupiec, Jan Pedersen, and Penelope Sibun. A practical part-of-speech tagger. In Proceedings of the Third Conference on Applied Natural Language Processing, pages 133-140,Trento, Italy, 1992. DOI: 10.3115/974499.974523 114
45. Jeffrey Dean and Sanjay Ghemawat. MapReduce:Simplified data processing on large clusters. In Proceedings of the 6th Symposium on Operating System Design and Implementation (OSDI 2004), pages 137-150, San Francisco, California, 2004. 1, 24, 25
46. Jeffrey Dean and Sanjay Ghemawat. MapReduce:Simplified data processing on large clusters. Communications of the ACM, 51(1):107-113, 2008. DOI: 10.1145/1327452.1327492 2
47. Jeffrey Dean and Sanjay Ghemawat. MapReduce: A flexible data processing tool. Communications of the ACM, 53(1):72-77, 2010. DOI: 10.1145/1629175.1629198 59
48. Giuseppe DeCandia, Deniz Hastorun, Madan Jampani, Gunavardhan Kakulapati, Avinash Lakshman, Alex Pilchin, Swami Sivasubramanian, Peter Vosshall, and Werner Vogels. Dynamo: Amazon's highly available key-value store. In Proceedings of the 21st ACM Symposium on Operating Systems Principles (SOSP 2007), pages 205-220, Stevenson,Washington, 2007. 145
49. Arthur P. Dempster,Nan M.Laird, and Donald B.Rubin. Maximum likelihood from incomplete data via theEMalgorithm. Journal of the Royal Statistical Society. SeriesB(Methodological), 39(1):1-38, 1977. 108
50. David J. DeWitt and Jim Gray. Parallel database systems: The future of high performance database systems. Communications of the ACM, 35(6):85-98, 1992. DOI: 10.1145/129888.129894 12
51. David J.DeWitt,Randy H.Katz,FrankOlken,LeonardD. Shapiro, Michael R. Stonebraker, and David Wood. Implementation techniques for main memory database systems. ACM SIGMOD Record, 14(2):1-8, 1984. DOI: 10.1145/971697.602261 63
52. Mark Dredze, Alex Kulesza, and Koby Crammer. Multi-domain learning by confidence-weighted parameter combination. Machine Learning, 79:123-149, 2010. DOI: 10.1007/s10994-009-5148-0 144
53. Susan Dumais, Michele Banko, Eric Brill, Jimmy Lin, and Andrew Ng. Web question answering: Is more always better? In Proceedings of the 25th Annual International ACMSIGIR Conference on Research and Development in Information Retrieval (SIGIR 2002), pages 291-298,Tampere, Finland, 2002. DOI: 10.1145/564376.564428 4
54. Chris Dyer, Aaron Cordova, Alex Mont, and Jimmy Lin. Fast, easy, and cheap: Construction of statistical machine translation models with MapReduce. In Proceedings of theThirdWorkshop on Statistical Machine Translation at ACL 2008, pages 199-207, Columbus, Ohio, 2008. 47, 135, 136
55. John R. Firth. A synopsis of linguistic theory 1930-55. In Studies in Linguistic Analysis, Special Volume of the Philological Society, pages 1-32. Blackwell, Oxford, 1957. 48
56. Qin Gao and Stephan Vogel. Training phrase-based machine translation models on the cloud: Open source machine translation toolkit Chaski. The Prague Bulletin of Mathematical Linguistics, 93:37-46, 2010. DOI: 10.2478/v10108-010-0004-8 138
57. Sanjay Ghemawat, Howard Gobioff, and Shun-Tak Leung. The Google File System. In Proceedings of the 19th ACM Symposium on Operating Systems Principles (SOSP 2003), pages 29-43, Bolton Landing, New York, 2003. DOI: 10.1145/945445.945450 29
58. Seth Gilbert and Nancy Lynch. Brewer's Conjecture and the feasibility of consistent, available, partition-tolerant web services. ACM SIGACT News, 33(2):51-59, 2002. DOI: 10.1145/564585.564601 32
59. Michelle Girvan and Mark E. J. Newman. Community structure in social and biological networks. Proceedings of the National Academy of Science, 99(12):7821-7826, 2002. DOI: 10.1073/pnas.122653799 85
60. Ananth Grama, Anshul Gupta, George Karypis, and Vipin Kumar. Introduction to Parallel Computing. Addison-Wesley, Reading, Massachusetts, 2003. 14, 103
61. Mark S.Granovetter.The strength of weak ties. The American Journal of Sociology, 78(6):1360-1380, 1973. DOI: 10.1086/225469 92
62. Mark S. Granovetter. The strength of weak ties: A network theory revisited. Sociological Theory, 1:201-233, 1983. DOI: 10.2307/202051 92
63. Zoltán Gyöngyi and Hector Garcia-Molina. Web spam taxonomy. In Proceedings of the First InternationalWorkshop on Adversarial Information Retrieval on theWeb (AIRWeb 2005), pages 39-47, Chiba, Japan, 2005. 96
64. Per Hage and Frank Harary. Island Networks: Communication, Kinship, and Classification Structures in Oceania. Cambridge University Press, Cambridge, England, 1996. 86
65. Alon Halevy, Peter Norvig, and Fernando Pereira. The unreasonable effectiveness of data. Communications of the ACM, 24(2):8-12, 2009. DOI: 10.1109/MIS.2009.36 5
66. James Hamilton. On designing and deploying Internet-scale services. In Proceedings of the 21st Large Installation System Administration Conference (LISA '07), pages 233-244, Dallas, Texas, 2007. 10
67. James Hamilton. Cooperative Expendable Micro-Slice Servers (CEMS): Low cost, low power servers for Internet-scale services. In Proceedings of the Fourth Biennial Conference on Innovative Data Systems Research (CIDR 2009), Asilomar, California, 2009. 9, 10
68. Jeff Hammerbacher. Information platforms and the rise of the data scientist. InToby Segaran and JeffHammerbacher, editors, Beautiful Data,pages 73-84.O'Reilly,Sebastopol, California, 2009. 6, 59, 146
69. Zelig S. Harris. Mathematical Structures of Language. Wiley, New York, 1968. 48
70. Md. Rafiul Hassan and Baikunth Nath. Stock market forecasting using hidden Markov models: A new approach. In Proceedings of the 5th International Conference on Intelligent Systems Design and Applications (ISDA '05), pages 192-196, Wroclaw, Poland, 2005. DOI: 10.1109/ISDA.2005.85 114
71. Bingsheng He,Wenbin Fang, Qiong Luo, Naga K. Govindaraju, and TuyongWang. Mars: A MapReduce framework on graphics processors. In Proceedings of the 17th International Conference on Parallel Architectures and Compilation Techniques (PACT 2008), pages 260-269, Toronto, Ontario, Canada, 2008. DOI: 10.1145/1454115.1454152 20
72. Tony Hey, Stewart Tansley, and Kristin Tolle. The Fourth Paradigm: Data-Intensive Scientific Discovery. Microsoft Research, Redmond,Washington, 2009. 3
73. Tony Hey, StewartTansley, and KristinTolle. Jim Gray on eScience:A transformed scientific method. In Tony Hey, Stewart Tansley, and Kristin Tolle, editors, The Fourth Paradigm: Data-Intensive Scientific Discovery. Microsoft Research, Redmond,Washington, 2009. 3
74. John Howard, Michael Kazar, Sherri Menees, David Nichols, Mahadev Satyanarayanan, Robert Sidebotham, and MichaelWest. Scale and performance in a distributed file system. ACMTransactions on Computer Systems, 6(1):51-81, 1988. DOI: 10.1145/35037.35059 29
75. Michael Isard, Mihai Budiu, Yuan Yu, Andrew Birrell, and Dennis Fetterly. Dryad: Distributed data-parallel programs from sequential building blocks. In Proceedings of the ACM SIGOPS/EuroSys European Conference on Computer Systems 2007 (EuroSys 2007), pages 59-72, Lisbon, Portugal, 2007. DOI: 10.1145/1272998.1273005 145
76. Adam Jacobs. The pathologies of big data. ACMQueue, 7(6), 2009. DOI: 10.1145/1563821.1563874 11
77. Joseph JaJa. An Introduction to Parallel Algorithms. Addison-Wesley, Reading, Massachusetts, 1992. 14, 103
78. Frederick Jelinek. Statistical methods for speech recognition. MIT Press, Cambridge, Massachusetts, 1997. 112, 114, 120
79. Daniel Jurafsky and James H. Martin. Speech and Language Processing. Pearson, Upper Saddle River, New Jersey, 2009. 5
80. U Kang, CharalamposTsourakakis, Ana Paula Appel, Christos Faloutsos, and Jure Leskovec. HADI: Fast diameter estimation and mining in massive graphs with Hadoop. Technical Report CMU-ML-08-117, School of Computer Science, Carnegie Mellon University, 2008. 103
81. U Kang, Charalampos E. Tsourakakis, and Christos Faloutsos. PEGASUS: A peta-scale graph mining system - implementation and observations. In Proceedings of the 2009 Ninth IEEE International Conference on Data Mining (ICDM 2009), pages 229-238, Miami, Floria, 2009. DOI: 10.1109/ICDM.2009.14 103
82. Howard Karloff, Siddharth Suri, and Sergei Vassilvitskii. A model of computation for Map-Reduce. In Proceedings of the 21st Annual ACM-SIAM Symposium on Discrete Algorithms (SODA 2010), Austin,Texas, 2010. 15
83. Aaron Kimball, Sierra Michels-Slettvet, and Christophe Bisciglia. Cluster computing for Web-scale data processing. In Proceedings of the 39th ACM Technical Symposium on Computer Science Education (SIGCSE 2008), pages 116-120, Portland, Oregon, 2008. DOI: 10.1145/1352135.1352177 71
84. Jon M. Kleinberg. Authoritative sources in a hyperlinked environment. Journal of the ACM, 46(5):604-632, 1999. DOI: 10.1145/324133.324140 65, 95
85. Philipp Koehn. Statistical Machine Translation. Cambridge University Press, Cambridge, England, 2010. 130, 133
86. Philipp Koehn, Franz J. Och, and Daniel Marcu. Statistical phrase-based translation. In Proceedings of the 2003 Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics (HLT/NAACL 2003), pages 48-54, Edmonton, Alberta, Canada, 2003. DOI: 10.3115/1073445.1073462 131
87. JohnD.Lafferty,AndrewMcCallum,andFernandoPereira. Conditional random fields:Probabilistic models for segmenting and labeling sequence data. In Proceedings of the Eighteenth International Conference on Machine Learning (ICML '01), pages 282-289, San Francisco, California, 2001. 138
88. Ronny Lempel and Shlomo Moran. SALSA: The Stochastic Approach for Link-Structure Analysis. ACM Transactions on Information Systems, 19(2):131-160, 2001. DOI: 10.1145/382979.383041 65, 95
89. Abby Levenberg, Chris Callison-Burch, and Miles Osborne. Stream-based translation models for statistical machine translation. In Proceedings of the 11th Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL HLT 2010), Los Angeles, California, 2010. 145
90. Abby Levenberg and Miles Osborne. Stream-based randomised language models for SMT. In Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, pages 756-764, Singapore, 2009. 145
91. Adam Leventhal. Triple-parity RAID and beyond. ACM Queue, 7(11), 2009. DOI: 10.1145/1661785.1670144 2
92. Jimmy Lin. An exploration of the principles underlying redundancy-based factoid question answering. ACM Transactions on Information Systems, 27(2):1-55, 2007. DOI: 10.1145/1229179.1229180 4
93. Jimmy Lin. Exploring large-data issues in the curriculum: A case study with MapReduce. In Proceedings of the ThirdWorkshop on Issues inTeaching Computational Linguistics (TeachCL-08) at ACL 2008, pages 54-61, Columbus, Ohio, 2008. 71
94. Jimmy Lin. Scalable language processing algorithms for the masses:Acase study in computing word co-occurrence matrices with MapReduce. In Proceedings of the 2008 Conference on Empirical Methods inNatural Language Processing (EMNLP 2008), pages 419-428, Honolulu, Hawaii, 2008. 47, 51
95. Jimmy Lin, Anand Bahety, Shravya Konda, and Samantha Mahindrakar. Low-latency, highthroughput access to static global resources within the Hadoop framework. Technical Report HCIL-2009-01, University of Maryland, College Park, Maryland, January 2009. 63
96. Dong C. Liu, Jorge Nocedal, Dong C. Liu, and Jorge Nocedal. On the limited memory BFGS method for large scale optimization. Mathematical Programming B, 45(3):503-528, 1989. DOI: 10.1007/BF01589116 139
97. Adam Lopez. Statistical machine translation. ACM Computing Surveys, 40(3):1-49, 2008. DOI: 10.1145/1380584.1380586 130
98. Grzegorz Malewicz, Matthew H. Austern, Aart J. C. Bik, James C. Dehnert, Ilan Horn, Naty Leiser, and Grzegorz Czajkowski. Pregel: A system for large-scale graph processing. In Proceedings of the 28th ACMSymposium on Principles of Distributed Computing (PODC 2009), page 6, Calgary, Alberta, Canada, 2009. DOI: 10.1145/1583991.1584010 86, 145
99. Grzegorz Malewicz, Matthew H. Austern, Aart J. C. Bik, James C. Dehnert, Ilan Horn, Naty Leiser, and Grzegorz Czajkowski. Pregel: A system for large-scale graph processing. In Proceedings of the 2010 ACM SIGMOD International Conference on Management of Data, Indianapolis, Indiana, 2010. DOI: 10.1145/1582716.1582723 86, 145
100. Robert Malouf. A comparison of algorithms for maximum entropy parameter estimation. In Proceedings of the Sixth Conference on Natural Language Learning (CoNLL-2002), pages 49-55,Taipei,Taiwan, 2002. DOI: 10.3115/1118853.1118871 140
101. Christopher D. Manning, Prabhakar Raghavan, and Hinrich Schütze. An Introduction to Information Retrieval. Cambridge University Press, Cambridge, England, 2008. 42, 83
102. Christopher D. Manning and Hinrich Schütze. Foundations of Statistical Natural Language Processing. MIT Press, Cambridge, Massachusetts, 1999. 5, 133
103. Elaine R. Mardis. The impact of next-generation sequencing technology on genetics. Trends in Genetics, 24(3):133-141, 2008. DOI: 10.1016/j.tig.2007.12.007 2
104. Michael D. McCool. Scalable programming models for massively multicore processors. Proceedings of the IEEE, 96(5):816-831, 2008. DOI: 10.1109/JPROC.2008.917731 13, 145
105. Marshall K. McKusick and Sean Quinlan. GFS: Evolution on fast-forward. ACM Queue, 7(7), 2009. DOI: 10.1145/1594204.1594206 32
106. John Mellor-Crummey, David Whalley, and Ken Kennedy. Improving memory hierarchy performance for irregular applications using data and computation reorderings. International Journal of Parallel Programming, 29(3):217-247, 2001.DOI: 10.1023/A:1011119519789 101
107. Donald Metzler, Jasmine Novak, Hang Cui, and Srihari Reddy. Building enriched document representations using aggregated anchor text. In Proceedings of the 32nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2009), pages 219-226, 2009. DOI: 10.1145/1571941.1571981 68, 88
108. David R. H. Miller,Tim Leek, and RichardM. Schwartz. A hidden Markov model information retrieval system. In Proceedings of the 22nd Annual International ACMSIGIR Conference on Research and Development in Information Retrieval (SIGIR 1999), pages 214-221, Berkeley, California, 1999. DOI: 10.1145/312624.312680 114
109. Alistair Moffat, William Webber, and Justin Zobel. Load balancing for term-distributed parallel retrieval. In Proceedings of the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2006), pages 348-355, Seattle, Washington, 2006. DOI: 10.1145/1148170.1148232 82
110. Kamal Nigam, John Lafferty, and Andrew McCallum. Using maximum entropy for text classification. In Proceedings of the IJCAI-99 Workshop on Machine Learning for Information Filtering, pages 61-67, Stockholm, Sweden, 1999. 138
111. Daniel Nurmi, Rich Wolski, Chris Grzegorczyk, Graziano Obertelli, Sunil Soman, Lamia Youseff, and Dmitrii Zagorodnov. The Eucalyptus open-source cloud-computing system. In Proceedings of the 9th IEEE/ACM International Symposium on Cluster Computing and the Grid, pages 124-131,Washington, D.C., 2009. DOI: 10.1109/CCGRID.2009.93 7
112. Franz J. Och and Hermann Ney. A systematic comparison of various statistical alignment models. Computational Linguistics, 29(1):19-51, 2003.DOI: 10.1162/089120103321337421 135
113. Christopher Olston and Marc Najork. Web crawling. Foundations and Trends in Information Retrieval, 4(3):175-246, 2010. DOI: 10.1561/1500000017 67
114. ChristopherOlston, Benjamin Reed, Utkarsh Srivastava,Ravi Kumar, and AndrewTomkins. Pig Latin: A not-so-foreign language for data processing. In Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data, pages 1099-1110, Vancouver, British Columbia, Canada, 2008. DOI: 10.1145/1376616.1376726 59, 146
115. KunleOlukotun and Lance Hammond. The future of microprocessors. ACMQueue, 3(7):27-34, 2005. DOI: 10.1145/1095408.1095418 13
116. Sean Owen and Robin Anil. Mahout in Action. Manning Publications Co., Greenwich, Connecticut, 2010. 141
117. Lawrence Page, Sergey Brin, Rajeev Motwani, and TerryWinograd. The PageRank citation ranking: Bringing order to the Web. Stanford Digital Library Working Paper SIDL-WP-1999-0120, Stanford University, 1999. 65, 95, 100
118. Bo Pang and Lillian Lee. Opinion mining and sentiment analysis. Foundations and Trends in Information Retrieval, 2(1-2):1-135, 2008. DOI: 10.1561/1500000011 3
119. David A. Patterson. The data center is the computer. Communications of the ACM, 52(1):105, 2008. 14
120. Andrew Pavlo, Erik Paulson, Alexander Rasin, Daniel J. Abadi, David J. DeWitt, Samuel Madden, and Michael Stonebraker. A comparison of approaches to large-scale data analysis. In Proceedings of the 35th ACM SIGMOD International Conference on Management of Data, pages 165-178, Providence, Rhode Island, 2009. DOI: 10.1145/1559845.1559865 59
121. Sasa Petrovic, Miles Osborne, and Victor Lavrenko. Streaming first story detection with application to Twitter. In Proceedings of the 11th Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL HLT 2010), Los Angeles, California, 2010. 145
122. Rob Pike,Sean Dorward,RobertGriesemer,and SeanQuinlan. Interpreting the data:Parallel analysis with Sawzall. Scientific Programming Journal, 13(4):277-298, 2005. 146
123. Eduardo Pinheiro,Wolf-DietrichWeber, and Luiz André Barroso. Failure trends in a large disk drive population. In Proceedings of the 5th USENIX Conference on File and Storage Technologies (FAST 2007), San Jose, California, 2008. 10, 26
124. Xiaoguang Qi and BrianD. Davison.Web page classification: Features and algorithms. ACM Computing Surveys, 41(2), 2009. DOI: 10.1145/1459352.1459357 82
125. Lawrence R. Rabiner. A tutorial on hidden Markov models and selected applications in speech recognition. In Readings in Speech Recognition, pages 267-296. Morgan Kaufmann Publishers, San Francisco, California, 1990. DOI: 10.1109/5.18626 115, 120
126. M.Mustafa Rafique, Benjamin Rose, Ali R. Butt, and Dimitrios S.Nikolopoulos. Supporting MapReduce on large-scale asymmetric multi-core clusters. ACM Operating Systems Review, 43(2):25-34, 2009. DOI: 10.1145/1531793.1531800 20
127. Colby Ranger, Ramanan Raghuraman, Arun Penmetsa, Gary Bradski, and Christos Kozyrakis. Evaluating MapReduce for multi-core and multiprocessor systems. In Proceedings of the 13th International Symposium onHigh-Performance Computer Architecture (HPCA 2007), pages 205-218, Phoenix, Arizona, 2007. DOI: 10.1109/HPCA.2007.346181 20
128. Delip Rao and David Yarowsky. Ranking and semi-supervised classification on large scale graphs using Map-Reduce. In Proceedings of the ACL/IJCNLP 2009Workshop on Graph-Based Methods for Natural Language Processing (TextGraphs-4), Singapore, 2009. 103
129. Michael A. Rappa. The utility business model and the future of computing services. IBM Systems Journal, 34(1):32-42, 2004. DOI: 10.1147/sj.431.0032 6
130. Sheldon M. Ross. Stochastic processes. Wiley, New York, 1996. 114
131. Thomas Sandholm and Kevin Lai. MapReduce optimization using regulated dynamic prioritization. In Proceedings of the Eleventh International Joint Conference on Measurement and Modeling of Computer Systems(SIGMETRICS '09), pages 299-310,Seattle,Washington,2009. DOI: 10.1145/1555349.1555384 25
132. Michael Schatz. High Performance Computing forDNASequence Alignment and Assembly. PhD thesis, University of Maryland, College Park, 2010. 103
133. Frank Schmuck and Roger Haskin. GPFS: A shared-disk file system for large computing clusters. In Proceedings of the First USENIX Conference on File and Storage Technologies, pages 231-244, Monterey, California, 2002. 29
134. Donovan A. Schneider and David J. DeWitt. A performance evaluation of four parallel join algorithms in a shared-nothing multiprocessor environment. In Proceedings of the 1989 ACM SIGMOD International Conference on Management of Data, pages 110-121,Portland, Oregon, 1989. DOI: 10.1145/67544.66937 60
135. Bianca Schroeder, Eduardo Pinheiro, andWolf-DietrichWeber. DRAM errors in the wild: A large-scale field study. In Proceedings of the Eleventh International Joint Conference on Measurement and Modeling of Computer Systems (SIGMETRICS '09), pages 193-204, Seattle, Washington, 2009. DOI: 10.1145/1555349.1555372 10, 26
136. Hinrich Schütze. Automatic word sense discrimination. Computational Linguistics, 24(1):97-123, 1998. 48
137. Hinrich Schütze and Jan O. Pedersen. A cooccurrence-based thesaurus and two applications to information retrieval. Information Processing and Management, 33(3):307-318, 1998. DOI: 10.1016/S0306-4573(96)00068-4 48
138. Satoshi Sekine and Elisabete Ranchhod. Named Entities: Recognition, Classification and Use. John Benjamins, Amsterdam, The Netherlands, 2009. 3
139. Kristie Seymore, Andrew Mccallum, and Ronald Rosenfeld. Learning hidden Markov model structure for information extraction. In Proceedings of the AAAI-99 Workshop on Machine Learning for Information Extraction, pages 37-42, Orlando, Florida, 1999. 114
140. Fei Sha and Fernando Pereira. Shallow parsing with conditional random fields. In Proceedings of the 2003 Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics (HLT/NAACL 2003), pages 134-141, Edmonton, Alberta, Canada, 2003. DOI: 10.3115/1073445.1073473 140
141. Noah Smith. Log-linear models. http://www.cs.cmu.edu/~nasmith/papers/smith. tut04.pdf, 2004. 140
142. Christopher Southan and Graham Cameron. Beyond the tsunami: Developing the infrastructure to deal with life sciences data. In Tony Hey, Stewart Tansley, and Kristin Tolle, editors, The Fourth Paradigm: Data-Intensive Scientific Discovery. Microsoft Research, Redmond, Washington, 2009. 2
143. Mario Stanke and Stephan Waack. Gene prediction with a hidden Markov model and a new intron submodel. Bioinformatics, 19 Suppl 2:ii215-225, October 2003. DOI: 10.1093/bioinformatics/btg1080 114
144. Michael Stonebraker, Daniel Abadi, David J. DeWitt, Sam Madden, Erik Paulson, Andrew Pavlo, and Alexander Rasin. MapReduce and parallel DBMSs: Friends or foes? Communications of the ACM, 53(1):64-71, 2010. DOI: 10.1145/1629175.1629197 59
145. Alexander S. Szalay,Peter Z.Kunszt, AniThakar, Jim Gray,Don Slutz, and Robert J.Brunner. Designing and mining multi-terabyte astronomy archives: The Sloan Digital Sky Survey. SIGMOD Record, 29(2):451-462, 2000. DOI: 10.1145/335191.335439 2
146. Wittawat Tantisiriroj, Swapnil Patil, and Garth Gibson. Data-intensive file systems for Internet services:Arose by any other name⋯.Technical Report CMU-PDL-08-114,Parallel Data Laboratory, Carnegie Mellon University, 2008. 29
147. Chandramohan A. Thekkath, Timothy Mann, and Edward K. Lee. Frangipani: A scalable distributed file system. In Proceedings of the 16th ACMSymposium on Operating Systems Principles (SOSP 1997), pages 224-237, Saint-Malo, France, 1997. DOI: 10.1145/268998.266694 29
148. Leslie G. Valiant. A bridging model for parallel computation. Communications of the ACM, 33(8):103-111, 1990. DOI: 10.1145/79173.79181 13, 14, 15, 86, 145
149. Luis M. Vaquero, Luis Rodero-Merino, Juan Caceres, and Maik Lindner. A break in the clouds: Towards a cloud definition. ACM SIGCOMM Computer Communication Review, 39(1):50-55, 2009. DOI: 10.1145/1496091.1496100 6
150. Stephan Vogel, Hermann Ney, and Christoph Tillmann. HMM-based word alignment in statistical translation. In Proceedings of the 16th International Conference on Computational Linguistics (COLING 1996), pages 836-841, Copenhagen, Denmark, 1996. DOI: 10.3115/993268.993313 114, 135
151. YiWang,Hongjie Bai, Matt Stanton,Wen-Yen Chen, and EdwardY.Chang. PLDA:Parallel latent Dirichlet allocation for large-scale applications. In Proceedings of the Fifth International Conference on Algorithmic Aspects in Information and Management (AAIM 2009), pages 301-314, San Francisco, California, 2009. DOI: 10.1007/978-3-642-02158-9-26 141
152. Duncan J. Watts and Steven H. Strogatz. Collective dynamics of 'small-world' networks. Nature, 393:440-442, 1998. DOI: 10.1038/30918 92
153. Xingzhi Wen and Uzi Vishkin. FPGA-based prototype of a PRAM-On-Chip processor. In Proceedings of the 5th Conference on Computing Frontiers, pages 55-66, Ischia, Italy, 2008. DOI: 10.1145/1366230.1366240 14
154. TomWhite. Hadoop: The Definitive Guide. O'Reilly, Sebastopol, California, 2009. 15, 23
155. Eugene Wigner. The unreasonable effectiveness of mathematics in the natural sciences. Communications in Pure and Applied Mathematics, 13(1):1-14, 1960. DOI: 10.1002/cpa.3160130102 5
156. Ian H.Witten, Alistair Moffat, and Timothy C. Bell. Managing Gigabytes: Compressing and Indexing Documents and Images. Morgan Kaufmann Publishing, San Francisco, California, 1999. DOI: 10.1023/A:1011472308196 69, 77, 78, 83
157. Jinxi Xu andW. Bruce Croft. Corpus-based stemming using cooccurrence of word variants. ACMTransactions on Information Systems, 16(1):61-81, 1998.DOI: 10.1145/267954.267957 48
158. Rui Xu and DonaldWunsch II. Survey of clustering algorithms. IEEETransactions onNeural Networks, 16(3):645-678, 2005. DOI: 10.1109/TNN.2005.845141 86
159. Yuan Yu, Michael Isard, Dennis Fetterly, Mihai Budiu, Úlfar Erlingsson, Pradeep Kumar Gunda, and Jon Currey. DryadLINQ:A system for general-purpose distributed data-parallel computing using a high-level language. In Proceedings of the 8th Symposium on Operating System Design and Implementation (OSDI 2008), pages 1-14, San Diego, California, 2008. 145
160. Matei Zaharia, Dhruba Borthakur, Joydeep Sen Sarma, Khaled Elmeleegy, Scott Shenker, and Ion Stoica. Job scheduling for multi-user MapReduce clusters. Technical Report UCB/EECS-2009-55, Electrical Engineering and Computer Sciences, University of California at Berkeley, 2009. 25
161. Matei Zaharia,Andy Konwinski, AnthonyD. Joseph,RandyKatz, and Ion Stoica. Improving MapReduce performance in heterogeneous environments. In Proceedings of the 8th Symposium on Operating System Design and Implementation (OSDI 2008), pages 29-42, San Diego, California, 2008. 25
162. Justin Zobel and Alistair Moffat. Inverted files for text search engines. ACM Computing Surveys, 38(6):1-56, 2006. 78, 83