An experience report on scaling tools for mining software repositories using mapreduce

ASE'10 - Proceedings of the IEEE/ACM International Conference on Automated Software Engineering

Volumn , Issue , 2010, Pages 275-284

  Shang, Weiyi ^a   Adams, Bram ^a   Hassan, Ahmed E ^a  

^a QUEEN S UNIVERSITY   (Canada)

Author keywords

Cloud computing; Mapreduce; Mining software repositories

Indexed keywords

ANALYSIS TECHNIQUES; CLOUD COMPUTING; ENGINEERING ANALYSIS; EXPERIENCE REPORT; HEURISTIC TECHNIQUES; LARGE DATA; MAP-REDUCE; MINING SOFTWARE REPOSITORIES; REPRESENTATIVE CASE; SCALABLE ANALYSIS; SEARCH-BASED; SOFTWARE ENGINEERING TOOLS; SOFTWARE REPOSITORIES;

AUTOMATION; COMPUTER SYSTEMS; HEURISTIC METHODS; RESEARCH;

SOFTWARE ENGINEERING;

EID: 78649774215     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1858996.1859050     Document Type: Conference Paper

References (39)

1. Amazon EC2. https://aws.amazon.com/ec2/.
2. checkmycode. http://www.checkmycode.org/.
3. CVS. http://www.cvshome.org/.
4. Debian counting. http://libresoft.es/debian-counting/.
5. MAHOUT. http://lucene.apache.org/mahout/.
6. Self-service, prorated super computing fun! http://open.blogs.nytimes. com/2007/11/01/.
7. SHARCNET. https://www.sharcnet.ca.
8. Vertica home page. http://www.vertica.com.
9. X-RIME home page. http://xrime.sourceforge.net/.
10. A. Acharya, G. Edjlali, and J. Saltz. The utility of exploiting idle workstations for parallel computation. SIGMETRICS Perform. Eval. Rev., 25(1):225-234, 1997.
11. S. Bajracharya, J. Ossher, and C. Lopes. Sourcerer: An internet-scale software repository. In SUITE'09: Proceedings of the 2009 ICSE Workshop on Search-Driven Development-Users, Infrastructure, Tools and Evaluation, pages 1-4, Washington, DC, USA, 2009. IEEE Computer Society.
12. O. Baysal and A. J. Malton. Correlating social interactions to release history during software evolution. In MSR'07: Proceedings of the Fourth International Workshop on Mining Software Repositories, page 7, Washington, DC, USA, 2007. IEEE Computer Society.
13. N. Bettenburg, R. Premraj, T. Zimmermann, and S. Kim. Extracting structural information from bug reports. In MSR'08: Proceedings of the 2008 international working conference on Mining software repositories, pages 27-30, New York, NY, USA, 2008. ACM.
14. J. Bevan, E. J. Whitehead,Jr., S. Kim, and M. Godfrey. Facilitating software evolution research with kenyon. In ESEC/FSE'05: Proceedings of the 10th European Software Engineering Conference, 2005.
15. J. Dean and S. Ghemawat. Mapreduce: simplified data processing on large clusters. Commun. ACM, 51, 2008.
16. P. Godefroid, M. Y. Levin, and D. Molnar. Automated whitebox fuzz testing. Technical report, MS-TR-2007-58, Microsoft, May 2007.
17. J. M. Gonzalez-Barahona, G. Robles, M. Michlmayr, J. J. Amor, and D. M. German. Macro-level software evolution: a case study of a large software compilation. Empirical Softw. Engg., 14(3):262-285, 2009.
18. C. Görg and P. Weisgerber. Error detection by refactoring reconstruction. In MSR'05: Proceedings of the 2005 international workshop on Mining software repositories, pages 1-5, New York, NY, USA, 2005. ACM.
19. W. Gropp, E. Lusk, and A. Skjellum. Using MPI: portable parallel programming with the message passing interface. MIT Press, 1999.
20. M. Harman. The current state and future of search based software engineering. In FOSE'07: 2007 Future of Software Engineering, pages 342-357, Washington, DC, USA, 2007. IEEE Computer Society. (Pubitemid 47484936)
21. M. Harman and B. F. Jones. Search-based software engineering. Information and Software Technology, 43(14):833 - 839, 2001.
22. A. E. Hassan. Mining software repositories to assist developers and support managers. PhD thesis, University of Waterloo, 2005.
23. A. E. Hassan. The road ahead for mining software repositories. In FoSM: Frontiers of Software Maintenance, pages 48-57, October 2008.
24. Z. M. Jiang, A. E. Hassan, G. Hamann, and P. Flora. Automatic identification of load testing problems. In ICSM'08: Proceedings of 24th IEEE International Conference on Software Maintenance, pages 307-316, Beijing, China, 2008. IEEE.
25. H. Kagdi, M. L. Collard, and J. I. Maletic. A survey and taxonomy of approaches for mining software repositories in the context of software evolution. J. Softw. Maint. Evol., 19(2):77-131, 2007.
26. T. Kamiya, S. Kusumoto, and K. Inoue. Ccfinder: A multilinguistic token-based code clone detection system for large scale source code. IEEE Transactions on Software Engineering, 28(7):654-670, 2002.
27. C. Kirsopp, M. J. Shepperd, and J. Hart. Search heuristics, case-based reasoning and software project effort prediction. In GECCO'02: Proceedings of the Genetic and Evolutionary Computation Conference, pages 1367-1374, San Francisco, CA, USA, 2002. Morgan Kaufmann Publishers Inc.
28. S. Livieri, Y. Higo, M. Matushita, and K. Inoue. Very-Large Scale Code Clone Analysis and Visualization of Open Source Programs Using Distributed CCFinder: D-CCFinder. In ICSE'07: Proceedings of the 29th International conference on Software Engineering, 2007.
29. W. Maalej and H.-J. Happel. From work to word: How do software developers describe their work? In MSR'09: Proceedings of the 2009 6th IEEE International Working Conference on Mining Software Repositories, pages 121-130, Washington, DC, USA, 2009. IEEE Computer Society.
30. A. Mockus. Amassing and indexing a large sample of version control systems: Towards the census of public source code history. In MSR'09: Proceedings of 6th IEEE International Working Conference on Mining Software Repositories, pages 11-20, 2009.
31. L. Moonen. Generating robust parsers using island grammars. In WCRE'01: Proceedings of the Eighth Working Conference on Reverse Engineering (WCRE'01), page 13, Washington, DC, USA, 2001. IEEE Computer Society.
32. A. P. Nikora and J. C. Munson. Understanding the nature of software evolution. In ICSM'03: Proceedings of the International Conference on Software Maintenance, page 83, Washington, DC, USA, 2003. IEEE Computer Society.
33. C. Olston, B. Reed, U. Srivastava, R. Kumar, and A. Tomkins. Pig latin: a not-so-foreign language for data processing. In SIGMOD'08: Proceedings of the 2008 ACM SIGMOD international conference on Management of data, pages 1099-1110, New York, NY, USA, 2008. ACM.
34. C. K. Roy, J. R. Cordy, and R. Koschke. Comparison and evaluation of code clone detection techniques and tools: A qualitative approach. Sci. Comput. Program., 74(7):470-495, 2009.
35. W. Shang, Z. M. Jiang, B. Adams, and A. E. Hassan. Mapreduce as a general framework to support research in mining software repositories (MSR). In MSR'09: Proceedings of 6th IEEE International Working Conference on Mining Software Repositories, pages 21-30, 2009.
36. Y. Shin, R. Bell, T. Ostrand, and E. Weyuker. Does calling structure information improve the accuracy of fault prediction? In MSR'09: Proceedings of the 2009 6th IEEE International Working Conference on Mining Software Repositories, pages 61-70, Washington, DC, USA, 2009. IEEE Computer Society.
37. M. Stonebraker, D. Abadi, D. J. DeWitt, S. Madden, E. Paulson, A. Pavlo, and A. Rasin. Mapreduce and parallel dbmss: friends or foes? Commun. ACM, 53(1):64-71, 2010.
38. W. Visser, C. S. Pǎsǎreanu, and S. Khurshid. Test input generation with java pathfinder. SIGSOFT Softw. Eng. Notes, 29(4):97-107, 2004.
39. T. White. Hadoop: The Definitive Guide. Oreilly & Associates Inc, 2009.