Sourcerer: An infrastructure for large-scale collection and analysis of open-source code

Science of Computer Programming

Volumn 79, Issue , 2014, Pages 241-259

  Bajracharya, Sushil ^a   Ossher, Joel ^a   Lopes, Cristina ^a  

^a UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Data mining; Internet scale code retrieval; Open source; Software information retrieval; Sourcerer; Static analysis

Indexed keywords

CODE RETRIEVALS; OPEN SOURCES; SOFTWARE ENGINEERING TOOLS; SOFTWARE INFORMATION RETRIEVALS; SOURCERER; STRUCTURAL INFORMATION; TEXTUAL INFORMATION; TOOLS AND TECHNIQUES;

DATA MINING; OPEN SYSTEMS; SEARCH ENGINES; SOFTWARE ENGINEERING; STATIC ANALYSIS; TOOLS;

COMPUTER PROGRAMMING LANGUAGES;

EID: 84885676163     PISSN: 01676423     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.scico.2012.04.008     Document Type: Article

References (68)

1. Sourcerer wiki page on sourcerer api search tool http://wiki.github.com/ sourcerer/Sourcerer/sas
2. Web site for apache software foundation http://apache.org
3. Web site for Java.net http://java.net
4. Web site for Tigris http://tgris.org
5. Black Duck's web page with Koders usage information, February 2010. http://corp.koders.com/about/.
6. Lucene web site, Jan 2010. http://lucene.apache.org.
7. Web page for codegenie, July 2010. http://sourcerer.ics.uci.edu/ codegenie.
8. Web page for Sourcerer's github repository, June 2010. http://github.com/sourcerer/Sourcerer.
9. Web Page on Apache Lucene Scoring, March 2010. http://lucene.apache.org/ java/2-4-0/scoring.html.
10. Web site for google code search, July 2010. http://www.google.com/ codesearch.
11. Web site for maven, July 2010. http://maven.apache.org.
12. Web site for maven's central repository, July 2010. http://repo1.maven. org/maven2/.
13. Web site for merobase, July 2010. http://www.merobase.com/.
14. Web site for solr, July 2010. http://lucene.apache.org/solr/.
15. Web page for sourcerer project and the sourcerer code search engine, July. http://sourcerer.ics.uci.edu.
16. S. Bajracharya, T. Ngo, E. Linstead, Y. Dou, P. Rigor, P. Baldi, and C. Lopes Sourcerer: A Search Engine for Open Source Code Supporting Structure-Based Search 2006 ACM Press New York, NY, USA pp. 681-682
17. S. Bajracharya, T. Ngo, E. Linstead, P. Rigor, Y. Dou, P. Baldi, C. Lopes, A study of ranking schemes in internet-scale code search. Technical Report UCI-ISR-07-8, UCI ISR, November 2007.
18. S. Bajracharya, J. Ossher, and C. Lopes Sourcerer: an internet-scale software repository Proceedings of the 2009 ICSE Workshop on Search-Driven Development-Users, Infrastructure, Tools and Evaluation 2009 IEEE Computer Society 1 4
19. S. Bajracharya, J. Ossher, C. Lopes, Leveraging usage similarity for effective retrieval of examples in code repositories, in: 18th International Symposium on the Foundations of Software Engineering, 2010.
20. S. Bajracharya, J. Ossher, C. Lopes, Searching api usage examples in code repositories with sourcerer api search, in: SUITE 2010: Second International Workshop on Search-driven Development - Users, Infrastructure, Tools and Evaluation, 2010.
21. P.F. Baldi, C.V. Lopes, E.J. Linstead, and S.K. Bajracharya A theory of aspects as latent topics Proceedings of the 23rd ACM SIGPLAN Conference on Object Oriented Programming Systems Languages and Applications 2008 ACM Nashville, TN, USA 543 562
22. J. Brandt, P.J. Guo, J. Lewenstein, M. Dontcheva, and S.R. Klemmer Two studies of opportunistic programming: interleaving web foraging, learning, and writing code Proceedings of the 27th International Conference on Human Factors in Computing Systems 2009 ACM Boston, MA, USA 1589 1598
23. M. Bruch, M. Monperrus, and M. Mezini Learning from examples to improve code completion systems Proceegings of FSE 2009 ACM Amsterdam, The Netherlands 213 222
24. S. Chatterjee, S. Juvekar, and K. Sen SNIFF: a search engine for java using free-form queries Fundamental Approaches to Software Engineering 2009 385 400
25. Y. Chen, E.R. Gansner, and E. Koutsofios A c++ data model supporting reachability analysis and dead code detection IEEE Transactions on Software Engineering 24 9 1998 682 694
26. D. Cubranic, G. Murphy, J. Singer, and K. Booth Hipikat: a project memory for software development Software Engineering, IEEE Transactions on 31 6 2005 446 465
27. S. Demeyer, S. Tichelaar, S. Ducasse, Famix 2.1 - the famoos information exchange model. Technical report, University of Bern, 2001.
28. A. Deshpande, and D. Riehle The total growth of open source Fourth Conference on Open Source Systems 2008 Springer Verlag
29. M. Eichberg, M. Mezini, K. Ostermann, and T. Schafer Xirc: a kernel for cross-artifact information engineering in software development environments WCRE'04: Proceedings of the 11th Working Conference on Reverse Engineering, WCRE'04 2004 IEEE Computer Society Washington, DC, USA 182 191
30. D. German, J. Gonzalez-Barahona, G. Robles, A model to understand the building and running inter-dependencies of software, in: Proc. 14th Working Conf. on Reverse Engineering, 2007, pp. 130-139.
31. J. Gosling, B. Joy, G. Steele, and G. Bracha Java(TM) Language Specification, The 3 edition 2005 Addison Wesley
32. G. Gousios, and D. Spinellis Alitheia Core: an extensible software quality monitoring platform Proceedings of the 31st International Conference on Software Engineering 2009 IEEE Computer Society 579 582
33. M. Grechanik, K.M. Conroy, and K.A. Probst Finding relevant applications for prototyping Proceedings of the Fourth International Workshop on Mining Software Repositories 2007 IEEE Computer Society 12
34. J. Hammond, What developers think, Jan 2010. http://www.drdobbs.com/ architect/222301141.
35. R. Hoffmann, J. Fogarty, and D.S. Weld Assieme: finding and leveraging implicit references in a web search interface for programmers Proceedings of the 20th Annual ACM Symposium on User Interface Software and Technology 2007 ACM Newport, Rhode Island, USA 13 22
36. R. Holmes, and G.C. Murphy Using structural context to recommend source code examples ICSE'05: Proceedings of the 27th International Conference on Software Engineering 2005 ACM Press New York, NY, USA 117 125
37. R. Holmes, and R.J. Walker Lightweight, semi-automated enactment of pragmatic-reuse plans Proceedings of the 10th International Conference on Software Reuse: High Confidence Software Reuse in Large Systems 2008 Springer-Verlag Beijing, China 330 342
38. J. Howison, M. Conklin, and K. Crowston FLOSSmole: a collaborative repository for FLOSS research data and analyses International Journal of Information Technology and Web Engineering 1 3 2006 17 26
39. Sourcerer web services http://sourcerer.ics.uci.edu/services
40. O. Hummel, W. Janjic, and C. Atkinson Code conjurer: pulling reusable software out of thin air IEEE Software 25 5 2008 45 52
41. K. Inoue, R. Yokomori, H. Fujiwara, T. Yamamoto, M. Matsushita, and S. Kusumoto Component rank: relative significance rank for software component search ICSE'03: Proceedings of the 25th International Conference on Software Engineering 2003 IEEE Computer Society Washington, DC, USA 14 24
42. R.M. Lawrence Page, Sergey Brin, T. Winograd, The pagerank citation ranking: bringing order to the web. Stanford Digital Library working paper SIDL-WP-1999-0120 of 11/11/1999 (see: http://dbpubs.stanford.edu/pub/1999-66).
43. O. Lemos, S.K. Bajracharya, J. Ossher, P.C. Masiero, C. Lopes, Applying test-driven code search to the reuse of auxiliary functionality, in: 24th Annual ACM Symposium on Applied Computing SAC 2009, 2009.
44. O.A.L. Lemos, S.K. Bajracharya, and J. Ossher CodeGenie:: a tool for test-driven source code search Companion to the 22nd ACM SIGPLAN Conference on Object-oriented Programming Systems and Applications Companion 2007 ACM Montreal, Quebec, Canada 917 918
45. O.A.L. Lemos, S.K. Bajracharya, J. Ossher, P.C. Masiero, and C.V. Lopes A test-driven approach to code search and its application to the reuse of auxiliary functionality Information and Software Technology 53 4 2011 294 306
46. O.A.L. Lemos, S.K. Bajracharya, J. Ossher, R.S. Morla, P.C. Masiero, P. Baldi, and C.V. Lopes CodeGenie: using test-cases to search and reuse source code Proceedings of the twenty-second IEEE/ACM international conference on Automated software engineering 2007 ACM Atlanta, Georgia, USA 525 526
47. E. Linstead, S. Bajracharya, T. Ngo, P. Rigor, C. Lopes, and P. Baldi Sourcerer: mining and searching internet-scale software repositories Data Mining and Knowledge Discovery 18 2 2009 300 336
48. E. Linstead, P. Rigor, S. Bajracharya, C. Lopes, and P. Baldi Mining concepts from code with probabilistic topic models Proceedings of the Twenty-Second IEEE/ACM International Conference on Automated Software Engineering 2007 ACM Atlanta, Georgia, USA 461 464
49. E. Linstead, P. Rigor, S. Bajracharya, C. Lopes, and P. Baldi Mining eclipse developer contributions via author-topic models Proceedings of the Fourth International Workshop on Mining Software Repositories 2007 IEEE Computer Society 30
50. E. Linstead, P. Rigor, S. Bajracharya, C. Lopes, and P. Baldi Mining internet-scale software repositories J. Platt, D. Koller, Y. Singer, S. Roweis, Advances in Neural Information Processing Systems 20 2008 MIT Press Cambridge, MA 929 936
51. M.A. Linton, Queries and views of programs using a relational database system, Technical Report UCB/CSD-83-164, EECS Department, University of California, Berkeley, 1983.
52. M.A. Linton Implementing relational views of programs SIGPLAN Not. 1984 ACM Press New York, NY, USA 132 140
53. C.V. Lopes, S.K. Bajracharya, J. Ossher, P.F. Baldi, UCI source code data sets, Irvine, CA: University of California, Bren School of Information and Computer Sciences http://www.ics.uci.edu/-lopes/datasets/.
54. Lucid imagination, Lucidworks for solr certified distribution reference guide, 2010. http://www.lucidimagination.com/Downloads/LucidWorks-for-Solr/ Reference-Guide.
55. M. Lungu, M. Lanza, T. Gîrba, and R. Robbes The small project observatory: visualizing software ecosystems Science of Computer Programming 75 4 2010 264 275
56. D. Mandelin, L. Xu, R. Bodík, and D. Kimelman Jungloid mining: helping to navigate the api jungle Proceedings of the 2005 ACM SIGPLAN Conference on Programming Language Design and Implementation 2005 ACM Press New York, NY, USA 48 61
57. C.D. Manning, P. Raghavan, and H. Schütze Introduction to Information Retrieval 1 edition 2008 Cambridge University Press
58. M. McCandless, E. Hatcher, and O. Gospodnetic Lucene in Action 2 edition 2010 Manning Publications
59. A. Michail Code web: data mining library reuse patterns Proceedings of the 23rd International Conference on Software Engineering 2001 IEEE Computer Society Toronto, Ontario, Canada 827 828
60. J. Ossher, S. Bajracharya, E. Linstead, P. Baldi, C. Lopes, SourcererDB: an aggregated repository of statically analyzed and cross-linked open source java projects, in: Mining Software Repositories, 2009. MSR'09. 6th IEEE International Working Conference on, 2009, pp. 183-186.
61. J. Ossher, S. Bajracharya, C. Lopes, Automated dependency resolution for open source software, in: 2010 7th IEEE Working Conference on Mining Software Repositories, MSR 2010, Cape Town, South Africa, 2010, pp. 130-140.
62. N. Sahavechaphan, and K. Claypool Xsnippet: mining for sample code OOPSLA'06: Proceedings of the 21st Annual ACM SIGPLAN Conference on Object-Oriented Programming Systems, Languages, and Applications 2006 ACM Press New York, NY, USA 413 430
63. S. Thummalapenta, and T. Xie Parseweb: a programmer assistant for reusing open source code on the web Proceedings of The Twenty-Second IEEE/ACM International Conference on Automated Software Engineering 2007 ACM Atlanta, Georgia, USA 204 213
64. S. Thummalapenta, T. Xie, SpotWeb: detecting framework hotspots and coldspots via mining open source code on the web, in: Automated Software Engineering, 2008. ASE 2008. 23rd IEEE/ACM International Conference on, 2008, pp. 327-336.
65. Web site for google code hosting http://code.google.com/projecthosting
66. Web site for Koders, 2010. http://www.koders.com.
67. Web site for Krugle, 2010. http://www.krugle.com.
68. Web site for sourceforge http://sourceforge.net