Development of scientific software: A systematic mapping, a bibliometrics study, and a paper repository

International Journal of Software Engineering and Knowledge Engineering

Volumn 23, Issue 4, 2013, Pages 463-506

  Farhoodi, Roshanak ^a   Garousi, Vahid ^b,c   Pfahl, Dietmar ^b,d   Sillito, Jonathan ^b  

^a PrairieFyre Software Inc   (Canada)

^b UNIVERSITY OF CALGARY   (Canada)

^c MIDDLE EAST TECHNICAL UNIVERSITY   (Turkey)

^d UNIVERSITY OF TARTU   (Estonia)

Author keywords

Bibliometrics study; Development of scientific software; Paper repository; Systematic mapping

Indexed keywords

BIBLIOMETRICS; ENGINEERING TECHNIQUES; ONLINE REPOSITORIES; RESEARCH COMMUNITIES; SCIENTIFIC CONTEXT; SCIENTIFIC SOFTWARES; SOFTWARE ARTIFACTS; SYSTEMATIC MAPPING;

INFORMATION SCIENCE; MAPPING; PROJECT MANAGEMENT; SOFTWARE ENGINEERING;

RESEARCH;

EID: 84880970555     PISSN: 02181940     EISSN: None     Source Type: Journal    
DOI: 10.1142/S0218194013500137     Document Type: Conference Paper

References (151)

1. S. Smith and W. Yu, A document driven methodology for developing a high quality Parallel Mesh Generation Toolbox, Advances in Engineering Software 40(11) (2009) 1155-1167.
2. S. Smith and L. Lai, A new requirements template for scientific computing, in International Workshop on Situational Requirements Engineering Processes - Methods, Techniques and Tools to Support Situation-Specific Requirements Engineering Processes, 2005.
3. D. W. Kane, M. M. Hohman, E. G. Cerami, M. W. Mccormick, K. F. Kuhlmman and J. A. Byrd, Agile methods in biomedical software development: A multi-site experience report, Bioinformatics 7(1) (2006) 273-285.
4. H.-t. Li, An Ontology-based process model for scientific research software development, in International Conference on Software Engineering and Data Mining, 2010, pp. 299-304.
5. W. S. Smith, J. Carette and J. McCutchan, Commonality analysis of families of physical models for use in scientific computing, in International Workshop on Software Engineering for Computational Science and Engineering, 2008.
6. R. Sanders and D. Kelly, Dealing with risk in scientific software development, IEEE Software 25(4) (2008) 21-28.
7. M. A. Heroux, J. M. Willenbring and M. N. Phenow, Improving the development process for CSE software, in International Conference on Parallel, Distributed and Network- Based Processing, 2007.
8. S. Larson and B. Morrison, Managing software requirements in the context of the scienti fic enterprise, in IEEE Aerospace Conference, 2000.
9. C. J. Roy, Practical software engineering strategies for scientific computing, in AIAA Computational Fluid Dynamics Conference, 2009.
10. J. Segal, Some challenges facing software engineers developing software for scientists, in Workshop on Software Engineering for Computational Science and Engineering, 2009.
11. S. Smith, Systematic development of requirements documentation for general purpose scientific computing software, in Proc. of IEEE International Conf. on Requirement Engineering, 2006, pp. 205-215.
12. J. Segal, When software engineers met research scientists: A case study, Empirical Software Engineering 10(1) (2005) 517-536.
13. J. P. Kenny, C. L. Janssen, M. S. Gordon, M. Sosonkina and T. L. Windus, A component approach to collaborative scientific software development: Tools and techniques utilized by the quantum chemistry science application partnership, Scientific Programming 16(4) (2008) 287-296.
14. R. H. Fogh, W. Boucher, W. F. Vranken, A. Pajon, T. J. Stevens, T. N. Bhat, J. Westbrook, J. M. C. Ionides and E. D. Laue, A framework for scientific data modeling and automated software development, Bioinformatics 21(8) (2005) 1678-1684.
15. I. Gorton, D. Chavarría-Miranda, M. Krishnan and J. Nieplocha, A high-performance event service for HPC applications, in International Workshop on Software Engineering for High Performance Computing Applications, 2007.
16. L. Gross, H. Mühlhaus, E. Thorne and K. Steube, A new design of scientific software using python and XML, Pure and Applied Geophysics 165 (2008) 653-670.
17. M. Sala, An object-oriented framework for the development of scalable parallel multilevel preconditioners, ACM Transactions on Mathematical Software 32(3) (2006).
18. F. Hernández, P. Bangalore and K. Reilly, Automating the development of scientific applications using domain-specific modeling, in Proceedings of the Second International Workshop on Software Engineering for High Performance Computing System Applications, 2005.
19. R. C. Gentleman, V. J. Carey, D. M. Bates et al., Bioconductor: Open software development for computational biology and bioinformatics, Genome Biology 5(10) (2004) R80.1-16.
20. R. P. Kendall, D. E. Post and A. Mark, Case study of the NENE code project, IEEE Magazine on Computing in Science and Engineering, 2010.
21. M. C. Smith, R. L. Kelscy, J. M. Young and A. Ginger, Creating a flexible environment for testing scientific software, in Proceedings of SPIE - The International Society for Optical Engineering, 2004, pp. 288-296.
22. T. Cickovski, T. Matthey and J. u. A. Izaguirre, Design patterns for generic objectoriented scientific software, in International Conference on Software Engineering, 2005.
23. D. W. I. Rouson, H. Adalsteinsson and J. Xia, Design patterns for multiphysics modeling in Fortran 2003 and C++, ACM Transactions on Mathematical Software 37(1) (2010).
24. D. C. Arnold and J. J. Dongarra, Developing an architecture to support the implementation and development of scientific computing applications, in Working Conference on the Architecture of Scientific Software, 2000, pp. 39-55.
25. D. Woollard, C. Mattmann and N. Medvidovic, Injecting software architectural constraints into legacy scientific applications, in Workshop on Software Engineering for Computational Science and Engineering, 2009, pp. 65-71.
26. A. Sharma and S. S. Sarangdevot, Investigating the application of AOP methodology in development of bioinformatics software using Eclipse-AJDT environment, in International Conference & Workshop on Emerging Trends in Technology, 2011.
27. D. E. Bernholdt, R. C. Armstrong and B. A. Allan, Managing complexity in modern high end scientific computing through component-based software engineering, in Workshop on Productivity and Performance in High-End Computing, 2004.
28. B. A. Allan, B. Norris, W. R. Elwasif and R. C. Armstrong, Managing scientific software complexity with Bocca and CCA, Scientific Programming 16(4) (2008) 315-327.
29. Donna Kaminskyj Long, Kiemele Liam, Celina Gibbs, Andrew Brownsword and Y. Coady, Mind the Gap! Bridging the dichotomy of design and implementation, in International Workshop on Software Engineering for Computational Science and Engineering, 2011.
30. H. P. L. Are Magnus Bruaset, Object-oriented design of preconditioned iterative methods in diffpack, ACM Transactions on Mathematical Software, 1997.
31. M. Urbano Cuadrado, M. D. Luque de Castro and M. A. Gómez-Nieto, Object-oriented techniques for design and development of standard software solutions in automation and data management in analytical chemistry, Trends in Analytical Chemistry, 2006.
32. M. Sala, K. S. Stanley and M. A. Heroux, On the design of interfaces to sparse direct solvers, ACM Transanctions on Mathematical Software 34(2) (2008).
33. K. Morris, D. W. I. Rouson and J. Xia, On the object-oriented design of referencecounted shadow objects, in International Workshop on Software Engineering for Computational Science and Engineering, 2011.
34. A.-S. S. Hai-Tao Li and Y.-X. Meng, Parallel development and independent integration model for scientific research software engineering, in International Conference on Computer Science and Software Engineering, 2008.
35. C. Blilie, Patterns in scientific software: An introduction, Computing in Science and Engineering 4(3) (2002) 48-53.
36. R. Neely, Practical software quality engineering on a large multiDisciplinary HPC development team, in International Workshop on Software Engineering for High Performance Computing Applications, 2004.
37. Rune Møllegaard Friborg and B. Vinter, Rapid development of scalable scientific software using a process oriented approach, Journal of Computational Science, 2011.
38. X. Jiao, M. T. Campbell and M. T. Heath, Roccom: An object-oriented, data-centric software integration framework for multiphysics simulations, in International Conference on Supercomputing, 2003, pp. 358-368.
39. G. Rodríguez-Gómez, J. Muñoz-Arteaga and R. Fernández-Del-Busto, Scientific software design through scientific computing patterns, in International Conference on Modelling, Simulation, and Optimization, 2004.
40. B. Norris, Software architecture issues in scientific component development, in International Conference on Applied Parallel Computing, 2004.
41. D. Woollard, T. Berger and W. Yamada, Software engineering for neural dynamics simulations: A case study, in International Workshop on Software Engineering for High Performance Computing Applications, 2004.
42. T. Q. Andrea Lani, Dries Kimpe, Herman Deconinck, Stefan Vandewalle and Stefaan Poedts, The COOLFluiD framework: Design solutions for high performance object oriented scientific computing software, in International Conference on Computational Science, 2005.
43. P. V. D. Velde and G. D. Mallinson, The design of a component-oriented framework for numerical simulation software, Advances in Engineering Software, 2007.
44. S. M. Gallo, R. Miller and C. M. Weeks, The design of a portable scientific tool: A case study using SnB, in Proceedings of the ACM/IEEE Conference on Supercomputing, 1996.
45. R. Armstrong, D. Gannon, A. Geist, K. Keahey, S. Kohn, L. McInnes, S. Parker and B. Smolinski, Toward a common component architecture for high-performance scientific computing, in IEEE International Symposium on High Performance Distributed Computing, 1999, pp. 13-24.
46. G. Scherp and W. Hasselbring, Towards a model-driven transformation framework for scientific workflows, in Procedia Computer Science (Conference Proceedings), 2010.
47. D. W. I. Rouson, Towards analysis-driven scientific software architecture: The case for abstract data type calculus, Scientific Programming, 2008.
48. C. Macaulay, D. Sloan, X. Jiang, P. Forbes, S. Loynton, J. R. Swedlow and P. Gregor, Usability and user-centered design in scientific software development, IEEE Software 26 (1) (2009) 96-102.
49. I. Gorton, C. Sivaramakrishnan, S. White, S. Purohit, M. Madison and K. Schuchardt, Velo: Riding the knowledge management wave for simulation and modeling, in International Workshop on Software Engineering for Computational Science and Engineering, 2011.
50. L. Hochstein and V. Basili, A case study of large-scale parallel scientific code development, IEEE Computer, 2008.
51. O. Marques and T. Drummond, Building a software infrastructure for computational science applications: Lessons and solutions, in Proceedings of the Second International Workshop on Software Engineering for High Performance Computing System Applications, 2005.
52. P. K. Chilana, C. L. P and A. J. Ko, Comparing bioinformatics software development by computer scientists and biologists: An exploratory study, in Workshop on Software Engineering for Computational Science and Engineering, 2009.
53. J. R. Cary, S. G. Shasharina, J. C. Cummings, J. V. W. Reynders and P. J. Hinker, Comparison of C++ and Fortran 90 for object-oriented scientific programming, Computer Physics Communications 105(1) (1997) 20-36.
54. B. J. Henz and D. R. Shires, Coupling scientific applications within an object-oriented programming framework: Improving development time and software quality, in International Conference on Parallel and Distributed Processing Techniques and Applications, 2004.
55. D. Hudak, N. Lubdan, V. Gadepally and A. Krishnamurthy, Developing a computational science IDE for HPC systems, in International Workshop on Software Engineering for High Performance Computing Applications, 2007.
56. R. Arora, P. Bangalore and M. Mernik, Developing scientific applications using generative programming, in Workshop on Software Engineering for Computational Science and Engineering, 2009, pp. 51-58.
57. R. Kendall, J. C. Carver, D. Fisher, D. Henderson, A. Mark, D. Post, C. E. R. Jr and S. Squires, Development of a weather forecasting code: A case study, IEEE Software 25(4) (2008) 59-65.
58. M. Hatlo, F. James, P. Mato, L. Moneta, M. Winkler and A. Zsenei, Developments of mathematical software libraries for the LHC experiments, IEEE Transactions on Nuclear Science, 2005.
59. S. M. Easterbrook and T. C. Johns, Engineering the software for understanding climate change, Computing in Science and Engineering 11(6) (2009) 65-74.
60. D. M. Beazley and P. S. Lomdahl, Feeding a large-scale physics application to Python, in International Python Conference, 1997, pp. 21-28.
61. D. Kelly, D. Hook and R. Sanders, Five recommended practices for computational scientists who write software, Computing in Science and Engineering 11(5) (2009) 48-53.
62. E. V. Wyk, M. Heimdahl and Y. Saad, From models to effcient code: It's all in the middle, in International Workshop on Software Engineering for High Performance Computing Applications, 2004.
63. J. E. Hannay, C. MacLeod, J. Singer, H. P. Langtangen, D. Pfahl and G. Wilson, How do scientists develop and use scientific software? in Workshop on Software Engineering for Computational Science and Engineering, 2009, pp. 1-8.
64. M. Umarji and C. Seaman, Informing design of a search tool for bioinformatics, in International Workshop on Software Engineering for Computational Science and Engineering, 2008.
65. V. Basili, T. Craveiro, D. Cruzes, K. Despain, B. Dorland, L. Hochstein, N. Zazworka and M. Zelkowitz, Large effcient table-top teraflop computing, in International Workshop on Software Engineering for Computational Science and Engineering, 2008.
66. P. F. Duboi, Maintaining correctness in scientific programs, IEEE Magazine on Computing in Science and Engineering, 2005.
67. T. E. Oliphant, Python for scientific computing, IEEE Magazine on Computing in Science and Engineering, 2007.
68. J. J. Marshall, R. R. Downs and S. Samadi, Relevance of software reuse in building advanced scientific data processing systems, Earth Science Informatics, 2010.
69. W. Yu and S. Smith, Reusability of FEA software: A program family approach, in Workshop on Software Engineering for Computational Science and Engineering, 2009, pp. 43-50.
70. Y. K. G. Iwai, T. Saski and Y. Watse, SAGA-based user environment for distributed computing resources: A universal Grid solution over multi-middleware infrastructures, in Procedia Computer Science (Conference Proceedings), 2010.
71. S. Faulk, E. Loh, Michael L. Van De Vanter, S. Squires and L. G. Votta, Scientific computing's productivity gridlock: How software engineering can help, IEEE Magazine on Computing in Science and Engineering, 2009.
72. D. Woollard, N. Medvidovic, Y. Gil and C. A. Mattmann, Scientific software as work- flows: From discovery to distribution, IEEE Software 25(4) (2008) 37-43.
73. K. S. Ackroyd, S. H. Kinder, G. R. Mant, M. C. Miller, C. A. Ramsdale and P. C. Stephenson, Scientific software development at a research facility, IEEE Software 25(4) (2008) 44-51.
74. P. Luksch, U. Maier, S. Rathmayer, A. M. Weidmann and F. Unger, Sempa: Software engineering for parallel scientific computing, IEEE Concurreny, 1997.
75. D. S. Mark Vigder, Janice Singer, Software automation in scientific research organizations, in International Workshop on Software Engineering for Computational Science and Engineering, 2008.
76. M. Erwig and Z. Fu, Software reuse for scientific computing through program generation, ACM Transactions on Software Engineering and Methodology 14(2) (2005) 168-198.
77. A. Dall'Osso, Using computer algebra systems in the development of scientific computer codes, Future Generation Computer Systems 19(2) (2003) 143-160.
78. T. L. Dahlgren and P. T. Devanbu, Adaptable assertion checking for scientific software components, in International Workshop on Software Engineering for High Performance Computing System Applications, 2004, pp. 64-69.
79. H. Naguib and Y. Li, Applying software engineering methods and tools to CSE research projects, in International Conference on Computational Science, 2010.
80. D. Kelly and R. Sanders, Assessing the quality of scientific software, in Workshop on Software Engineering for Computational Science & Engineering, 2008.
81. D. Post, R. Kendall and W. Earl, Case study of the falcon code project, in Proceedings of the Second International Workshop on Software Engineering for High Performance Computing System Applications, 2005.
82. M. G. Cox and P. M. Harris, Design and use of reference data sets for testing scientific software, Analytica Chimica Acta 380(2-3) (1999) 339-351.
83. R. G. Diane Kelly, Yizhen Shao, Examining random and designed tests to detect code mistakes in scientific software, Journal of Computational Science, 2011.
84. G. Avrunin, S. Siegel and A. Siegel, Finite-state verification for high performance computing, in International Workshop on Software Engineering for High Performance Computing Applications, 2005.
85. R. Palmer, S. Barrus, Y. Yang, G. Gopalakrishnan and R. M. Kirby, Gauss: A framework for verifying scientific computing software, in Proceedings of the Workshop on Software Model Checking, 2005.
86. S. Asgari, L. Hochstein, V. Basili, J. Carver, J. Hollingsworth and F. Shull, Generating testable hypotheses from tacit knowledge for high productivity computing, in International Workshop on Software Engineering for High Performance Computing Applications, 2005.
87. M. Heroux, Improving CSE software through reproducibility requirements, in International Workshop on Software Engineering for Computational Science and Engineering, 2011.
88. T. L. Dahlgren and P. T. Devanbu, Improving scientific software component quality through assertions, in International Conference on Software Engineering, 2005.
89. R. Graya and D. Kelly, Investigating test selection techniques for scientific software using Hook's mutation sensitivity testing, in International Conference on Computational Science, 2010.
90. S. A. Vilkomir, W. T. Swain, J. H. Poore and K. T. Clarno, Modeling input space for testing scientific computational software: A case study, in International Conference on Computational Science, 2008, pp. 291-300.
91. D. Hook and D. Kelly, Mutation sensitivity testing, IEEE Design & Test 11(6) (2009) 40-47.
92. D. Kelly, S. Thorsteinson and D. Hook, Scientific software testing: Analysis with four dimensions, IEEE Software, 2011.
93. J. C. Carver, R. P. Kendall, S. E. Squires and D. E. Post, Software development environments for scientific and engineering software: A series of case studies, in International conference on Software Engineering, 2007, pp. 550-559.
94. D. E. Post and R. P. Kendall, Software project management and quality engineering practices for complex, coupled multiphysics, massively parallel computational simulations: Lessons learned from ASCI, International Journal of High Performance Computing Applications, 2004.
95. C. Morris, Some lessons learned reviewing scientific code, in International Workshop on Software Engineering for Computational Science and Engineering, 2008.
96. J. Segal, Some problems of professional end user developers, in IEEE Symposium on Visual Languages and Human-Centric Computing, 2007, pp. 111-118.
97. H. Remmel, B. Paech, C. Engwer and P. Bastian, Supporting the testing of scientific frameworks with software product line engineering - A proposed approach, in International Workshop on Software Engineering for Computational Science and Engineering, 2011.
98. D. Hook and D. Kelly, Testing for trustworthiness in scientific software, in Workshop on Software Engineering for Computational Science and Engineering, 2009, pp. 59-64.
99. R. Sanders and D. Kelly, The challenge of testing scientific software, in Conference for the Association for Software Testing, 2008.
100. L. Hatton, The T-Experiments: Errors in scientific software, IEEE Computational Science & Engineering 4(2) (1997) 27-38.
101. S. Booth and D. Henty, Verification strategies for high performance computing software, in International Workshop on Software Engineering for High Performance Computing Applications, 2004.
102. E. Loh, M. L. V. D. Vanter and L. G. Votta, Can software engineering solve the HPCS problem? in Proceedings of the Second International Workshop on Software Engineering for High Performance Computing System Applications, 2005.
103. D. Kelly, Determining factors that affect long-term evolution in scientific application software, Journal of Systems and Software 82(5) (2009) 851-861.
104. M. Wagner, Evolution from a scientific application to an applicable product, in European Conference on Software Maintenance and Reengineering, 2007.
105. Y. Li, Reengineering a scientific software and lessons learned, in International Workshop on Software Engineering for Computational Science and Engineering, 2011.
106. Jeffrey Overbey, Spiros Xanthos, Ralph Johnson and B. Foote, Refactorings for Fortran and high-performance computing, in International Workshop on Software Engineering for High Performance Computing Applications, 2005.
107. J. Pitt-Francis, M. O. Bernabeu, J. Cooper, A. Garny, L. Momtahan, J. Osborne, P. Pathmanathan, B. Rodriguez, J. P. Whiteley and D. J. Gavaghan, Chaste: Using agile programming techniques to develop computational biology software, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences 366(1878) (2008) 3111-3136.
108. W. A. Wood and W. L. Kleb, Exploring XP for scientific research, IEEE Software 20(3) (2003) 30-36.
109. D. Kane, Introducing agile development into bioinformatics: An experience report, in Proceedings of the Conference on Agile Development, 2003, pp. 132-140.
110. J. Segal, Models of scientific software development, in International Workshop on Software Engineering for Computational Science and Engineering, 2008.
111. J. Segal, Software development cultures and cooperation problems: A field study of the early stages of development of software for a scientific community, Computer Supported Cooperative Work 18(5-6) (2009) 581-606.
112. D. E. Post, The challenge for computational science, in International Workshop on Software Engineering for High Performance Computing Applications, 2004.
113. J. Willenbring, M. Heroux and R. Heaphy, The Trilinos software lifecycle model, in International Workshop on Software Engineering for High Performance Computing Applications, 2007.
114. M.-S. Wu, J. L. Bentz, F. Peng, M. Sosonkina, M. S. Gordon and R. A. Kendall, Integrating performance tools with large-scale scientific software, in IEEE International Parallel and Distributed Processing Symposium, 2007, pp. 1-8.
115. M. Vigder, End-user software development in a scientific organization, in ICSE Workshop on Software Engineering Foundations for End User Programming, 2009.
116. T. Epperly, S. R. Kohn and G. Kumfert, Component technology for high-performance sceintific simulation software, in Working Conference on the Architecture of Scientific Software, 2000, pp. 69-86.
117. J. P. Kenny, S. J. Benson, Y. Alexeev, J. Sarich, C. L. Janssen, L. C. Mcinnes, M. Krishnan, J. Nieplocha, E. Jurrus, C. Fahlstrom and T. L. Windus, Component-based integration of chemistry and optimization software, Journal of Computational Chemistry 25(14) (2004) 1717-1725.
118. V. Maxville, Preparing scientists for scalable software development, in Workshop on Software Engineering for Computational Science and Engineering, 2009.
119. J. Howison and J. D. Herbsleb, Scientific software production: Incentives and collaboration, in Proceedings of the ACM Conference on Computer Supported Cooperative Work, 2011.
120. S. Samadi, N. Almaeh, R. Wolfe, S. Olding and D. Isaac, Strategies for enabling software reuse within the earth science community, in International Geoscience and Remote Sensing Symposium, 2004.
121. C. Halverson, C. Swart, J. Brezin, J. Richards and C. Danis, Towards an ecologically valid study of programmer behavior for scientific computing, in International Workshop on Software Engineering for Computational Science and Engineering, 2008.
122. V. R. Basili, D. Cruzes, J. C. Carver, L. M. Hochstein, J. K. Hollingsworth, M. V. Zelkowitz and F. Shull, Understanding the high-performance-computing community: A software engineer's perspective, IEEE Software 25(4) (2008) 29-36.
123. J. F. Cremer, R. S. Palmer and R. E. Zippel, Creating scientific software, Transactions of the Society for Computer Simulation International 14(1) (1997) 37-49.
124. S. M. Baxter, S. W. Day, J. S. Fetrow and S. J. Reisinger, Scientific software development is not an oxymoron, PLoS Computational Biology, 2006.
125. J. Segal, Scientists and software engineers: A tale of two cultures, in Proceedings of Annual Meeting of the Psychology of Programming Interest Group, 2008.
126. R. Baxter, Software engineering is software engineering, in International Workshop on Software Engineering for High Performance Computing Applications, 2004.
127. Y. Alexeev, B. A. Allan, R. C. Armstrong and D. E. Bernholdt, Component-based software for high-performance scientific computing, Journal of Physics 16, 2005.
128. R. A. Bartlett, Integration strategies for computational science & engineering software, in Workshop on Software Engineering for Computational Science and Engineering, 2009, pp. 35-42.
129. C. Morris and J. Segal, Some challenges facing scientific software developers: The case of molecular biology, in IEEE International Conference on e-Science, 2009.
130. L. Nguyen-Hoan, S. Flint and R. Sankaranarayana, A survey of scientific software development, in International Symposium on Empirical Software Engineering and Measurement, 2010. Other studies
131. D. F. Kelly, A software chasm: Software engineering and scientific computing, IEEE Software 24(6) (2007) 118-119.
132. V. K. Decyk, C. D. Norton and H. J. Gardner, Why Fortran? Computing in Science and Engineering 9(4) (2007) 68-71.
133. R. L. Glass and T. Y. Chen, An assessment of systems and software engineering scholars and institutions (1998-2002), Journal of Systems and Software 68(1) (2003) 77-84.
134. R. L. Glass and T. Y. Chen, An assessment of systems and software engineering scholars and institutions (1999-2003), Journal of Systems and Software 76(1) (2005) 91-97.
135. V. Garousi and T. Varma, A bibliometric assessment of Canadian software engineering scholars and institutions (1996-2006), Canadian Journal on Computer and Information Science 3(2) (2010) 19-29.
136. V. Garousi and T. Varma, A bibliometrics analysis of Canadian electrical and computer engineering institutions (1996-2006) based on IEEE journal publications, Canadian Journal on Computer and Information Science 5(4) (2012) 1-24.
137. D. Budgen, M. Turner, P. Brereton and B. Kitchenham, Using mapping studies in software engineering, in Proceedings of Annual Workshop of the Psychology of Programming Interest Group, 2008, pp. 195-204.
138. K. Petersen, R. Feldt, S. Mujtaba and M. Mattsson, Systematic mapping studies in software engineering, in International Conference on Evaluation and Assessment in Software Engineering, 2008.
139. M. T. Sletholt, J. E. Hannay, D. Pfahl, H. C. Benestad and H. P. Langtangen, A literature review of agile practices and their effects in scientific software development, in International Workshop on Software Engineering for Computational Science and Engineering, 2011.
140. B. Kitchenham and S. Charters, Guidelines for performing systematic literature reviews in software engineering, in Evidence-Based Software Engineering, 2007.
141. V. Garousi, Classification and trend analysis of UML books (1997-2009), Journal on Software & System Modeling (SoSyM), to Appear.
142. F. Elberzhager, J. Münch and V. T. N. Nha, A systematic mapping study on the combination of static and dynamic quality assurance techniques, Information and Software Technology 54 (2012) 1-15.
143. S. Ali, L. Briand, H. Hemmati and R. K. Panesar-Walawege, A systematic review of the application and empirical investigation of search-based test-case generation, IEEE Transactions on Software Engineering 36(6) (2010) 742-762.
144. V. R. Basili, Software modeling and measurement: The Goal/Question/Metric paradigm, Technical report, University of Maryland at College Park 1992.
145. R. Farhoodi, V. Garousi, D. Pfahl and J. Sillito, Systematic mapping of developing scientific software, online repository on Google Docs, http://www.softqual.ucalgary.ca/projects/2012/SM CSE/, Last accessed: April 2012.
146. P. Runeson, A. Rainer, M. Host and B. Regnell, Case Study Research in Software Engineering: Guidelines and Examples (John Wiley & Sons, 2012).
147. B. D. Nicola, Bibliometrics and Citation Analysis: From the Science Citation Index to Cybermetrics: (Scarecrow Press, 2009).
148. R. Gentleman, R Programming for Bioinformatics (CRC Press, 2009).
149. Y. Liu and A. Milanova, Static analysis for inference of explicit information flow, in Proceedings of ACM Workshop on Program Analysis for Software Tools and Engineering, 2008.
150. R. Kendall, D. Fisher, D. Henderson, J. C. Carver, A. Mark, D. Post, C. E. Rhoades, Jr., and S. Squires, Development of a weather forecasting code: A case study, IEEE Software 25(4) (2008) 59-65.
151. T. D. Cook and D. T. Campbell, Quasi-Experimentation Design and Analysis Issues for Field Settings (Houghton Miffn, 1979).