Evaluation of large research initiatives: Outcomes, challenges, and methodological considerations

New Directions for Evaluation

Volumn 2008, Issue 118, 2008, Pages 61-72

  Quinlan, Kathleen M ^a   Kane, Mary ^a   Trochim, William M K ^b  

^a System Concepts   (United States)

^b Department of Molecular Biology and Genetics   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 69649107815     PISSN: 10976736     EISSN: 1534875X     Source Type: Journal    
DOI: 10.1002/ev.261     Document Type: Article

References (40)

1. Ailes, C., Roessner, D., & Feller, I. (1997). The impact on industry of interaction with engineering research centers (Final report prepared for the National Science Foundation). Arlington, VA: SRI International.
2. Retrieved August 8, 2006, from http://www.sri.com/policy/csted/reports/sandt/erc/
3. Andersen, L., Gwaltney, M., Sundra, D., Brownson, R., Kane, M., Cross, A., et al. (2006). Using concept mapping to develop a logic model for the prevention research centers program. Preventing Chronic Disease, 3(1).
4. Retrieved April 3, 2007, from www.cdc.gov/pcd/issues/2006/jan/05_0153.htm
5. Brainard, J. (2002a, June 21). New director discusses NIH's massive budget. Chronicle of Higher Education.
6. Retrieved June 13, 2007, from http://chronicle.com/weekly/v48/i41/41a02503.htm
7. Brainard, J. (2002b, March 29). New science measures related by OMB. Chronicle of Higher Education.
8. Retrieved June 13, 2007, from http://chronicle.com/weekly/v48/i29/29a02502.htm
9. Committee on Facilitating Interdisciplinary Research, National Academy of Sciences, National Academy of Engineering, Institute of Medicine, & Committee on Science, Engineering, and Public Policy. (2004). Facilitating interdisciplinary research. Washington, DC: National Academies Press.
10. Committee on Science, Engineering, and Public Policy, National Academy of Sciences, National Academy of Engineering, & Institute of Medicine. (2000). Experiments in international benchmarking of US research fields. Washington, DC: National Academies Press.
11. Concept Systems. (2007). Development of an evaluation framework for DMID's regional centers of excellence for biodefense and emerging infectious diseases program: Concept mapping summary report. Ithaca, NY: Concept Systems.
12. Corley, E., Melkers, J., & Johns, K. (2006, May). Layered and evolving networks: Innovative evaluation methods for interdisciplinary research in university-based research centers. Paper presented at Atlanta Conference on S&T Policy, Atlanta.
13. Edgerton, D. E. H. (1999). Before big science: The pursuit of modern chemistry and physics, 1800–1940. Annals of Science, 56(1), 100–107.
14. Howell, E. M., & Yemane, A. (2006). An assessment of evaluation designs: Case studies of twelve large federal evaluations. American Journal of Evaluation, 27(2), 219–236.
15. Inventory and Evaluation of Clinical Research Networks. (2006a). Best practices study final report. Rockville, MD: WESTAT.
16. Inventory and Evaluation of Clinical Research Networks. (2006b). Core and descriptive survey final report. Rockville, MD: WESTAT.
17. Jefferson, T., & Godlee, F. (1999). Peer review in health sciences. London: British Medical Journal Publishing Group.
18. Kane, M., & Trochim, W. M. K. (2007). Concept mapping for planning and evaluation. Thousand Oaks, CA: Sage.
19. Kostoff, R. N. (1994a). Assessing research impact—Federal peer-review practices. Research Evaluation, 18(1) 31–40.
20. Kostoff, R. N. (1994b). Quantitative qualitative federal research impact evaluation practice. Technological Forecasting and Social Change, 45(2), 189–205.
21. Kostoff, R. N. (1995). Research requirements for research impact assessment. Research Policy, 24(6), 869–882.
22. Loscalzo, J. (2006). NIH-sponsored clinical trials worth the cost, Reuters Health Information: The NIH budget and the future of biomedical research. New England Journal of Medicine, 354(16), 1665–1667.
23. Manning, F. G., McGeary, M., Estabrook, R., & Committee for Assessment of NIH Centers of Excellence Programs. (2004). National Institute of Health extramural center programs: Criteria for initiation and evaluation. Washington, DC: Institute of Medicine, Board on Health Sciences Policy.
24. Nash, S. J., & Stillman, B. W. (2003). Large-scale biomedical science: Exploring strategies for future research. Washington, DC: National Academies Press.
25. National Academy of Sciences, National Academy of Engineering, & Institute of Medicine. (1996). An assessment of the National Science Foundation's science and technology centers program. Washington, DC: National Academies Press.
26. National Academy of Sciences, National Academy of Engineering, & Institute of Medicine. (2001). Implementing the Government Performance and Results Act for research: A status report. Washington, DC: National Academies Press.
27. National Institute of Allergy and Infectious Diseases (NIAID). (2006). NIAID announces leadership for newly restructured HIV/AIDS clinical trials networks, 2006.
28. Retrieved August 17, 2006, from http://www3.niaid.nih.gov/news/newsreleases/2006/leadership.htm
29. National Research Council. (1999). Evaluating federal research programs: Research and the Government Performance and Results Act. Washington, DC: National Academies Press.
30. Osareh, F. (1996). Bibliometrics, citation analysis and co-citation analysis: A review of literature. Libri, 46(3), 149–158.
31. Smith, R. (2001). Measuring the social impact of research: Difficult but necessary. British Medical Journal, 323, 528.
32. Stokols, D., Fuqua, J., Gress, J., Harvey, R., Phillips, K., Baezconde-Garbanati, L., et al. (2003). Evaluating transdisciplinary science. Nicotine and Tobacco Research, 5, S-1, S21–S39.
33. Trochim, W. (1989). An introduction to concept mapping for planning and evaluation. In W. Trochim (Ed.), of Evaluation and Program Planning, 12, 1–16.
34. U.S. Congress. (2006). National Institutes of Health Reform Act of 2006. 109th Cong., H.R. 6164.
35. Retrieved March 9, 2007, from http://www.govtrack.us/congress/billtext.xpd?bill=h1096164
36. U.S. General Accounting Office. (2000). NIH research: Improvements needed in monitoring external grants (GAO-HEHS-AIMD-00–139). Washington, DC: U.S. General Accounting Office.
37. U.S. Office of Management and Budget. (1993). Government Performance Results Act of 1993. Executive Office of the President of the United States, 1993.
38. Retrieved August 17, 2006, from http://www.whitehouse.gov/omb/mgmt-gpra/gplaw2m.html
39. Vonortas, N. S., & Malerba, F. (2005, October). Using social network methodology to evaluate research and development programs. Paper presented at the joint conference of the Canadian Evaluation Society and the American Evaluation Association, Toronto, Ontario.
40. Zuckerman, B. L., & Kupfer, L. (2005, October). Social network-based design of collaborative research program evaluation. Paper presented at the joint conference of the Canadian Evaluation Society and the American Evaluation Association, Toronto, Ontario.