The knowledge pool: Measurement challenges in evaluating fundamental research programs

Evaluation and Program Planning

Volumn 20, Issue 1, 1997, Pages 77-89

  Cozzens, Susan E ^a  

^a NATIONAL SCIENCE FOUNDATION   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 0011583567     PISSN: 01497189     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0149-7189(96)00038-9     Document Type: Article

References (43)

1. Anderson, J. (1989). New approaches to evaluation in U.K. funding agencies, SPSG Concept Paper No. 9. Science Policy Support Group, London.
2. Callon, M., Callon, Law, J., & Rip, A. (Eds.). (1986). Mapping the dynamics of science and technology. London: Macmillan.
3. Chubin, D. E., & Moitra, S. D. (1975). Content analysis of references: Adjunct or alternative to citation counting? Social Studies of Science, 5, 423-441.
4. Cole, S., Cole, J. R., & Simon, G.A. (1981). Chance and consensus in peer review. Science, 214, 881-886.
5. Cook, T. D., & Shadish, W. R. (1986). Evaluation: The worldly science. Annual Review of Psychology, 37, 193-232.
6. Cozzens, S. E. et al. (1989). Literature-based data in research evaluation: A manager's guide to bibliometrics. Report to the National Science Foundation, Washington, DC.
7. Cozzens, S. E. et al. (Practitioners' Working Group on Research Evaluation). (1994). Evaluation of fundamental research programs: A review of the issues. Prepared at the request of the Office of Science and Technology Policy. Available from the author.
8. Gibbons, M., & Johnston, R. (1974). The roles of science in technological innovation. Research Policy, 3, 220-242.
9. Gore, A. (1993). From red tape to results: Creating a government that works better and costs less. Report of the National Performance Review, Washington, DC.
10. Groszyck, W. (1994). Assessment of FY 1994 GPRA pilot project performance plans. OMB memorandum.
11. Hagstrom, W. O. (1965), The scientific community. New York: Basic Books.
12. Hagstrom, W. O. (1971). Inputs, outputs, and the prestige of university science departments. Sociology of Education, 44, 375-397.
13. Healey, P., Rothman, H., & Hoch, P. K. (1986). An experiment in science mapping for research planning. Research Policy, 15, 233-251.
14. Kostoff, R. N. (1988). Evaluation of proposed and existing accelerated research programs by the Office of Naval Research. IEEE Transactions in Engineering Management, 35, 4.
15. Kostoff, R. N. (1994). Research impact assessment: Where are we now? Unpublished manuscript, Office of Naval Research.
16. Luukkonen, T., & Stahle, B. (1993). Evaluation of research fields: Scientists' views. Nord, 15, Nordic Council of Ministers, Copenhagen.
17. Martin, B. R., & Skea, J. E. F. (1992). Academic research performance indicators: An assessment of the possibilities. Brighton: Science Policy Research Unit, U.K.
18. Merton, R. K. (1973). The sociology of science: Theoretical and empirical investigations. Chicago: University of Chicago Press.
19. The Mitre Corporation. (1978). Evaluative study of the Materials Research Laboratory Program. MTR 7764.
20. National Institutes of Health. (1994). Report of the External Advisory Committee of the Director's Advisory Committee on the Intramural Research Program. Bethedsa, MD: NIH.
21. National Science Foundation. (1987). Sources of financial support for research prize winners. (Report no. NSF 87-87). Washington, DC: Author.
22. Narin, F. et al. (1976). Evaluative bibliometrics: The use of publication and citation analysis in the evaluation of scientific activity, Project No. 704R. Prepared in Fulfillment of Contract NSF C-627 with the National Science Foundation. Cherry Hill, NJ: Computer Horizons.
23. Narin, F., Olivastro, D., & Stevens, K. (1994). Bibliometrics/theory, practice and problems. Evaluation Review, 18(1), 65-76.
24. National Research Council. (1995). Evolving the high performance computing and communications initiative to support the nation's information infrastructure. Washington, DC: National Academy Press.
25. O'Brien, C. (1994). Quantity no longer counts in Britain. Science, 264(24 June), 1840.
26. Ormala, E. (1989). Nordic experiences of the evaluation of technical research and development. Research Policy, 18, 313-342.
27. Panetta, L. E. (1994). Memorandum for the Heads of Departments and Agencies designated at Pilot Projects under P.L. 103-162. Washington, DC: Executive Office of the President, Office of Management and Budget.
28. Phillimore, A. J. (1989). University research performance indicators in practice: The University Grants Committee's evaluation of British universities, 1985-86. Research Policy, 18, 255-271.
29. Research Corporation. (1982). Study of patents resulting from NSF chemistry program. Final report on NSF Contract EVL-8107270, New York.
30. Rigter, H. (1986). Evaluation of performance of health research in the Netherlands. Research Policy, 15, 33-48.
31. Rist, R. C. (Ed.) (1990). Program evaluation and the management of government: Patterns and prospects across eight nations. New Brunswick, NJ: Transaction Publishers.
32. Roth, W. V., Jr. (1995). Improving government performance. Speech given at the Brookings Institution.
33. Shadish, W. R. Jr., Cook, T. D., & Leviton, L. C. (1991). Foundations of program evaluation: Theories of practice. Newbury Park, CA: Sage.
34. Skoie, H. (1993). EC research and technology policies: Some characteristics of its development and future perspectives. Oslo, Norway: Institute for Studies in Research and Higher Education.
35. Small, H., & Garfield, E. (1985). The geography of science: Disciplinary and national mappings. Journal of Information Science, 11, 147-159.
36. Storer, N. (1966) The social system of science. New York: Holt, Rinehart & Winston.
37. Travis, G. D. L., & Collins, H. M. (1991). New light on old boys: Cognitive and institutional particularism in the peer review system. Science, Technology, & Human Values, 76(3), 322-341.
38. U.S. Senate. (1993). Government Performance and Results Act of 1993. 103nd Congress, 1st Session, Report 103-58. Washington, DC: U.S. G.P.O.
39. Van Raan, A. F. J. (1993). Advanced bibliometric methods to assess research performance and scientific development: Basic principles and recent practical applications. Review report, University of Leiden report CWTS-93-05.
40. Virgo, J. A. (1977). A statistical procedure for evaluating the importance of scientific papers. The Library Quarterly, 47(4), 415-430.
41. Wiklund, A. (1994), Swedish Natural Sciences Research Council. Interview with Author.
42. Wye, C. G. (1992). Evaluation in the federal government: Changes, trends, and opportunities. San Francisco, CA: Jossey-Bass.
43. Ziman, J. (1994). Prometheus bound: Science in a dynamic steady state. Cambridge: Cambridge University Press.