Environmental complexity and stakeholder theory in formal research network evaluations

Prometheus: Critical Studies in Innovation

Volumn 30, Issue 3, 2012, Pages 291-314

  Wixted, Brian ^a   Holbrook, J Adam ^a  

^a SIMON FRASER UNIVERSITY   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84868125751     PISSN: 08109028     EISSN: 14701030     Source Type: Journal    
DOI: 10.1080/08109028.2012.727276     Document Type: Article

References (91)

1. ACIL Tasman (2006) Review of the Impact of Some Recent CSIRO Research Activities, report to CSIRO, Canberra, available from http://www.csiro.au/resources/pflj.html [accessed December 2011].
2. Aldrich, H. 1979. Organizations and Environments, Englewood Cliffs, NJ: Prentice-Hall.
3. Arnold, E. (2005) What the Evaluation Record Tells Us About Framework Program Performance, Technopolis Group, Brighton, available from http://www.technopolis-group.com/site/downloads/index.htm [accessed December 2011].
4. Atkinson-Grosjean, J. 2006. Public Science; Private Interests: Cultures and Commerce in Canada's Networks of Centres of Excellence, Toronto: University of Toronto Press.
5. Audretsch, D., Bozeman, B., Comb, K., Feldman, M., Link, A., Siegel, D., Stephan, P., Tassey, G. and Wessner, C. 2002. The economics of science and technology. Journal of Technology Transfer, 27(2): 155-210.
6. Australian Government (2011) Cooperative Research Centres Program Selection Round Application Instructions Selection Round 14 (2011), Department of Innovation, Industry, Science and Research, Canberra, available from https://www.crc.gov.au/Information/ShowInformation.aspx?Doc=14th_Selection_rounds&key=bulletin-board-selection-rounds_14&Heading=14th [accessed June 2011].
7. Australian Research Council (2010) ARC Research Networks, available from http://www.arc.gov.au/ncgp/networks/networks_default.htm [accessed January 2012].
8. Australian Research Council. 2011. Excellence in Research for Australia 2010 National Report, Canberra: Australian Research Council.
9. Barker, K. 2007. The UK Research Assessment Exercise: the evolution of a national research evaluation system. Research evaluation, 16(1): 3-12.
10. Bernstein, A., Hicks, V., Borbey, P. and Campbell, T. (2006) 'A framework to measure the impacts of investments in health research', presentation to the Blue Sky II conference, What Indicators for Science, Technology and Innovation Policies in the 21st Century?, Ottawa, September, available from http://www.oecd.org/dataoecd/10/42/37450246.pdf [accessed October 2008].
11. Bernstein, S., Lebow, R., Stein, J. and Weber, S. (2000) 'God gave physics the easy problems: adapting social science to an unpredictable world', European Journal of International Relations, 6, 1, pp. 43-76.
12. Borgatti, S., Mehra, A., Brass, D. and Labianca, G. 2009. Network analysis in the social sciences. Science, 323: 892-895.
13. Bozeman, B. and Rogers, J. 2002. A churn model of knowledge value. Research Policy, 31: 769-794.
14. Cooperative Research Centres (2012) Cooperative Research Centres website, available from https://www.crc.gov.au/Information/default.aspx [accessed October 2012].
15. Corbyn, Z. (2008) 'RCUK abandons impact formula', Times Higher Education, 6 March, available from [accessed October 2008].
16. Crane, A. and Ruebottom, T. 2011. Stakeholder theory and social identity: rethinking stakeholder identification. Journal of Business Ethics, 102(1): 77-87.
17. Crane, D. 1972. Invisible Colleges, Chicago, IL: University of Chicago Press.
18. Creech, H. (2001) Form Follows Function: Management and Governance of a Formal Knowledge Network, International Institute for Sustainable Development, Winnipeg.
19. Creech, H. and Ramji, A. (2004) Knowledge Networks: Guidelines for Assessment, International Institute for Sustainable Development (IISD), Winnipeg, available from http://www.iisd.org/pdf/2004/networks_guidelines_for_assessment.pdf [accessed December 2011].
20. Cutler, T. (2008) Venturous Australia: Building Strength in Innovation, Department of Industry, Innovation, Science and Research, Canberra, available from http://www.innovation.gov.au/innovationreview/Pages/home.aspx [accessed December 2011].
21. Dasgupta, P. and David, P. 1994. Toward a new economics of science. Research Policy, 23(5): 487-521.
22. Demarteau, M. 2002. A theoretical framework and grid for analysis of program-evaluation practices. Evaluation, 8(4): 454-473.
23. Dess, G. and Beard, D. 1984. Dimensions of organizational task environments. Administrative Science Quarterly, 29: 52-73.
24. Edler, J. and Rigby, J. (2004) Research Network Programs Evaluation for the Austrian Science Fund (FWF), PREST, University of Manchester, available from http://www.fwf.ac.at/de/downloads/pdf/networks_evaluation.pdf [accessed December 2011].
25. Etzkowitz, H. and Leydesdorff, L. 2000. The dynamics of innovation: from national systems and "mode 2" to a triple helix of university-industry-government relations. Research Policy, 29(2): 109-123.
26. Fahrenkrog, G., Polt, W., Rojo, J., Tübke, A. and Zinöcker, K. (eds) (2002) RTD Evaluation Toolbox: Assessing the Socio-Economic Impact of RTD Policies, IPTS Technical Report Series, IPTS, Seville.
27. Freeman, C. (1968) 'Science and economy at the national level' in OECD (ed.) Problems of Science Policy, OECD, Paris, pp. 60-65.
28. Garrett-Jones, S., Wixted, B. and Turpin, T. 2004. Some international benchmarks for evaluating Australian health and medical research. Research Evaluation, 13(3): 155-166.
29. Gaughan, M. and Ponomariov, B. 2008. Faculty publication productivity, collaboration, and grants velocity: using curricula vitae to compare center-affiliated and unaffiliated scientists. Research Evaluation, 17(2): 103-110.
30. Geisler, E. 2000. The Metrics of Science and Technology, Westport, CT: Quorum Books.
31. Gibbons, M., Limoges, C., Nowotny, H., Schwartzman, S., Scott, P. and Trow, M. 1994. The New Production of Knowledge. The Dynamics of Science and Research in Contemporary Societies, Thousand Oaks, CA: Sage.
32. Gläser, J., Spurling, T. and Butler, L. 2004. Intraorganisational evaluation: are there "Least evaluable units"?. Research Evaluation, 13(1): 19-32.
33. Godin, B. 2007. Science, accounting and statistics: the input-output framework. Research Policy, 36(9): 1388-1403.
34. Granovetter, M. 1973. The strength of weak ties. American Journal of Sociology, 6: 1360-1380.
35. Hansen, H. 2005. Choosing evaluation models A discussion on evaluation design. Evaluation, 11(4): 447-462.
36. Hickling Arthurs Low (2002) Evaluation Framework for the Canada Foundation for Innovation, report prepared for the Canada Foundation for Innovation, available from http://www.innovation.ca/evaluation/ef_hal_e.pdf [accessed August 2007].
37. Holbrook, A., Wixted, B., Chee, F., Klingbeil, M. and Shaw-Garlock, G. (2009) Measuring the Return on Investment in Research in Universities: The Value of the Human Capital Produced by these Programs, report to the British Columbia Ministry of Technology, Trade and Economic Development, available from http://www.sfu.ca/cprost/reports.html [accessed December 2009].
38. Holbrook, J., Wixted, B., Lewis, B. and Cressman, D. (2012) 'The structure and construction of formal research networks: a policy oriented understanding of stakeholder engagement', mimeo.
39. Howells, J. and Edler, J. 2011. Structural innovations: towards a unified perspective?. Science and Public Policy, 38(2): 157-167.
40. Howlett, M. and Ramesh, M. 1998. Policy subsystem configurations and policy change: operationalizing the postpositivist analysis of the politics of the policy process. Policy Studies Journal, 26(3): 466-481.
41. Insight Economics. 2006. Economic Impact Study of the CRC Program, report prepared for the Australian Government Department of Education, Canberra: Science and Training.
42. Kilduff, M. and Brass, D. 2010. Organizational social network research: core ideas and key debates. Academy of Management Annuals, 4: 317-357.
43. Kishchuk, N. (2005) Performance Report: SSHRC's Major Collaborative Research Initiatives (MCRI) Program, SSHRC, Ottawa, March, available from http://www.sshrc-crsh.gc.ca/about-au_sujet/publications/mcri_performance_e.pdf [accessed October 2012].
44. Klenk, N., Hickey, G., Maclellan, J., Gonzales, R. and Cardille, J. 2009. Social network analysis: a useful tool for visualizing and evaluating forestry research. International Forestry Review, 11(1): 134-140.
45. Leydesdorff, L., Dolfsma, W. and Panne van der, G. 2006. Measuring the knowledge base of an economy in terms of triple-helix relations among technology, organization, and territory. Research Policy, 35: 181-199.
46. Leydesdorff, L. and Meyer, M. 2006. Triple helix indicators of knowledge-based innovation systems: introduction to the special issue. Research Policy, 35: 1441-1449.
47. Matthews, D. (2011) 'Deep economic impact: new mission to ensure university research benefits UK business', Times Higher Education, 10 July, available from http://www.timeshighereducation.co.uk/story.asp?storycode=416762 [accessed December 2011].
48. Mattsson, P., Laget, P., Vindefjärd, A. and Sundberg, C. 2010. What do European research collaboration networks in life sciences look like?. Research Evaluation, 19(5): 373-384.
49. McCarthy, I., Lawrence, T., Wixted, B. and Gordon, B. 2010. A multidimensional conceptualization of environmental velocity. Academy of Management Review, 35(4): 604-626.
50. Milward, B. and Provan, K. (2006) A Manager's Guide to Choosing and Using Collaborative Networks, IBM Center for the Business of Government, available from http://www.businessofgovernment.org/ [accessed June 2011].
51. Mitchell, R., Agle, B. and Wood, D. 1997. Toward a theory of stakeholder identification and salience. defining the principles of who and what really counts. Academy of Management Review, 22: 853-886.
52. Mote, J., Jordan, G., Hage, J. and Whitestone, Y. 2007. New directions in the use of network analysis in research and product development evaluation. Research Evaluation, 16(3): 191-203.
53. MSFHR PIWG (2008) Evaluation Framework Version 1.7 Health of Population Networks Performance Indicator Working Group, Michael Smith Foundation for Health Research, Vancouver.
54. National Centres of Excellence (2007) Annual, Report 2006-2007.
55. Nelson, R. 1959. The simple economics of basic scientific research. Journal of Political Economy, 67(3): 297-306.
56. Networks of Centres of Excellence (2007a) Evaluation of the Networks of Centres of Excellence Program, available from http://www.nce.gc.ca/pubs/reports/2007/evaluation/NCEEvaluationReport2007-eng.pdf [accessed December 2011].
57. Networks of Centres of Excellence (2007b) Available from http://www.nce.gc.ca/pubs/reports/2007/selec-renewal-oct07_e.pdf [accessed December 2011].
58. Networks of Centres of Excellence of Canada (2004) The Networks of Centres of Excellence Program: 15 Years of Innovation and Leadership, available from http://www.nce.gc.ca/pubs/history/NCE-histEN.pdf [accessed August 2007].
59. Neurath, W. and Katzmair, H. (2004) 'Networks of innovation-evaluation and monitoring of technology programs based on social network analysis (SNA)', Plattform Forschungs- und Technologieevaluierung GesbR., 20, April [accessed August 2007].
60. Nooteboom, B. (2009) A Cognitive Theory of the Firm: Learning, Governance and Dynamic Capabilities, Edward Elgar, Cheltenham.
61. Nutley, S., Walter, I. and Davies, H. 2007. Using Evidence. How Research Can Inform Public Services, Bristol: Policy Press.
62. Office of Science and Innovation (2007) Measuring Economic Impacts of Investment in the Research Base and Innovation-A New Framework for Measurement, available from http://www.berr.gov.uk/dius/science/science-funding/framework/page9306.html [accessed April 2008].
63. Organisation for Economic Cooperation and Development (OECD) (2006) Science, Technology and Industry Outlook, Organisation for Economic Cooperation and Development, Paris.
64. Organisation for Economic Cooperation and Development (OECD). 2007. Innovation and Growth Rationale for an Innovation Strategy, Paris: Organisation for Economic Cooperation and Development.
65. Organisation for Economic Cooperation and Development (OECD) (2008) Assessing the Socio-Economic Impacts of Public R&D, OECD workshop, June, Organisation for Economic Cooperation and Development Paris, available from http://www.oecd.org/document/7/0,3343,en_2649_34273_40469255_1_1_1_1,00.html [accessed August 2008].
66. Parmar, B., Freeman, R., Harrison, J., Wicks, A., Purnell, L. and de Colle, S. 2010. Stakeholder theory: the state of the art. Academy of Management Annals, 4: 403-445.
67. Pavitt, K. 1991. What makes basic research economically useful?. Research Policy, 20(2): 109-119.
68. Protogerou, A., Caloghirou, Y. and Siokas, E. 2010. Policy-driven collaborative research networks in Europe. Economics of Innovation and New Technology, 19(4): 349-372.
69. Provan, K. and Milward, B. (2001) 'Do networks really work? A framework for evaluating public-sector organizational networks', Public Administration Review, 61, 4, pp.414-23.
70. RCUK (2008) Research Councils UU Response to HEFCE's Consultation on the New Research Excellence Framework (REF), available from http://www.rcuk.ac.uk/news/070220a.htm [accessed March 2008].
71. Research Excellence Framework (REF) (2011) Assessment Framework and Guidance on Submissions, Research Excellence Framework, available from http://www.hefce.ac.uk/research/ref/pubs/2011/02_11/ [accessed February 2012].
72. Rigby, J. (2005) 'Evaluating the FWF's research networks', Plattform Forschungs- und Technologieevaluierung GesbR, 24, September.
73. Roelofsen, A., Boon, W., Kloet, R. and Broerse, J. 2011. Stakeholder interaction within research consortia on emerging technologies: learning how and what?. Research Policy, 40(3): 341-354.
74. Rogers, J., Bozeman, B. and Chompalov, I. 2001. Obstacles and opportunities in the application of network analysis to the evaluation of R&D. Research Evaluation, 10(3): 161-172.
75. Ryan, C. (2008) Evaluating Performance of Collaborative Research Networks: A Socio-economic Framework for Assessing Funded Research Projects, Books on Demand, available from http://www.bod.de/index.php?id=296&objk_id=167105 [accessed August 2008].
76. Ryan, J. 2011. Irish experience of cross-sector research collaboration initiatives. Science and Public Policy, 38(2): 147-155.
77. Sala, A., Landoni, P. and Verganti, R. 2011. R&D networks: an evaluation framework. International Journal of Technology Management, 53(1): 19-43.
78. Salazar, M. and Holbrook, J. 2007. Canadian STI policy: the product of regional networking. Regional Studies, 41(8): 1-13.
79. Salter, A. and Martin, B. 2001. The economic benefits of publicly funded basic research: a critical review. Research Policy, 30: 509-532.
80. Scott, W. (1981) Organizations: Rational, Natural, and Open Systems, Prentice-Hall, Englewood Cliffs, NJ.
81. Slatyer, R. 1994. Cooperative research centres: the concept and its implementation. Higher Education, 28: 147-158.
82. Sorenson, O., Rivkin, J. and Fleming, L. 2006. Complexity, networks and knowledge flow. Research Policy, 35: 994-1017.
83. Steen, J., Macaulay, S. and Kastelle, T. 2011. Small worlds: the best network structure for innovation?. Prometheus, 29(1): 39-50.
84. Stein, G., Stren, R., Fitzgibbon, J. and Maclean, M. 2001. Networks of Knowledge: Collaborative Innovation in International Learning, Toronto: University of Toronto Press.
85. Stephan, P. 1996. The economics of science. Journal of Economic Literature, 34: 1199-1235.
86. Stokols, D., Hall, K., Taylor, B. and Moser, R. 2008. The science of team science. overview of the field and introduction to the supplement. American Journal of Preventive Medicine, 35(2): S77-S89.
87. Strogatz, S. 2001. Exploring complex networks. Nature, 410: 268-276.
88. Thune, T. and Gulbrandsen, M. 2011. Institutionalization of university-industry interaction: an empirical study of the impact of formal structures on collaboration patterns. Science and Public Policy, 38(2): 99-107.
89. Turpin, T. and Fernández-Esquinas, M. 2011. The policy rationale for cross-sector research collaboration and contemporary consequences. Science and Public Policy, 38(2): 82-86.
90. US Department of Health and Human Services (2007) Fiscal Year 2008 National Institutes of Health-Volume II Overview-Performance Detail, available from http://nihperformance.nih.gov/ [accessed September 2007].
91. Van der Valk, T. and Gijsbers, G. (2010) 'The use of social network analysis in innovation studies: mapping actors and technologies', Innovation: Management, Policy and Practice, 12, 1, pp.5-17.