A 'perverse incentive' from bibliometrics: Could National Research Assessment Exercises (NRAEs) restrict literature availability for nature conservation?

Scientometrics

Volumn 95, Issue 1, 2013, Pages 243-255

  Calver, Michael C ^a   Lilith, Maggie ^a   Dickman, Christopher R ^b  

^a MURDOCH UNIVERSITY   (Australia)

^b UNIVERSITY OF SYDNEY   (Australia)

Author keywords

Australia; ERA; Nature conservation; New Zealand; NRAE; PBRF; RAE; REF; Threatening process; UK; USA; VTR

Indexed keywords

EID: 84874793060     PISSN: 01389130     EISSN: None     Source Type: Journal    
DOI: 10.1007/s11192-012-0908-1     Document Type: Article

References (63)

1. Adler, R., Ewing, J. & Taylor, P. (2008). Citation statistics. A report from the International Mathematical Union (IMU) in cooperation with the International Council of Industrial and Applied Mathematics (ICIAM) and the Instititute of Mathematical Statistics (IMS). http://www. mathunion. org/fileadmin/IMU/Report/CitationStatistics. pdf. Accessed 28 August 2012.
2. Adler, N. J., & Harzing, A.-W. (2009). When knowledge wins: transcending the sense and nonsense of academic rankings. Academy of Management Learning and Education, 8, 72-95.
3. ARC (2012). ERA 2012 frequently asked questions. Available from http://www. arc. gov. au/era/faq. htm. Accessed 12 August 2011.
4. Bollen, J., van de Sompel, H., Hagberg, A., & Chute, R. (2009). A principal component analysis of 39 scientific impact measures. PLoS ONE, 4(6), e6022. doi: 10. 1371/journal. pone. 0006022.
5. Box, S. (2010). Performance-based funding for public research in tertiary education institutions: Country experiences. In OECD Performance-based Funding for Public Research in Tertiary Education Institutions: Workshop Proceedings. Paris: OECD Publishing. doi: 10. 1787/9789264094611-en.
6. Broadbent, J. (2010). The UK research assessment exercise: Performance measurement and resource allocation. Australian Accounting Review, 20, 14-23.
7. Bryant, K. & Calver, M. (2012). Adaptive radiation in Australian journals in the Arbustocene ERA: an empty niche for JANCO? In P. B. Banks, D. Lunney & C. R. Dickman (Eds.), Science under siege (in press). Sydney: Royal Zoological Society of New South Wales.
8. Butler, L. (2007). Assessing university research: a plea for a balanced approach. Science and Public Policy, 34, 565-574.
9. Butler, L. (2010). Impacts of performance-based research funding systems: a review of the concerns and the evidence. In OECD Performance-based Funding for Public Research in Tertiary Education Institutions: Workshop Proceedings. Paris: OECD Publishing. doi: 10. 1787/9789264094611-en.
10. Butler, L., & McAllister, I. (2009). Authors' response to reviews. Political Studies Review, 7, 84-87.
11. Calver, M. C., & King, D. R. (2000). Why publication matters in conservation biology. Pacific Conservation Biology, 6, 2-8.
12. Calver, M. C., Grayson, J., Lilith, M., & Dickman, C. R. (2011). Applying the precautionary principle to the issue of impacts by pet cats on urban wildlife. Biological Conservation, 144, 1895-1901.
13. Calver, M., Wardell-Johnson, G., Bradley, S., & Taplin, R. (2010). What makes a journal international? A case study using conservation biology journals. Scientometrics, 85, 387-400.
14. Carr, K. (2011). Improvements to Excellence in Research for Australia. Canberra: Australian Government. Available from http://archive. innovation. gov. au/ministersarchive2011/carr/MediaReleases/Pages/IMPROVEMENTSTOEXCELLENCEINRESEARCHFORAUSTRALIA. html. Accessed 9 April 2011.
15. Clark, J. A., Hoekstra, J. M., Boersma, P. D., & Kareiva, P. (2002). Improving US Endangered Species Act recovery plans: key findings and recommendations of the SCB recovery plan project. Conservation Biology, 16, 1510-1519.
16. Colledge, L., De Moya-Anegón, F., Guerrero-Bote, V., López-Illescas, C., El Aisati, M., & Moed, H. F. (2010). SJR and SNIP: Two new journal metrics in Elsevier's Scopus. Serials, 23, 215-221.
17. Cooper, S., & Poletti, A. (2011). The new ERA of journal ranking: the consequences of Australia's fraught encounter with 'quality'. Australian Universities' Review, 53, 57-65.
18. Corsi, M., D'Ippoliti, C., & Lucidi, F. (2010). Pluralism at risk? Heterodox economic approaches and the evaluation of economic research in Italy. American Journal of Economics and Sociology, 69, 1495-1529.
19. Debachere, M.-C. (1995). Problems in obtaining grey literature. IFLA Journal, 21, 94-98.
20. Deville, A., & Harding, R. (1997). Applying the precautionary principle. Sydney: The Federation Press.
21. Edgar, F., & Geare, A. (2010). Characteristics of high- and low-performing university departments as assessed by the New Zealand performance based research funding (PBRF) exercise. Australian Accounting Review, 20, 55-63.
22. Elton, L. (2000). The UK research assessment exercise: unintended consequences. Higher Education Quarterly, 54, 274-283.
23. Fairfull, S. J., & Williams, R. J. (2003). Community involvement in natural resource management: Lessons for future water management in catchments of New South Wales. In P. Hutchings & D. Lunney (Eds.), Conserving marine environments: Out of sight, out of mind (pp. 55-61). Sydney: Royal Zoological Society of New South Wales.
24. Falagas, M. E., & Alexiou, V. G. (2008). The top-ten in journal impact factor manipulation. Archivum Immunologiae Et Therapiae Experimentalis, 56, 223-226.
25. Gihus, N. E., & Sivertsen, G. (2009). Publishing affects funding in neurology. European Journal of Neurology, 17, 147-151.
26. Gowrishankar, J., & Divakar, P. (1999). Sprucing up one's impact factor (multiple letters). Nature, 401(6751), 321-322.
27. Hicks, D. (2009). Evolving regimes of multi-university research evaluation. Higher Education, 57, 541-552.
28. Hicks, D. (2010). Overviews of performance-based research funding systems. In OECD Performance-based Funding for Public Research in Tertiary Education Institutions: Workshop Proceedings. Paris: OECD Publishing. doi: 10. 1787/9789264094611-en.
29. Hodder, A. P. W., & Hodder, C. (2010). Research culture and New Zealand's performance-based research fund: Some insights from bibliographic compilations of research outputs. Scientometrics, 84, 1-15.
30. Horwitz, P. & Wardell-Johnson, G. (2009). Cultural conservation biology. In M. Calver, A. Lymbery, J. McComb & M. Bamford, M. (Eds.) Environmental biology (pp. 559-578). Melbourne: Cambridge University Press.
31. Jacsó, P. (2010). Comparison of journal impact rankings in the SCImago journal and country rank and the journal citation reports databases. Online Information Review, 34, 642-657.
32. Lane, J. (2010). Let's make science metrics more scientific. Nature, 464, 488-489.
33. Lane, J., & Bertuzzi, S. (2011). Measuring the results of science investments. Science, 331, 678-680.
34. Lawrence, P. A. (2007). The mismeasurement of science. Current Biology, 17, R583-R585.
35. Luwel, M. (2010). Highlights and reflections: rapporteur's report. In OECD Performance-based Funding for Public Research in Tertiary Education Institutions: Workshop Proceedings. Paris: OECD Publishing. doi: 10. 1787/9789264094611-en.
36. Marsh, H., Smith, B., King, M., & Evans, T. (2012). A new era for research education in Australia? Australian Universities' Review, 54, 83-93.
37. Martin, B. R. (2011). The research excellence framework and the 'impact agenda'. Are we creating a Frankenstein monster? Research Evaluation, 20, 247-254.
38. McNay, I. (1998). The research assessment exercise (RAE) and after: 'you never know how it will turn out'. Perspectives: Policy and Practice in Higher Education, 2, 19-22.
39. Meffe, G. (2006). The success-and challenges-of conservation biology. Conservation Biology, 20, 931-933.
40. Molas-Gallart, J., & Tang, P. (2011). Tracing 'productive interactions' to identify social impacts: an example from the social sciences. Research Evaluation, 20, 219-226.
41. Northcott, D., & Linacre, S. (2010). Producing spaces for academic discourse: The impact of research assessment exercises and journal quality rankings. Australian Accounting Review, 52, 38-54.
42. OECD (2010). Performance-based Funding for Public Research in Tertiary Education Institutions: Workshop Proceedings. Paris: OECD Publishing. doi: 10. 1787/9789264094611-en.
43. Oppenheim, C. (2008). Out with the old and in with the new: the RAE, bibliometrics and the new REF. Journal of Librarianship and Information Science, 40, 147-149.
44. Ortega-Argueta, A., Baxter, G., & Hockings, M. (2011). Compliance of Australian threatened species recovery plans with legislative requirements. Journal of Environmental Management, 92, 2054-2060.
45. Oswald, A. J. (2010). A suggested method for the measurement of world-leading research (illustrated with data on economics). Scientometrics, 84, 99-113.
46. Primack, R. (2009). Why did we reject your paper? Biological Conservation, 142, 1559.
47. Rafols, I., Leydesdorff, L., O'Hare, A., Nightingale, P., & Stirling, A. (2012). How journal rankings can suppress interdisciplinary research: A comparison between innovation studies and business and management. Research Policy, 41, 1262-1282.
48. Roa, T., Beggs, J. R., Williams, J., & Mohler, H. (2009). New Zealand's performance based research funding (PBRF) model undermines Maori research. Journal of the Royal Society of New Zealand, 39, 233-238.
49. Sampson, K. A., & Comer, K. (2010). When the governmental tail wags the disciplinary dog: some consequences of national funding policy on doctoral research in New Zealand. Higher Education Research and Development., 29, 275-289.
50. Schuch, S., Bock, J., Leuschner, C., Schaefer, M., & Wesche, K. (2011). Minor changes in orthopteran assemblages of Central European protected dry grasslands during the last 40 years. Journal of Insect Conservation, 15, 811-822.
51. Schuch, S., Bock, J., Krause, B., Wesche, K., & Schaefer, M. (2012a). Long-term population trends in three grassland insect groups: A comparative analysis of 1951 and 2009. Journal of Applied Entomology, 136, 321-331.
52. Schuch, S., Wesche, K., & Schaefer, M. (2012b). Long-term decline in the abundance of leafhoppers and planthoppers (Auchenorrhyncha) in Central European protected dry grasslands. Biological Conservation, 149, 75-83.
53. SCImago (2007). SJR-SCImago Journal & Country Rank. Consejo Superior de Investigaciones Científicas (CSIC), University of Granada, Extremadura, Carlos III (Madrid) & Alcalá de Henares, Spain. Available from http://www. scimagojr. com. Accessed July-August 2010.
54. SCImago (2012). SCImago institutions rankings. SIR World Rankings 2011: Global ranking. Available from http://www. scimagoir. com/ Accessed 9 April 2012.
55. Shewan, L. G., & Coats, A. J. S. (2006). The research quality framework and its implications for health and medical research: time to take stock? Medical Journal of Australia, 184, 463-466.
56. Smith, S., Ward, V., & House, A. (2011). 'Impact' in the proposals for the UK's Research Excellence Framework: shifting the boundaries of academic autonomy. Research Policy, 40, 1369-1379.
57. Spaapen, J., & van Drooge, L. (2011). Productive interactions as a tool for social impact assessment of research. Research Evaluation, 20, 211-218.
58. Steele, C., Butler, L., & Kingsley, D. (2006). The publishing imperative: the pervasive influence of publication metrics. Learned Publishing, 19, 277-290.
59. Stergiou, K. I. S., & Tsikliras, A. C. (2006). Underrepresentation of regional ecological research output by bibliometric indices. Ethics in Science and Environmental Politics, 6, 15-17.
60. Stinchcombe, J., & Moyle, L. C. (2002). The influence of the academic conservation biology literature on endangered species recovery planning. Conservation Ecology 6(2), 15. http://www. consecol. org/vol16/iss12/art15/.
61. UNESCO (2005). The precautionary principle. World Commission on the Ethics of Scientific Knowledge and Technology (COMEST). United Nations Educational, Scientific and Cultural Organization, Paris.
62. Visser, G. (2009). Tourism geographies and the South African National Research Foundation's Researcher Rating System: international connections and local disjunctures. Tourism Geographies, 11, 43-72.
63. Witten, K., & Hammond, K. (2010). What becomes of social science knowledge: New Zealand researchers' experiences of knowledge transfer modes and audiences. Kotuitui, 5, 3-12.