Bias against novelty in science: A cautionary tale for users of bibliometric indicators

Research Policy

Volumn 46, Issue 8, 2017, Pages 1416-1436

  Wang, Jian ^a,b   Veugelers, Reinhilde ^a,c,d   Stephan, Paula ^e,f  

^a UNIVERSITY OF LEUVEN   (Belgium)

^b HARVARD LAW SCHOOL   (United States)

^c BRUEGEL   (Belgium)

^d CEPR   (United Kingdom)

^e GEORGIA STATE UNIVERSITY   (United States)

^f NATIONAL BUREAU OF ECONOMIC RESEARCH   (United States)

Author keywords

Bibliometrics; Breakthrough research; Evaluation; Impact; Novelty

Indexed keywords

ENGINEERING; INNOVATION;

BIBLIOMETRIC INDICATORS; BIBLIOMETRICS; COMPLEX RELATIONSHIPS; EVALUATION; IMPACT; NOVELTY; SCIENTIFIC DISCIPLINE; SCIENTIFIC RESEARCHES;

PAPER;

EID: 85025428088     PISSN: 00487333     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.respol.2017.06.006     Document Type: Article

References (66)

1. Adams, J.D., Black, G.C., Clemmons, J.R., Stephan, P.E., Scientific teams and institutional collaborations: evidence from US universities, 1981–1999. Res. Policy 34:3 (2005), 259–285, 10.1016/j.respol.2005.01.014.
2. Alberts, B., Overbuilding research capacity. Science, 329(5997), 2010, 1257, 10.1126/science.1197077.
3. Arrow, K., Economic Welfare and the Allocation of Resources for Invention The Rate and Direction of Inventive Activity: Economic and Social Factors. 1962, Princeton University Press, Princeton, NJ, 609–626.
4. Arthur, W.B., The Nature of Technology: What It Is and How It Evolves. 2009, Free Press, New York, NY.
5. Azoulay, P., Graff Zivin, J.S., Manso, G., Incentives and creativity: evidence from the academic life sciences. RAND J. Econ. 42:3 (2011), 527–554, 10.1111/j. 1756-2171.2011.00140.x.
6. Azoulay, P., Güler, I., Koçak, Ö., Murciano-Goroff, R., Anttila-Hughes, J., Are Recombinant Scientific Articles More Impactful?. 2012.
7. Bornmann, L., Leydesdorff, L., Wang, J., How to improve the prediction based on citation impact percentiles for years shortly after the publication date?. J. Inform. 8:1 (2014), 175–180, 10.1016/j.joi.2013.11.005.
8. Boudreau, K.J., Guinan, E.C., Lakhani, K.R., Riedl, C., Looking across and looking beyond the knowledge frontier: intellectual distance, novelty, and resource allocation in science. Manage. Sci., 2016, 10.1287/mnsc.2015.2285.
9. Boyack, K. W., & Klavans, R., 2014. Atypical combinations are confounded by disciplinary effects. STI 2014 Leiden, 64.
10. Burt, R.S., Structural holes and good ideas. Am. J. Sociol. 110:2 (2004), 349–399, 10.1086/421787.
11. Butler, L., Explaining Australia's increased share of ISI publications—the effects of a funding formula based on publication counts. Res. Policy 32:1 (2003), 143–155, 10.1016/s0048-7333(02)00007-0.
12. Carroll, S.B., Brave Genius: a Scientist, a Philosopher and Their Daring Adventures from the French Resistance to the Nobel Prize. 1st ed., 2013, Crown Publishers, New York, NY.
13. Chen, C., Chen, Y., Horowitz, M., Hou, H., Liu, Z., Pellegrino, D., Towards an explanatory and computational theory of scientific discovery. J. Inform. 3:3 (2009), 191–209, 10.1016/j.joi.2009.03.004.
14. Cook, G., Sebastian seung's quest to map the human brain. N.Y. Times Magaz., 2015, MM27 (11 January 2015).
15. Costas, R., Van Leeuwen, T., Van Raan, A., Is scientific literature subject to a ‘Sell By Date’? A general methodology to analyze the ‘durability'of scientific documents. J. Am. Soc. Inf. Sci. Technol. 61:2 (2010), 329–339.
16. Costas, R., van Leeuwen, T.N., van Raan, A.F.J., Effects of the durability of scientific literature at the group level: case study of chemistry research groups in the Netherlands. Res. Policy 42:4 (2013), 886–894, 10.1016/j.respol.2012.11.006.
17. Denk, W., Horstmann, H., Serial block-face scanning electron microscopy to reconstruct three-dimensional tissue nanostructure. PLoS Biol. 2:11 (2004), 1900–1909, 10.1371/journal.pbio.0020329.
18. Fleming, L., Recombinant uncertainty in technological search. Manage. Sci. 47:1 (2001), 117–132, 10.1287/mnsc.47.1.117.10671.
19. Foster, J.G., Rzhetsky, A., Evans, J.A., Tradition and innovation in scientists’ research strategies. Am. Sociol. Rev. 80:5 (2015), 875–908, 10.1177/0003122415601618.
20. Garfield, E., Premature discovery or delayed recognition–Why. Curr. Contents 21 (1980), 5–10.
21. Gibbons, M., The New Production of Knowledge. 1994, Sage Publications, London, UK.
22. Glänzel, W., Schubert, A., A new classification scheme of science fields and subfields designed for scientometric evaluation purposes. Scientometrics 56:3 (2003), 357–367, 10.1023/A:1022378804087.
23. Hicks, D., Wouters, P., Waltman, L., de Rijcke, S., Rafols, I., The leiden manifesto for research metrics. Nature 520 (2015), 429–431.
24. Hicks, D., The four literatures of social science. Moed, H.F., Glänzel, W., Schmoch, U., (eds.) Handbook of Quantitative Science and Technology Research, 2004, Springer, New York, NY, 473–496.
25. Hicks, D., Performance-based university research funding systems. Res. Policy 41:2 (2012), 251–261, 10.1016/j.respol.2011.09.007.
26. Horrobin, D.F., The philosophical basis of peer review and the suppression of innovation. JAMA 263:10 (1990), 1438–1441, 10.1001/jama.1990.03440100162024.
27. Kaiser, D., Drawing Theories Apart: The Dispersion of Feynman Diagrams in Postwar Physics. 2009, University of Chicago Press, Chicago, IL.
28. Katz, J.S., Hicks, D., How much is a collaboration worth? A calibrated bibliometric model. Scientometrics 40:3 (1997), 541–554 (Retrieved from ://000071723600016).
29. Klahr, D., Simon, H.A., Studies of scientific discovery: complementary approaches and convergent findings. Psychol. Bull., 125(5), 1999, 524.
30. Klavans, R., Boyack, K., Towards the development of an article-level indicator of conformity, innovation and deviation. Paper Presented at the 18th International Conference on Science and Technology Indicators, 2013.
31. Kolata, G., Grant system leads cancer researchers to play it safe. N.Y. Times, 2009 (Retrieved from) http://www.nytimes.com/2009/06/28/health/research/28cancer.html.
32. Kuhn, T.S., The Structure of Scientific Revolutions. 1962, University of Chicago Press, Chicago, IL.
33. Larivière, V., Haustein, S., Börner, K., Long-Distance interdisciplinarity leads to higher scientific impact. PLoS One, 10(3), 2015, e0122565, 10.1371/journal.pone.0122565.
34. Lee, Y.-N., Walsh, J.P., Wang, J., Creativity in scientific teams: unpacking novelty and impact. Res. Policy 44:3 (2015), 684–697, 10.1016/j.respol.2014.10.007.
35. March, J.G., Exploration and exploitation in organizational learning. Org. Sci. 2:1 (1991), 71–87, 10.1287/orsc.2.1.71.
36. Martin, B.R., Editors’ JIF-boosting stratagems −Which are appropriate and which not?. Res. Policy 45:1 (2016), 1–7, 10.1016/j.respol.2015.09.001.
37. Mednick, S.A., The associative basis of the creative process. Psychol. Rev. 69:3 (1962), 220–232, 10.1037/h0048850.
38. Merton, R.K., The Sociology of Science. 1973, University of Chicago Press, Chicago, IL.
39. Monastersky, R., The number that's devouring science. Chronicle Higher Educ., 52(8), 2005, 14.
40. NPR, 2013. Scientists win Nobel for work on how cells communicate. Retrieved from http://www.npr.org/templates/story/story.php?storyId=230192033.
41. Nelson, R.R., Winter, S.G., An Evolutionary Theory of Economic Change. 1982, Belknap Press of Harvard University Press, Cambridge, MA.
42. Petsko, G.A., Goodbye, Columbus. Genome Biol., 13(5), 2012, 155, 10.1186/gb-2012-13-5-155.
43. Phiel, C.J., Zhang, F., Huang, E.Y., Guenther, M.G., Lazar, M.A., Klein, P.S., Histone deacetylase is a direct target of valproic acid, a potent anticonvulsant, mood stabilizer, and teratogen. J. Biol. Chem. 276:39 (2001), 36734–36741, 10.1074/jbc.M101287200.
44. Planck, M., Scientific Autobiography, and Other Papers. 1950, Williams & Norgate, London, UK.
45. Rinia, E., Van Leeuwen, T., Bruins, E., Van Vuren, H., Van Raan, A., Measuring knowledge transfer between fields of science. Scientometrics 54:3 (2002), 347–362, 10.1023/A:1016078331752.
46. Romer, P.M., The origins of endogenous growth. J. Econ. Perspect. 8:1 (1994), 3–22 (Retrieved from) http://www.jstor.org/stable/2138148.
47. Rzhetsky, A., Foster, J.G., Foster, I.T., Evans, J.A., Choosing experiments to accelerate collective discovery. Proc. Natl. Acad. Sci. 112:47 (2015), 14569–14574, 10.1073/pnas.1509757112.
48. Schumpeter, J.A., Business Cycles; A Theoretical, Historical, and Statistical Analysis of the Capitalist Process. 1 st ed, 1939, McGraw-Hill Book Company, Inc, New York, NY; London, UK.
49. Shi, F., Foster, J.G., Evans, J.A., Weaving the fabric of science: dynamic network models of science's unfolding structure. Soc. Netw. 43 (2015), 73–85 http://dx.doi.org/10.1016/j.socnet.2015.02.006.
50. Simon, H.A., Langley, P.W., Bradshaw, G.L., Scientific discovery as problem solving. Synthese 47:1 (1981), 1–27.
51. Simon, H.A., Scientific discovery and the psychology of problem solving. Colodny, R.G., (eds.) Mind and Cosmos; Essays in Contemporary Science and Philosophy, 1966, University of Pittsburgh Press, Pittsburgh, 22–40.
52. Simonton, D.K., Creativity in Science: Chance, Logic, Genius, and Zeitgeist. 2004, Cambridge University Press, Cambridge, UK; New York, NY.
53. StataCorp, 2016. nbreg. Retrieved from http://www.stata.com/manuals13/rnbreg.pdf.
54. Stent, G., Prematurity and uniqueness in scientific discovery. Sci. Am. 227:6 (1972), 84–93.
55. The National Academies, Facilitating Interdisciplinary Research. 2004, The National Academies Press, Washington DC.
56. Uzzi, B., Mukherjee, S., Stringer, M., Jones, B., Atypical combinations and scientific impact. Science 342:6157 (2013), 468–472 10.1126/science.1240474.
57. Van Raan, A., Sleeping beauties in science. Scientometrics 59:3 (2004), 467–472.
58. Varian, H. H.R., 2009. The Economics of Combinatorial Innovation. Guglielmo Marconi Lecture, Lisbon Council. Retrieved from https://www.youtube.com/watch?v=hqaA-fgdXEE.
59. Verhoeven, D., Bakker, J., Veugelers, R., Measuring technological novelty with patent-based indicators. Res. Policy 45:3 (2016), 707–723 http://dx.doi.org/10.1016/j.respol.2015.11.010.
60. Walsh, D., Not safe for funding: the N. S. F. and the economics of science. New Yorker, 2013 (Retrieved from) http://www.newyorker.com/tech/elements/not-safe-for-funding-the-n-s-f-and-the-economics-of-science.
61. Wang, J., Thijs, B., Glänzel, W., Interdisciplinarity and impact: distinct effects of variety, balance, and disparity. PLoS One, 10(5), 2015, e0127298, 10.1371/journal.pone.0127298.
62. Wang, J., Citation time window choice for research impact evaluation. Scientometrics 94:3 (2013), 851–872, 10.1007/s11192-012-0775-9.
63. Weitzman, M.L., Recombinant growth. Q. J. Econ. 113:2 (1998), 331–360 Retrieved from http://www.jstor.org/stable/2586906.
64. Wyatt, H., Knowledge and prematurity: the journey from transformation to DNA. Perspect. Biol. Med. 18:2 (1975), 149–156.
65. Yegros-Yegros, A., Rafols, I., D'Este, P., Does interdisciplinary research lead to higher citation impact? the different effect of proximal and distal interdisciplinarity. PLoS One, 10(8), 2015, e0135095, 10.1371/journal.pone.0135095.
66. Zhang, R., Wang, J., Mei, Y., Search for evergreens in science: A functional data analysis. Journal of Informetrics 11:3 (2017), 629–644, 10.1016/j.joi.2017.05.007.