Standards for applying bibliometrics to the evaluation of research institutes in the natural sciences;Standards für die anwendung der bibliometrie bei der evaluation von forschungsinstituten im bereich der naturwissenschaften

Zeitschrift fur Evaluation

Volumn 11, Issue 2, 2012, Pages 233-260

  Bornmann, Lutz ^a   Bowman, Benjamin F ^b   Bauer, Johann ^b   Marx, Werner ^c   Schier, Hermann ^c   Palzenberger, Margit ^d  

^a MAX PLANCK INSTITUTE   (Germany)

^b MAX PLANCK INSTITUTE OF BIOCHEMISTRY   (Germany)

^c MAX PLANCK INSTITUT FÜR FESTKÖRPERFORSCHUNG   (Germany)

^d MAX PLANCK DIGITAL LIBRARY   (Germany)

Author keywords

Bibliometrics; Citations; Publication Output; Standards

Indexed keywords

EID: 84877798878     PISSN: 16195515     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Article

References (74)

1. Abbott, Alison/Cyranoski, David/Jones, Nicola/Maher, Brendan/Schiermeier, Quirin/Van Noorden, Richard (2010): Metrics: Do Metrics Matter? In: Nature, 465, S. 860-862.
2. Abramo, Giovanni/Cicero, Tindaro/D'Angelo, Ciriaco A. (2011): Assessing the Varying Level of Impact Measurement Accuracy as a Function of the Citation Window Length. In: Journal of Informetrics, 5(4), S. 659-667. doi: 10.1016/j.joi.2011.06.004.
3. Abramo, Giovanni/D'Angelo, Ciriaco A./Di Costa, Flavia (2010): Testing the Trade-Off Between Productivity and Quality in Research Activities. In: Journal of the American Society for Information Science and Technology, 61(1), S. 132-140.
4. Aksnes, Dag W. (2003): A Macro Study of Self-Citation. In: Scientometrics, 56(2), S. 235-246.
5. Albarrán, Pedro/Crespo, Juan/Ortuño, Ignacio/Ruiz-Castillo, Javier (2011): The Skewness of Science in 219 Sub-Fields and a Number of Aggregates. In: Scientometrics, 88(2), S. 385-397. doi: 10.1007/s11192-011-0407-9.
6. American Psychological Association (2009): Publication Manual of the American Psychological Association. Washington: American Psychological Association (APA) (6. Aufl.).
7. Andres, Ana (2011): Measuring Academic Research: How to Undertake a Bibliometric Study. New York: Neal-Schuman Publishers.
8. Barbui, Corrado/Cipriani, Andrea/Malvini, Lara/Tansella, Michele (2006): Validity of the Impact Factor of Journals as a Measure of Randomized Controlled Trial Quality. In: Journal of Clinical Psychiatry, 67(1), S. 37-40.
9. Berghmans, Thierry/Meert, Anne-Pascale/Mascaux, Céline/Paesmans, Marianne/Lafitte, Jean-Jacques/Sculier, Jean-Paul (2003): Citation Indexes do Not Reflect Methodological Quality in Lung Cancer Randomised Trials. In: Annals of Oncology, 14(5), S. 715-721.
10. Bornmann, Lutz (2010): Mimicry in Science? In: Scientometrics, 86(1), S. 173-177.
11. Bornmann, Lutz (2011): Scientific Peer Review. In: Annual Review of Information Science and Technology, 45, S. 199-245.
12. Bornmann, Lutz/Daniel, Hans-Dieter (2009): Universality of Citation Distributions: A Validation of Radicchi et al.'s Relative Indicator cf = c/c0 at the Micro Level Using Data from Chemistry. In: Journal of the American Society for Information Science and Technology, 60(8), S. 1664-1670.
13. Bornmann, Lutz/Marx, Werner/Schier, Hermann/Rahm, Erhard/Thor, Andreas/Daniel, Hans-Dieter (2009): Convergent Validity of Bibliometric Google Scholar Data in the Field of Chemistry: Citation Counts for Papers That Were Accepted by Angewandte Chemie International Edition or Re jected but Published Elsewhere, Using Google Scholar, Science Citation Index, Scopus, and Chemical Abstracts. In: Journal of Informetrics, 3(1), S. 27-35. doi: 10.1016/j.joi.2008.11.001.
14. Bornmann, Lutz/Marx, Werner/Winterhager, Matthias (2012): Wahrheit und Klarheit: Uneinheitliche Namen von Universitäten und ihre Folgen für die Forschungsevaluation. In: Forschung & Lehre, 12(8), S. 650-651.
15. Bornmann, Lutz/Mutz, Rüdiger/Daniel, Hans-Dieter (2010): The h Index Research Output Measurement: Two Approaches to Enhance its Accuracy. In: Journal of Informetrics, 4(3), S. 407-414. doi: 10.1016/j.joi.2010.03.005.
16. Bornmann, Lutz/Mutz, Rüdiger/Marx, Werner/Schier, Hermann/Daniel, Hans-Dieter (2011): A Multilevel Modelling Approach to Investigating the Predictive Validity of Editorial Decisions: Do the Editors of a High-Profile Journal Select Manuscripts That Are Highly Cited After Publication? In: Journal of the Royal Statistical Society-Series A (Statistics in Society), S. 174(4), 857-879. doi: 10.1111/j.1467-985X.2011.00689.x.
17. Bornmann, Lutz/Mutz, Rüdiger/Neuhaus, Christoph/Daniel, Hans-Dieter (2008): Use of Citation Counts for Research Evaluation: Standards of Good Practice for Analyzing Bibliometric Data and Presenting and Interpreting Results. In: Ethics in Science and Environmental Politics, 8, S. 93-102. doi: 10.3354/esep00084.
18. Bortz, Jürgen/Lienert, Gustav A. (2008): Kurzgefasste Statistik für die klinische Forschung. Leitfaden für die verteilungsfreie Analyse kleiner Stichproben. Heidelberg: Springer Medizin Verlag (3., aktual. u. bearb. Aufl.).
19. Bortz, Jürgen/Schuster, Christoph (2010): Statistik für Human-und Sozialwissenschaftler. Berlin: Springer (7. Aufl.).
20. Boyack, Kevin W. (2004): Mapping Knowledge Domains: Characterizing PNAS. In: Proceedings of the National Academy of Sciences of the United States of America, 101, S. 5192-5199.
21. Calver, Michael C./Bradley, J. Stuart (2009): Should We Use the Mean Citations per Paper to Summarise a Journal's Impact or to Rank Journals in the Same Field? In: Scientometrics, 81(3), S. 611-615. doi: 10.1007/s11192-008-2229-y.
22. Danell, Rickard (2011): Can the Quality of Scientific Work Be Predicted Using Information on the Author's Track Record? In: Journal of the American Society for Information Science and Technology, 62(1), S. 50-60. doi: 10.1002/asi.21454.
23. Daniel, Hans-Dieter (2001): Wissenschaftsevaluation. Neuere Entwicklungen und heutiger Stand der Forschungs-und Hochschulevaluation in ausgewählten Ländern (CEST 2001/2). Bern: Center for Science and Technology Studies (CEST).
24. Daniel, Hans-Dieter/Mittag, Sandra/Bornmann, Lutz (2007): The Potential and Problems of Peer Evaluation in Higher Education and Research. In: Cavalli, Allessandro (Hg.): Quality Assessment for Higher Education in Europe. London: Portland Press, S. 71-82.
25. De Bellis, Nicola (2009): Bibliometrics and Citation Analysis: From the Science Citation Index to Cybermetrics. Lanham: Scarecrow Press.
26. Feist, Gregory J. (2006): The Psychology of Science and the Origins of the Scientific Mind. New Haven: Yale University Press.
27. Gagolewski, Marek (2011): Bibliometric Impact Assessment With R and the CITAN Package. In: Journal of Informetrics, 5(4), S. 678-692. doi: 10.1016/j.joi.2011.06.006.
28. García-Pérez, Miguel A. (2010): Accuracy and Completeness of Publication and Citation Records in the Web of Science, PsycINFO, and Google Scholar: A Case Study for the Computation of h Indices in Psychology. In: Journal of the American Society for Information Science and Technology, 61(10), S. 2070-2085. doi: 10.1002/asi.21372.
29. Garfield, Eugene (1972): Citation Analysis as a Tool in Journal Evaluation: Journals Can Be Ranked by Frequency and Impact of Citations for Science Policy Studies. In: Science, 178(4060), S. 471-479.
30. Garfield, Eugene (1979): Citation Indexing-Its Theory and Application in Science, Technology, and Humanities. New York: John Wiley & Sons Ltd.
31. Garfield, Eugene (2006): The History and Meaning of the Journal Impact Factor. In: Journal of the American Medical Association, 295(1), S. 90-93.
32. Glänzel, Wolfgang/Debackere, Koenraad/Thijs, Bart/Schubert, András (2006): A Concise Review on the Role of Author Self-Citations in Information Science, Bibliometrics and Science Policy. In: Scientometrics, 67(2), S. 263-277.
33. Glänzel, Wolfgang/Thijs, Bart/Schubert, András/Debackere, Koenraad (2009): Subfield-Specific Normalized Relative Indicators and a New Generation of Relational Charts: Methodological Foundations Illustrated on the Assessment of Institutional Research Performance. In: Scientometrics, 78(1), S. 165-188.
34. Hemlin, Sven (1996): Research on Research Evaluations. In: Social Epistemology, 10(2), S. 209-250.
35. Hirsch, Jorge E. (2005): An Index to Quantify an Individual's Scientific Research Output. In: Proceedings of the National Academy of Sciences of the United States of America, 102(46), S. 16569-16572. doi: 10.1073/pnas.0507655102.
36. Hornbostel, Stefan (1997): Wissenschaftsindikatoren. Bewertungen in der Wissenschaft. Opladen, Germany: Westdeutscher Verlag.
37. Huang, Mu-Hsuan/Lin, Chi-Shiou/Chen, Dar-Zen (2011): Counting Methods, Country Rank Changes, and Counting Inflation in the Assessment of National Research Productivity and Impact. In: Journal of the American Society for Information Science and Technology, 62(12), S. 2427-2436. doi: 10.1002/asi.21625.
38. Jacso, Peter (2009): Google Scholar's Ghost Authors. In: Library Journal, 134(18), S. 26-27.
39. Jacso, Peter (2010): Metadata Mega Mess in Google Scholar. In: Online Information Review, 34(1), S. 175-191. doi: 10.1108/14684521011024191.
40. Jann, Ben (2005): Tabulation of Multiple Response. The Stata Journal, 5(1), S. 92-122.
41. Kenna, Ralf/Berche, Bertrand (2011): Normalization of Peer-Evaluation Measures of Group Research Quality Across Academic Disciplines. [Article]. In: Research Evaluation, 20(2), S. 107-116. doi: 10.3152/095820211x12941371876625.
42. Larivière, Vincent/Gingras, Yves (2011): Averages of Ratios vs. Ratios of Averages: An Empirical Analysis of Four Levels of Aggregation. In: Journal of Informetrics, 5(3), S. 392-399. doi: 10.1016/j.joi.2011.02.001.
43. Lehmann, Sune/Jackson, Andrew D./Lautrup, Benny E. (2008): A Quantitative Analysis of Indicators of Scientific Performance. In: Scientometrics, 76(2), S. 369-390. doi: 10.1007/s11192-007-1868-8.
44. Leydesdorff, Loet/Bornmann, Lutz/Mutz, Rüdiger/Opthof, Tobias (2011): Turning the Tables in Citation Analysis One More Time: Principles for Comparing Sets of Documents. In: Journal of the American Society for Information Science and Technology, 62(7), S. 1370-1381.
45. Martin, Ben R./Irvine, John (1983): Assessing Basic Research-Some Partial Indicators of Scientific Progress in Radio Astronomy. In: Research Policy, 12(2), S. 61-90.
46. Marx, Werner (2011): Special Features of Historical Papers From the Viewpoint of Bibliometrics. In: Journal of the American Society for Information Science and Technology, 62(3), S. 433-439. doi: 10.1002/asi.21479.
47. Marx, Werner/Schier, Hermann/Andersen, Ole Krogh (2006): Using Time-Dependent Citation Rates (Sales Curves) for Comparing Scientific Impacts. Verfügbar unter http://arxiv.org/ftp/ physics/papers/0611/0611284.pdf [31.08.2011].
48. McClellan, Jason E. (2003): Specialist Control: the Publications Committee of the Académie Royal des Sciences (Paris) 1700-1793 (Transactions of the American Philosophical Society, v. 93). Philadelphia: American Philosophical Society.
49. Meho, Lokman I./Spurgin, Kristina M. (2005): Ranking the Research Productivity of Library and Information Science Faculty and Schools: An Evaluation of Data Sources and Research Methods. In: Journal of the American Society for Information Science and Technology, 56(12), S. 1314-1331.
50. Merton, Robert K. (1985): Entwicklung und Wandel von Forschungsinteressen. Aufsätze zur Wissenschaftssoziologie. Frankfurt am Main: Suhrkamp.
51. Moed, Henk F. (2005): Citation Analysis in Research Evaluation. Dordrecht: Springer.
52. Moed, Henk F./van Leeuwen, Thed N./Reedijk, John (1996): A Critical Analysis of the Journal Impact Factors of Angewandte Chemie and the Journal of the American Chemical Society-Inaccuracies in Published Impact Factors Based on Overall Citations Only. In: Scientometrics, 37(1), S. 105-116.
53. Mulligan, Adrian/Mabe, Michael (2006): Journal Futures: Researcher Behaviour at Early Internet Maturity. Paper presented at the UK Serials Group (UKSG) Annual Conference, University of Warwick, UK.
54. NORIA-net (2011): Comparing Research at Nordic Universities Using Bibliometric Indicators: A Publication From the NORIA-net 'Bibliometric Indicators for the Nordic Universities'. Oslo: NordForsk.
55. Pendlebury, David A. (2008): Using Bibliometrics in Evaluating Research. Philadelphia: Research Department, Thomson Scientific.
56. Raan, Anthony F. J. van (2004): Measuring Science: Capita Selecta of Current Main Issues. In: Moed, Henk F./Glänzel, Wolfgang/Schmoch, Ulrich (Hg.): Handbook of Quantitative Science and Technology Research: The Use of Publication and Patent Statistics in Studies of S&T Systems. Dordrecht: Kluwer Academic Publishers, S. 19-50.
57. Radicchi, Filippo/Castellano, Claudio (2011): Rescaling Citations of Publications in Physics. [Article]. Physical Review E, 83(4). doi: 10.1103/PhysRevE.83.046116.
58. Radicchi, Filippo/Fortunato, Santo/Castellano, Claudio (2008): Universality of Citation Distributions: Toward an Objective Measure of Scientific Impact. In: Proceedings of the National Academy of Sciences, 105(45), S. 17268-17272. doi: 10.1073/pnas.0806977105.
59. Randić, Milan (2009): Citations Versus Limitations of Citations: Beyond Hirsch Index. In: Scientometrics, 80(3), S. 809-818. doi: 10.1007/s11192-008-2128-2.
60. Rehn, Catharina/Kronman, Ulf/Wadskog, Daniel (2007): Bibliometric Indicators-Definitions and Usage at Karolinska Institutet. Stickholm, Sweden: Karolinska Institutet University Library.
61. Research Evaluation and Policy Project (2005): Quantitative Indicators for Research Assessment-a Literature Review (REPP discussion paper 05/1). Canberra: Research Evaluation and Policy Project, Research School of Social Sciences, The Australian National University.
62. Ruegg, Rosalie/Feller, Irwin (2003): A Toolkit for Evaluating Public R&D Investment: Models, Methods, and Findings from ATP's First Decade. Gaithersburg: National Institute of Standards and Technology.
63. Seglen, Per O. (1992): The Skewness of Science. In: Journal of the American Society for Information Science, 43(9), S. 628-638.
64. Shadbolt, Nigel/Brody, Tim/Carr, Les/Harnad, Stevan (2006): The Open Research Web: A Preview of the Optimal and the Inevitable. In: Jacobs, Neil (Hg.): Open Access: Key Strategic, Technical and Economic Aspects. Oxford: Chandos, S. 195-208.
65. Sheskin, David J. (2007). Handbook of Parametric and Nonparametric Statistical Procedures. Boca Raton: Chapman & Hall/CRC (4th ed.).
66. Smith, Richard (1988): Problems with Peer Review and Alternatives. In: British Medical Journal, 296 (6624), S. 774-777.
67. Snyder, Herbert/Bonzi, Susan (1998): Patterns of Self-Citation Across Disciplines (1980-1989). In: Journal of Information Science, 24(6), S. 431-435.
68. StataCorp (2011): Stata Statistical Software: Release 12. College Station, TX, USA: Stata Corporation.
69. Strotmann, Andreas/Zhao, Dangzhi (2010): Combining Commercial Citation Indexes and Open-Access Bibliographic Databases to Delimit Highly Interdisciplinary Research Fields for Citation Analysis. In: Journal of Informetrics, 4(2), S. 194-200. doi: 10.1016/j.joi.2009.12.001.
70. Tijssen, Robert/van Leeuwen, Thed N. (2006): Centres of Research Excellence and Science Indicators. Can 'Excellence' Be Captured in Numbers? In: Glänzel, Wolfgang (Hg.): Ninth International Conference on Science and Technology Indicators. Leuven: Katholieke Universiteit Leuven, S. 146-147.
71. Tijssen, Robert/Visser, Martijn/van Leeuwen, Thed N. (2002): Benchmarking International Scientific Excellence: Are Highly Cited Research Papers an Appropriate Frame of Reference? In: Scientometrics, 54(3), S. 381-397.
72. Vinkler, Peter (2010): The Evaluation of Research by Scientometric Indicators. Oxford: Chandos Publishing.
73. Wright, Malcom/Armstrong, J. Scott (2007): Verification of Citations: Fawlty Towers of Knowledge? Verfügbar unter: http://mpra.ub.uni-muenchen.de/4149/ [6.08.2012].
74. Ziman, John (2000): Real Science: What it is, and What it Means. Cambridge: Cambridge University Press.