Definition of a model based on bibliometric indicators for assessing applicants to academic positions

Journal of the Association for Information Science and Technology

Volumn 65, Issue 3, 2014, Pages 560-577

  Vieira, Elizabeth S ^a   Cabral, José A S ^a   Gomes, José A N F ^a  

^a UNIVERSITY OF PORTO   (Portugal)

Author keywords

[No Author keywords available]

Indexed keywords

BIBLIOMETRIC INDICATORS; COMPOSITE INDICATORS; DISCRETE CHOICE MODELS; MODEL-BASED OPC; PREDICTIVE POWER; ROBUST PREDICTIONS;

EID: 84900518023     PISSN: 23301635     EISSN: 23301643     Source Type: Journal    
DOI: 10.1002/asi.22981     Document Type: Article

References (49)

1. Abramo, G., D'Angelo, C. A., & Caprasecca, A. (2009). Allocative efficiency in public research funding: Can bibliometrics help? Research Policy, 38(1), 206-215. doi: 10.1016/j.respol.2008.11.001
2. Aksnes, D. W. (2003). A macro study of self-citation. Scientometrics, 56(2), 235-246.
3. Aksnes, D. W., & Taxt, R. E. (2004). Peer reviews and bibliometric indicators: a comparative study at a Norwegian university. Research Evaluation, 13(1), 33-41. doi: 10.3152/147154404781776563
4. Antonakis, J., & Lalive, R. (2008). Quantifying scholarly impact: Iqp versus the Hirsch h. Journal of the American Society for Information Science and Technology, 59(6), 956-969. doi: 10.1002/asi.20802
5. Barnett, V., & Lewis, T. (1994). Outliers in statistical data (3 rd ed.):Wiley.
6. Batista, P. D., Campiteli, M. G., Kinouchi, O., & Martinez, A. S. (2006). Is it possible to compare researchers with different scientific interests? Scientometrics, 68(1), 179-189.
7. Bonzi, S., & Snyder, H.W. (1991). Motivations for citation-A comparison of self citation and citation to others. Scientometrics, 21(2), 245-254. doi: 10.1007/bf02017571
8. Bornmann, L., & Daniel, H. D. (2005). Selection of research fellowship recipients by committee peer review. Reliability, fairness and predictive validity of Board of Trustees' decisions. Scientometrics, 63(2), 297-320. doi: 10.1007/s11192-005-0214-2
9. Bornmann, L., & Daniel, H. D. (2006). Selecting scientific excellence through committee peer review-Acitation analysis of publications previously published to approval or rejection of post-doctoral research fellowship applicants. Scientometrics, 68(3), 427-440. doi: 10.1007/s11192-006-0121-1
10. Bornmann, L., & Daniel, H. D. (2007). Convergent validation of peer review decisions using the h index-Extent of and reasons for type I and type II errors. Journal of Informetrics, 1(3), 204-213. doi: 10.1016/ j.joi.2007.01.002
11. Bornmann, L., Marx, W., Gasparyan, A. Y., & Kitas, G. (2012). Diversity, value and limitations of the journal impact factor and alternative metrics. Rheumatology International, 32(7), 1861-1867. doi: 10.1007/s00296-011- 2276-1
12. Bornmann, L., Wallon, G., & Ledin, A. (2008). Does the Committee Peer Review Select the Best Applicants for Funding? An Investigation of the Selection Process for Two European Molecular Biology Organization Programmes. Plos One, 3(10). doi: e348010.1371/journal.pone.0003480
13. Burrell, Q., & Rousseau, R. (1995). Fractional counts for authorship attribution-A numerical study. Journal of the American Society for Information Science, 46(2), 97-102.
14. Cole, J.R., & Cole, S. (1973). Social stratification in science Chicago. The University of Chicago Press.
15. Field, A. (2005). Discovering Statistics Using SPSS (2nd edition ed.): SAGE Publications.
16. Fok, D., Paap, R., & Van Dijk, B. (2012). A rank-ordered logit model with unobserved heterogeneity in ranking capabilities. Journal of Applied Econometrics, 27(5), 831-846. doi: 10.1002/jae.1223
17. Franceschet, M., & Costantini, A. (2011). The first Italian research assessment exercise: A bibliometric perspective. Journal of Informetrics, 5(2), 275-291. doi: 10.1016/j.joi.2010.12.002
18. Hagen, N. T. (2008). Harmonic Allocation of Authorship Credit: Source- Level Correction of Bibliometric Bias Assures Accurate Publication and Citation Analysis. Plos One, 3(12). doi: 402110.1371/journal.pone.0004021
19. Hair, J.F. Black,W.C. Babin, B.J. Anderson, R.E. (2010). Multivariate Data Analysis (7th Edition ed.): Published by Prentice Hall.
20. Hausman, J. A. (1978). Specification tests in econometrics. Econometrica, 46(6), 1251-1271. doi: 10.2307/1913827
21. Hodge, S. E., & Greenberg, D. A. (1981). Publication credit. Science, 213(4511), 950-950.
22. Ioannidis, J. P. A. (2008). Measuring Co-Authorship and Networking- Adjusted Scientific Impact. Plos One, 3(7). doi: e277810.1371/journal.pone. 0002778
23. Jayasinghe, U. W., Marsh, H. W., & Bond, N. (2001). Peer review in the funding of research in higher education: The Australian experience. Educational Evaluation and Policy Analysis, 23(4), 343-364. doi: 10.3102/01623737023004343
24. Jayasinghe, U. W., Marsh, H. W., & Bond, N. (2003). A multilevel crossclassified modelling approach to peer review of grant proposals: the effects of assessor and researcher attributes on assessor ratings. Journal of the Royal Statistical Society Series a-Statistics in Society, 166, 279-300. doi: 10.1111/1467-985x.00278
25. Jayasinghe, U. W., Marsh, H. W., & Bond, N. (2006). A new reader trial approach to peer review in funding research grants:AnAustralian experiment. Scientometrics, 69(3), 591-606.
26. King, D. A. (2004). The scientific impact of nations. Nature, 430(6997), 311-316. doi: 10.1038/430311a
27. Maroco, J. (2007). Análise Estatística-Com utilização do SPSS (3rd ed.): Edições Sílabo, LDA.
28. Marsh, H. W., Bonds, N. W., & Jayasinghe, U. W. (2007). Peer review process: Assessments by applicant-nominated referees are biased, inflated, unreliable and invalid. Australian Psychologist, 42(1), 33-38. doi: 10.1080/00050060600823275
29. Marsh, H. W., Jayasinghe, U. W., & Bond, N. W. (2008). Improving the peer-review process for grant applications-Reliability, validity, bias, and generalizability. American Psychologist, 63(3), 160-168. doi: 10.1037/0003-066x.63.3.160
30. Moed, H.F. (2009, 6 Frebuary, 2013). Measuring contextual citation impact of scientific journals. from arXiv:0911.2632v1
31. Nederhof, A. J., & van Raan, A. F. J. (1987). Peer -review and bibliometric indicators of scientific performance-a comparison of cum laude doctorates with ordinary doctorates in physics. Scientometrics, 11(5-6), 333-350.
32. Norris, M., & Oppenheim, C. (2003). Citation counts and the Research Assessment Exercise V-Archaeology and the 2001 RAE. Journal of Documentation, 59(6), 709-730.
33. OECD. (2010). Performance-based Funding for Public Research in Tertiary Education Institutions: Workshop Proceedings
34. Papatheodorou, S. I., Trikalinos, T. A., & Ioannidis, J. P. A. (2008). Inflated numbers of authors over time have not been just due to increasing research complexity. Journal of Clinical Epidemiology, 61(6), 546-551. doi: 10.1016/j.jclinepi.2007.07.017
35. Pestana, M. H., & Gageiro, J.N. (1998). Análise de dados para ciências sociais -A complementariedade do SPSS (1a ed.). Lisboa: Edições Sílabo, Lda.
36. Reale, E., Barbara, A., & Costantini, A. (2007). Peer review for the evaluation of academic research: lessons from the Italian experience. Research Evaluation, 16(3), 216-228. doi: 10.3152/095820207x227501
37. Reuters, T. (2013). Journal Citations Reports. Retrieved 6 February, 2013, from http://admin-apps.webofknowledge.com/JCR/JCR?SID
38. Rinia, E. J., van Leeuwen, T. N., van Vuren, H. G., & van Raan, A. F. J. (1998). Comparative analysis of a set of bibliometric indicators and central peer review criteria-Evaluation of condensed matter physics in the Netherlands. Research Policy, 27(1), 95-107.
39. SCImago. (2007). SJR-SCImago Journal & Country Rank. Retrieved February 06, 2013, from http://www.scimagojr.com
40. SCImago. (2011). SIR-SCImago Institutions Rankings. Retrieved November 2011, from http://www.scimagoir.com/index.php
41. Taylor, J. (2011). The Assessment of Research Quality in UK Universities: Peer Review or Metrics? British Journal of Management, 22(2), 202-217. doi: 10.1111/j.1467-8551.2010.00722.x
42. Train, K. (2009). Discrete Choice Methods with Simulation (Second ed. Vol. 2009): Cambridge University Press.
43. Van Hooydonk, G. (1997). Fractional counting of multiauthored publications: Consequences for the impact of authors. Journal of the American Society for Information Science, 48(10), 944-945.
44. Van Raan, A. F. J. (2006). Comparison of the Hirsch-index with standard bibliometric indicators and with peer judgment for 147 chemistry research groups. Scientometrics, 67(3), 491-502. doi: 10.1556/ Scient.67.2006.3.10
45. Vieira, E.S., & Gomes, J.A.N.F. (2009). A comparison of Scopus and Web of Science for a typical university. Scientometrics, 81(2), 587-600. doi: 10.1007/s11192-009-2178-0
46. Vieira, E.S., & Gomes, J.A.N.F. (2011). An impact indicator for researchers. Scientometrics, 89(2), 607-629. doi: 10.1007/s11192-011-0464-0
47. Wagenmakers, E. J., & Farrell, S. (2004). AIC model selection using Akaike weights. Psychonomic Bulletin & Review, 11(1), 192-196. doi:10.3758/bf03206482
48. Weeks, W. B., Wallace, A. E., & Kimberly, B. C. S. (2004). Changes in authorship patterns in prestigious US medical journals. Social Science & Medicine, 59(9), 1949-1954. doi: 10.1016/j.socscimed.2004.02. 029
49. Wood, M., Roberts, M., & Howell, B. (2004). The reliability of peer reviews of papers on information systems. Journal of Information Science, 30(1), 2-11. doi: 10.1177/016551504041673