How to calculate the practical significance of citation impact differences? An empirical example from evaluative institutional bibliometrics using adjusted predictions and marginal effects

Journal of Informetrics

Volumn 7, Issue 2, 2013, Pages 562-574

  Bornmann, Lutz ^a   Williams, Richard ^b  

^a MAX PLANCK INSTITUTE   (Germany)

^b UNIVERSITY OF NOTRE DAME   (United States)

Author keywords

Adjusted predictions at representative values; Average adjusted predictions; Average marginal effects; Evaluative bibliometrics; Highly cited papers; Marginal effects at representative values; Practical significance

Indexed keywords

INFORMATION SCIENCE; REGRESSION ANALYSIS; SOCIETIES AND INSTITUTIONS;

BIBLIOMETRICS; HIGHLY-CITED PAPERS; MARGINAL EFFECTS; PRACTICAL SIGNIFICANCE; REPRESENTATIVE VALUES;

FORECASTING;

EID: 84875823113     PISSN: 17511577     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.joi.2013.02.005     Document Type: Article

References (26)

1. Aldrich J.H., Nelson F.D. Linear probability, logit, and probit models 1984, Sage Publications, Newbury Park, CA, USA.
2. American Psychological Association Publication manual of the American Psychological Association 2009, American Psychological Association (APA), Washington, DC, USA. 6th ed.
3. Berry W.D., Feldman S. Multiple regression in practice 1985, Sage Publications, Thousand Oaks, CA, USA.
4. Bornmann L. How to analyse percentile citation impact data meaningfully in bibliometrics: The statistical analysis of distributions, percentile rank classes and top-cited papers. Journal of the American Society for Information Science and Technology 2013, 64(3):587-595.
5. Bornmann L., Daniel H.-D. What do citation counts measure? A review of studies on citing behavior. Journal of Documentation 2008, 64(1):45-80. 10.1108/00220410810844150.
6. Bornmann L., Leydesdorff L., Mutz R. The use of percentiles and percentile rank classes in the analysis of bibliometric data: opportunities and limits. Journal of Informetrics 2013, 7(1):158-165.
7. Bornmann L., Mutz R. The advantage of the use of samples in evaluative bibliometric studies. Journal of Informetrics 2013, 7(1):89-90. 10.1016/j.joi.2012.08.002.
8. Bornmann, L., Mutz, R., & Daniel, H.-D. (in press). A multilevel-statistical reformulation of citation-based university rankings: the Leiden Ranking 2011/2012. Journal of the American Society for Information Science and Technology.
9. Bornmann L., Mutz R., Marx W., Schier H., Daniel H.-D. A multilevel modelling approach to investigating the predictive validity of editorial decisions: do the editors of a high-profile journal select manuscripts that are highly cited after publication?. Journal of the Royal Statistical Society - Series A (Statistics in Society) 2011, 174(4):857-879. 10.1111/j.1467-985X. 2011.00689.x.
10. Garfield E. The history and meaning of the Journal Impact Factor. Journal of the American Medical Association 2006, 295(1):90-93.
11. Hardin J., Hilbe J. Generalized linear models and extensions 2012, Stata Corporation, College Station, Texas, USA.
12. Hosmer D.W., Lemeshow S. Applied logistic regression 2000, John Wiley & Sons, Inc., Chichester, UK. 2nd ed.
13. Levy P., Lemeshow S. Sampling of population - methods and applications 2008, Wiley, New York, NY, USA. 4th ed.
14. Long J.S., Freese J. Regression models for categorical dependent variables using Stata 2006, Stata Press, Stata Corporation, College Station, TX, USA. 2nd ed.
15. Lozano G.A., Larivière V., Gingras Y. The weakening relationship between the impact factor and papers' citations in the digital age. Journal of the American Society for Information Science and Technology 2012, 63(11):2140-2145. 10.1002/asi.22731.
16. Marx W. Special features of historical papers from the viewpoint of bibliometrics. Journal of the American Society for Information Science and Technology 2011, 62(3):433-439. 10.1002/asi.21479.
17. Mitchell M.N. Interpreting and visualizing regression models using Stata 2012, Stata Corporation, College Station, TX, USA.
18. Organisation for Economic Co-operation and Development. (2007). Revised field of science and technology (FOS) classification in the Frascati manual. Paris, France: Working Party of National Experts on Science and Technology Indicators, Organisation for Economic Co-operation and Development (OECD).
19. Rabe-Hesketh S., Everitt B. A handbook of statistical analyses using Stata 2004, Chapman & Hall/CRC, Boca Raton, FL, USA.
20. Schneider J.W. Caveats for using statistical significance tests in research assessments. Journal of Informetrics 2013, 7(1):50-62.
21. Schubert A., Braun T. Relative indicators and relational charts for comparative assessment of publication output and citation impact. Scientometrics 1986, 9(5-6):281-291.
22. SCImago Research Group SIR World Report 2012 2012, University of Granada, Granada, Spain.
23. van Raan A. Properties of journal impact in relation to bibliometric research group performance indicators. Scientometrics 2012, 92(2):457-469. 10.1007/s11192-012-0747-0.
24. Vinkler P. The evaluation of research by scientometric indicators 2010, Chandos Publishing, Oxford, UK.
25. Waltman L., Calero-Medina C., Kosten J., Noyons E.C.M., Tijssen R.J.W., van Eck N.J., et al. The Leiden Ranking 2011/2012: data collection, indicators, and interpretation. Journal of the American Society for Information Science and Technology 2012, 63(12):2419-2432.
26. Williams R. Using the margins command to estimate and interpret adjusted predictions and marginal effects. The Stata Journal 2012, 12(2):308-331.