Sampling issues in bibliometric analysis

Journal of Informetrics

Volumn 10, Issue 4, 2016, Pages 1225-1232

  Williams, Richard ^a   Bornmann, Lutz ^b  

^a UNIVERSITY OF NOTRE DAME   (United States)

^b MAX PLANCK INSTITUTE   (Germany)

Author keywords

Bibliometrics; Percentiles; Population; Power analysis; Sampling

Indexed keywords

COMPUTER APPLICATIONS; SAMPLING;

BIBLIOMETRIC ANALYSIS; BIBLIOMETRICS; CONFIDENCE INTERVAL; INFERENTIAL STATISTICS; PERCENTILES; POPULATION; POWER ANALYSIS; SIGNIFICANCE TESTING;

STATISTICAL METHODS;

EID: 84999825401     PISSN: 17511577     EISSN: 18755879     Source Type: Journal    
DOI: 10.1016/j.joi.2015.11.004     Document Type: Article

References (29)

1. Behnke, J., Lassen sich Signifikanztests auf Vollerhebungen anwenden? Einige essayistische Anmerkungen. Politische Vierteljahresschrift 46:1 (2005), O1–O15, 10.1007/s11615-005-0240-y.
2. Berk, R.A., Western, B., Weiss, R.E., Reply to Bollen, Firebaugh, and Rubin. Sociological Methodology 25 (1995), 481–485.
3. Berk, R.A., Western, B., Weiss, R.E., Statistical inference for apparent populations. Sociological Methodology 25 (1995), 421–458.
4. Bielby, W. (2013). Managerial discretion, employment discrimination, and title VII class actions: is there life after dukes v. Wal-Mart? Talk given at the University of Notre Dame, November 1, 2013.
5. Bollen, K.A., Apparent and nonapparent significance tests. Sociological Methodology 25 (1995), 459–468.
6. Bornmann, L., The problem of citation impact assessments for recent publication years in institutional evaluations. Journal of Informetrics 7:3 (2013), 722–729 http://dx.doi.org/10.1016/j.joi.2013.05.002.
7. Bornmann, L., How are excellent (highly cited) papers defined in bibliometrics? A quantitative analysis of the literature. Research Evaluation 23:2 (2014), 166–173, 10.1093/reseval/rvu002.
8. Bornmann, L., Mutz, R., The advantage of the use of samples in evaluative bibliometric studies. Journal of Informetrics 7:1 (2013), 89–90, 10.1016/j.joi.2012.08.002.
9. Bornmann, L., Williams, R., How to calculate the practical significance of citation impact differences? An empirical example from evaluative institutional bibliometrics using adjusted predictions and marginal effects. Journal of Informetrics 7:2 (2013), 562–574, 10.1016/j.joi.2013.02.005.
10. Cameron, A.C., Trivedi, P.K., Microeconometrics using stata. 2010, Stata Press, College Station, TX.
11. Cohen, J., Statistical power analysis for the behavioral sciences. (2nd ed.), 1988, Lawrence Erlbaum Associates, Publishers, Hillsdale, NJ, USA.
12. Cronin, B., The citation process: The role and significance of citations in scientific communication. 1984, Taylor Graham, Oxford, UK.
13. Deming, W.E., Stephan, F.F., On the interpretation of censuses as samples. Journal of the American Statistical Association 36:213 (1941), 45–49, 10.2307/2278811.
14. Demings, W.E., Stephan, F.F., On the Interpretation of Censuses as Samples. Journal of the American Statistical Association, 36(213), 1941, 5.
15. Firebaugh, G., Will Bayesian inference help? A skeptical view. Sociological Methodology 25 (1995), 469–472.
16. Gelman, A. How does statistical analysis differ when analyzing the entire population rather than a sample? (2009). 〈http://andrewgelman.com/2009/07/03/how_does_statis/〉 Retrieved on 07.01.14.
17. Hicks, D., Wouters, P., Waltman, L., de Rijcke, S., Rafols, I., Bibliometrics: The Leiden manifesto for research metrics. Nature 520:7548 (2015), 429–431.
18. Levy, P., Lemeshow, S., Sampling of population—methods and applications. (4th ed.), 2008, Wiley, New York, NY, USA.
19. Manitoba Center for Health Policy. Concept: Population data and significance testing: A brief discussion. (2001). Retrieved from 〈http://mchp-appserv.cpe.umanitoba.ca/viewConcept.php?conceptID=1055〉 07.01.14.
20. Opthof, T., Leydesdorff, L., Caveats for the journal and field normalizations in the CWTS (Leiden) evaluations of research performance. Journal of Informetrics 4:3 (2010), 423–430.
21. Rubin, D.B., Bayes, Neyman, and calibration. Sociological Methodology 25 (1995), 473–479.
22. Schneider, J., Testing university rankings statistically: Why this is not such a good idea after all. Some reflections on statistical power, effect sizes, random sampling and imaginary populations. Archambault, E., Gingras, Y., Lariviere, V., (eds.) The 17th international conference on science and technology indicators, 2012, Repro-UQAM, Montreal, Canada, 719–732.
23. Schneider, J., Null hypothesis significance tests. A mix-up of two different theories: the basis for widespread confusion and numerous misinterpretations. Scientometrics 102:1 (2015), 411–432, 10.1007/s11192-014-1251-5.
24. Schneider, J.W., Caveats for using statistical significance tests in research assessments. Journal of Informetrics 7:1 (2013), 50–62 http://dx.doi.org/10.1016/j.joi.2012.08.005.
25. Schubert, A., Braun, T., Relative indicators and relational charts for comparative assessment of publication output and citation impact. Scientometrics 9:5–6 (1986), 281–291.
26. StataCorp. Stata power and sample size reference manual Release 13. 2013, Stata Press, College Station, Texas.
27. Waner, S., Costenoble, S.R. Calculus applied to probability and statistics. (1996). Retrieved from 〈http://www.zweigmedia.com/RealWorld/cprob/cprob3.html〉 08.01.14.
28. Williams, R., Bornmann, L., The substantive and practical significance of citation impact differences between institutions: Guidelines for the analysis of percentiles using effect sizes and confidence intervals. Ding, Y., Rousseau, R., Wolfram, D., (eds.) Measuring scholarly impact: Methods and practice, 2014, Springer, Heidelberg, Germany, 259–281.
29. Wilsdon, J., Allen, L., Belfiore, E., Campbell, P., Curry, S., Hill, S., Johnson, B. The metric tide: Report of the independent review of the role of metrics in research assessment and management. (2015). DOI: 10.13140/RG.2.1.4929.1363.