The bootstrap: A technique for data-driven statistics. Using computer-intensive analyses to explore experimental data

Clinica Chimica Acta

Volumn 359, Issue 1-2, 2005, Pages 1-26

  Henderson, A Ralph ^a,b  

^a UNIVERSITY OF WESTERN ONTARIO   (Canada)

^b NONE   (Canada)

Author keywords

Bootstrap; Computer intensive methods; Jackknife; Non parametric statistics; Permutation tests; Random number generation

Indexed keywords

ACCURACY; ANALYTICAL ERROR; BIOMEDICAL TECHNOLOGY ASSESSMENT; COMPUTER ANALYSIS; CONFIDENCE INTERVAL; CORRELATION ANALYSIS; DATA ANALYSIS; LINEAR REGRESSION ANALYSIS; MATHEMATICAL ANALYSIS; MEDICAL LITERATURE; MEDICAL RESEARCH; NONLINEAR REGRESSION ANALYSIS; PREDICTION; PRIORITY JOURNAL; PUBLICATION; REVIEW; STATISTICAL ANALYSIS; TECHNOLOGY; THEORETICAL STUDY;

COMPUTERS; DATA INTERPRETATION, STATISTICAL; STATISTICS, NONPARAMETRIC;

EID: 23644442557     PISSN: 00098981     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.cccn.2005.04.002     Document Type: Review

References (144)

1. D. Altman, and S.N. Goodman Transfer of technology from statistical journals to the biomedical literature. Past trends and future predictions JAMA 272 1994 129 132
2. B. Efron Bootstrap methods: another look at the jackknife Ann Stat 7 1979 1 26
3. M.R. Chernick Bootstrap methods A practitioner's guide 1999 John Wiley & Sons New York 1 264
4. R Development Core Team. R: A language and environment for statistical computing. http://www.R-project.org Accessed 10-19-2004.
5. R.R. Wilcox Introduction to robust estimation and hypothesis testing 1997 Academic Press San Diego 1 296
6. Harrell F, Alzola C. An Introduction to S and the Hmisc and Design libraries. http://www.biostat.mc.Vanderbilt.edu/twikj/pub/Main/RS/sintro.pdf Accessed 10-19-2004.
7. Insightful Corporation S-plus resample library user's manual 2002 Insightful Corporation Seattle (WA) 1 83
8. D.G. Altman, D. Machin, TN. Bryant, and M.J. Gardner Confidence interval analysis program 2000 British Medical Journal London
9. D. Altman D. Machin T.N. Bryant M.J. Gardner Statistics with confidence-confidence intervals and statistical guidelines 2nd ed. 2000 British Medical Journal London 1 240
10. Analyze-it for Microsoft Excel, Leeds. UK; 2003.
11. K. Linnet CBstat. A program for statistical analysis in clinical biochemistry 2004 Risskov, Denmark
12. T.R. Willemain Bootstrap on a shoestring: resampling using spreadsheets Am Stat 48 1994 40 42
13. I. Meineke An add-in implementation of the RESAMPLING syntax under Microsoft EXCEL Comput Methods Programs Biomed 63 2000 99 104
14. B.F.J. Manly Randomization, bootstrap and Monte Carlo methods in biology 2nd ed. 1997 Chapman & Hall London 1 399
15. D.J. Hand F. Daly A.D. Lunn K.J. McConway E. Ostrowski A handbook of small data sets 1994 Chapman & Hall London 1 458
16. E.K. Harris, and J.C. Boyd Statistical bases of reference values in laboratory medicine 1995 Marcel Dekker, Inc. New York 1 361
17. J.R. Beck, and E.K. Shultz The use of relative operating characteristic (ROC) curves in test performance evaluation Arch Pathol Lab Med 110 1986 13 20
18. W.J. Krzanowski An introduction to statistical modelling 1998 Arnold London 1 252
19. A.R. Henderson, J.A. Stark, M.J. McQueen, R.L. Patten, S. Krishna, and D.E. Wood Is determination of creative kinase-2 after electrophoretic separation accurate? Clin Chem 40 1994 177 183
20. A.R. Henderson, S. Krishnan, S. Webb, C.M. Cheung, D.J. Nazir, and H. Richardson Proficiency testing of creatine kinase and creatine kinase-2: the experience of the Ontario Laboratory Proficiency Testing Program Clin Chem 44 1998 124 133
21. D.A. Smith, F.Y. Leung, G. Jablonsky, and A.R. Henderson Determination, by radioimmunoassay, of the mass of lactate dehydrogenase isoenzyme one in human serum and of its rate of removal from serum after a myocardial infarction Clin Chem 33 1987 1863 1868
22. P. Sprent, and N.C. Smeeton Applied nonparametric statistical methods 3rd ed. 2001 Chapman & Hall/CRC Boca Raton 1 461
23. K. Linnet Two-stage transformation systems for normalization of reference distributions evaluated Clin Chem 33 1987 381 386
24. J.L. Simon, and P. Bruce Resampling: a tool for everyday statistical work Chance 4 1991 22 32
25. J.L. Simon, and P. Bruce The new biostatistics of resampling MD Comput 12 1995 115 121
26. P. Diaconis, and B. Efron Computer-intensive methods in statistics Sci Am 248 1983 116 130
27. B. Efron, and R. Tibshirani Statistical data analysis in the computer age Science 253 1991 390 395
28. P. Sprent Data driven statistical methods 1998 Chapman & Hall London 1 406
29. B. Efron, and R. Tibshirani An introduction to the bootstrap 1993 Chapman & Hall New York 1 436
30. B. Efron, and R. Tibshirani Bootstrap methods for standard errors, confidence intervals, and other measures of statistical accuracy Stat Sci 1 1986 54 77
31. B. Efron Better bootstrap confidence intervals J Am Stat Assoc 82 1987 171 200
32. J.G. Booth, and S. Sarkar Monte Carlo approximation of bootstrap variances Am Stat 52 1998 354 357
33. P. Hall The bootstrap and Edgeworth expansion 1992 Springer-Verlag New York 1 352
34. C.E. Lunneborg Data analysis by resampling: concepts and applications 2000 Duxbury Press Pacific Grove (CA) 1 568
35. A.C. Davison, and D.V. Hinkley Bootstrap methods and their application 1997 Cambridge University Press Cambridge 1 582
36. K. Diem J. Seldrup C. Lentner Geigy scientific tables. Volume 2: introduction to statistics, statistical tables and mathematical formulae- percentiles., volume 2 8th ed. 1982 Ciba-Geigy Limited Basle, Switzerland 197
37. International Federation of Clinical Chemistry Approved recommendation (1987) on the theory of reference values: Part 5. Statistical treatment of collected reference values. Determination of reference limits J Clin Chem Clin Biochem 25 1987 645 656
38. K. Linnet Nonparametric estimation of reference intervals by simple and bootstrap-based procedures [Technical brief] Clin Chem 46 2000 867 869
39. T. Hesterberg Tail-specific linear approximations for efficient bootstrap simulations J Comput Graph Stat 4 1995 113 133
40. H.A. David, and H.N. Nagaraja Order statistics 3rd ed. 2003 Wiley-Interscience Hoboken (NJ) 1 458
41. T.C. Hesterberg Weighted average importance sampling and defensive mixture distributions Technometrics 37 1995 185 194
42. B. Efron Nonparametric standard errors and confidence intervals Can J Stat 9 1981 139 172
43. Hesterberg T. Bootstrap tilting confidence intervals. http://www.insightful.com/Hesterberg/bootstrap/default.asp Accessed 8-30-2003.
44. R.G. Miller A trustworthy jackknife Ann Math Stat 35 1964 1594 1605
45. R.G. Miller The jackknife-a review Biometrika 61 1974 1 15
46. J.W. Tukey Bias and confidence in not-quite large samples (Abstract) Ann Math Stat 29 1958 614
47. S. Wasserman, and U. Bockenholt Bootstrapping: applications to psychophysiology Psychophysiology 26 1989 208 221
48. J.W. Tukey Exploratory data analysis 1977 Addison-Wesley Publishing Company Reading (MA) 1 688
49. B. Efron Jackknife-after-bootstrap standard errors and influence functions J R Stat Soc, B 54 1992 83 127
50. F.S. Roberts Applied combinatorics 1984 Prentice-Hall, Inc. Eaglewood Cliffs 1 606
51. A.R. Feinstein Principles of medical statistics 2002 Chapman & Hall/CRC Boca Raton (FL) 1 701
52. J.E. Gentle Random number generation and Monte Carlo methods 2nd ed. 2003 Springer-Verlag New York 1 381
53. D.E. Knuth Chapter 3: random numbers The art of computer programming: seminumerical algorithms 3rd ed. 1998 Addison-Wesley Reading (MA) 1 193
54. Marsaglia G. DIEHARD: a battery of tests of randomness. http://www.stat.fsu.edu/∼geo/diehard.html Accessed 7-10-2003.
55. National Institute for Standards and Technology. A statistical test suite for random and pseudorandom number generators for cryptographic applications. http://www.csrc.nist.gov/rng/ Accessed 7-10-2003.
56. L'Ecuyer P. TestU01. http://www.iro.umontreal.ca/∼lecuyer/ Accessed 7-10-2003.
57. Insightful Corporation S-Plus 6 for Windows guide to statistics vol. 2 2001 Insightful Corporation Seattle (WA) 1 621
58. B.D. Ripley Thoughts on pseudorandom number generators J Comput Appl Math 31 1990 153 163
59. B.D. McCullough Assessing the reliability of statistical software: I. Part Am Stat 52 1998 358 366
60. B.D. McCullough Assessing the reliability of statistical software: I.I. Part Am Stat 53 1999 149 159
61. L'Ecuyer P. Software for Uniform Random Number Generation: Distinguishing the Good and the Bad. http://www.iro.umontreal.ca/~lecuyer/papers.html Accessed 5-29-2004.
62. I.D. Hill 95% confidence limits for the median J Stat Comput Simul 28 1987 80 81
63. K.R. Nair Table of confidence interval for the median in samples from any continuous population Sankhya 4 1938 551 558
64. F.E. Harrell, and C.E. Davis A new distribution-free quantile estimator Biometrika 69 1982 635 640
65. P. Horn, and A.J. Pesce Reference intervals: an update Clin Chim Acta 334 2003 5 23 [Review]
66. W.J. Conover Chapter 6: statistics of the Kolmogorov-Smirnov type Practical nonparametric statistics 2nd ed. 1980 John Wiley & Sons New York 344 393
67. W.J. Conover Chapter 5: some methods based on ranks Practical nonparametric statistics 2nd ed. 1980 John Wiley & Sons New York 213 343
68. R.A. Fisher The design of experiments 8th ed. 1971 Hafner Publishing Company New York 1 248
69. E. Pitman Significance tests that may be applied to samples from any populations J R Stat Soc, Suppl 4 1937 119 130
70. E. Pitman Significance tests which may be applied to samples from any populations: II. The correlation coefficient test J R Stat Soc, Suppl 4 1937 225 232
71. E. Pitman Significance tests which may be applied to samples from any populations: III. The analysis of variance test Biometrika 29 1937 322 335
72. D. Salsburg Chapter 1: the lady drinking tea The lady tasting tea. How statistics revolutionized science in the Twentieth century 2002 AWH Freeman/Owl Book New York 1 8
73. S. Senn Chapter 3: trials of life Dicing with death: chance, risk, and health 2003 Cambridge University Press Cambridge 50 68
74. E.S. Edgington Randomization tests 3rd ed. 1995 Marcel Dekker, Inc. New York 1 409
75. E.W. Noreen Computer-intensive methods for testing hypothesis. An introduction 1989 John Wiley & Sons New York 1 229
76. M. Hollander, and D. Wolfe Nonparametric statistical methods 2nd ed. 1999 John Wiley & Sons, Inc. New York 1 787
77. G.E. Noether Sample size determination for some common nonparametric tests J Am Stat Assoc 82 1987 645 647
78. A. Hald Statistical theory with engineering applications 1952 John Wiley & Sons, Inc. New York 1 783
79. A. Stuart, J.K. Ord, and S. Arnold Kendall's advanced theory of statistics 6th ed. Classical inference and the linear model vol. 2A 1999 Arnold London 1 885
80. O. Kempthorne, and T.E. Doerfler The behaviour of some significance tests under experimental randomization Biometrika 56 1969 231 248
81. G.E. Noether On a theorem of Pitman Ann Math Stat 26 1955 64 68
82. W. Hoeffding The large-sample power of tests based on permutations of observations Ann Math Stat 23 1952 169 192
83. D. Altman Practical statistics for medical research 1991 Chapman and Hall London 1 611
84. P.J. Cornbleet, and N. Gochman Incorrect least-square regression coefficients in method-comparison analysis Clin Chem 25 1979 432 438
85. K. Linnet Estimation of the linear relationship between the measurements of two methods with proportional errors Stat Med 9 1990 1463 1473
86. W. Bablok, H. Passing, R. Bender, and B. Schneider A general regression procedure for method transformation. Application of linear regression procedures for method comparison studies in clinical chemistry: I.I.I. Part J Clin Chem Clin Biochem 26 1988 783 790
87. H. Passing, and W. Bablok Comparison of several regression procedures for method comparison studies and determination of sample sizes. Application of linear regression procedures for method comparison studies in Clinical Chemistry: I.I. Part J Clin Chem Clin Biochem 22 1984 431 445
88. H. Passing, and W. Bablok A new biometrical procedure for testing the equality of measurements from two different analytical methods. Application of linear regression procedures for method comparison studies in clinical chemistry: I. Part J Clin Chem Clin Biochem 21 1983 709 720
89. A. Krause, and M. Olsen The basics of S-PLUS 3rd ed. 2002 Springer-Verlag New York 1 419
90. R.D. Cook, and S. Weisberg Residuals and influence in regression 1982 Chapman & Hall New York 1 230
91. M. Bland An introduction to medical statistics 2nd ed. 1995 Oxford University Press Oxford 1 396
92. K. Diem J. Seldrup C. Lentner Geigy scientific tables. Volume 2: Introduction to statistics, statistical tables and mathematical formulae- z-Transformation 8 ed. 1982 Ciba-Geigy Basle 64 67
93. M.J. Crawley Statistical computing An introduction to data analysis using S-Plus 2002 John Wiley & Sons, Ltd. Chichester, England 1 761
94. M.J. Healy Statistical analysis of radioimmunoassay data Biochem J 130 1972 207 210
95. B. Nix, and D. Wild Chapter 9: data processing J.P. Gosling Immunoassays: a practical approach 2000 Oxford University Press Oxford 239 261
96. B.S. Everitt, and S. Rabe-Hesketh Analyzing medical data using S-Plus 2001 Springer-Verlag New York 1 485
97. M.S. Pepe The statistical evaluation of medical tests for classification nad prediction 2003 Oxford University Press Oxford 1 302
98. D. Bamber The area above the ordinal dominance graph and the area below the receiver operating graph J Math Psychol 12 1975 387 415
99. J.A. Hanley, and B.J. McNeil The meaning and use of the area under a receiver operating characteristic (ROC) curve Radiology 143 1982 29 36
100. D. Mossman Resampling techniques in the analysis of non-binormal ROC data Med Decis Mak 15 1995 358 366
101. X.H. Zhou, N.A. Obuchowski, and D.K. McClish Statistical methods in diagnostic medicine 2002 John Wiley & Sons, Inc. New York 1 437
102. Atkinson EJ, Mahoney D. ROC program in S-Plus. http://www.mayo.edu/hsr/ Sfunc.html Accessed 10-15-2004.
103. E.R. Delong, D.M. DeLong, and D.L. Clarke-Pearson Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach Biometrics 44 1988 837 845
104. K.O. Hajian-Tilaki, and J.A. Hanley Comparison of three methods for estimating the standard error of the area under the curve in ROC analysis of quantitative data Acad Radiol 9 2002 1278 1285
105. S.W. Greenhouse, and N. Mantel The evaluation of diagnostic tests Biometrics 6 1950 399 412
106. K. Linnet Comparison of quantitative diagnostic tests: type I error, power, and sample size Stat Med 6 1987 147 158
107. S. Wieand, M.H. Gail, B.R. James, and K.L. James A family of nonparametric statistics for comparing diagnostic markers with paired or unpaired data Biometrika 76 1989 585 592
108. H. Schäfer Efficient confidence bounds for ROC curves Stat Med 13 1994 1551 1561
109. K. Jensen, H.H. Müller, and H. Schäfer Regional confidence bands for ROC curves Stat Med 19 2000 493 509
110. R.W. Platt, J.A. Hanley, and H. Yang Bootstrap confidence intervals for the sensitivity of a quantitative diagnostic test Stat Med 19 2000 313 322
111. C.B. Begg, and R.A. Greenes Assessment of diagnostic tests when disease verification is subject to selection bias Biometrics 39 1983 207 215
112. D.F. Ransohoff, and A.R. Feinstein Problems of spectrum and bias in evaluating the efficacy of diagnostic tests N Engl J Med 299 1978 926 930
113. G.A. Young Bootstrap: more than a stab in the dark? Stat Sci 9 1994 382 415
114. J. Carpenter, and J. Bithell Bootstrap confidence intervals: when, which, what? A practical guide for medical statisticians Stat Med 19 2000 1141 1164
115. J.F. Walsh, and M. Reznikoff Bootstrapping: a tool for clinical research J Clin Psychol 46 1990 928 930
116. S. Pollack, P. Bruce, M. Borenstein, and J. Lieberman The resampling method of statistical analysis Psychopharmacol Bull 30 1994 227 234
117. J. Ludbrook Issues in biomedical statistics: comparing means by computer-intensive tests Aust N Z J Surg 65 1995 812 819 [Review]
118. W. Zhu Making bootstrap statistical inferences: a tutorial Res Q Exerc Sport 68 1997 44 55
119. D.A. Fiellin, and A.R. Feinstein Bootstraps and jackknives: new, computer-intensive statistical tools that require no mathematical theories J Invest Med 46 1998 22 26 [Review]
120. V.L. Landry An introduction to the biostatistical technique of resampling J Burn Care Rehabil 20 1999 1 6
121. R. Wehrens, H. Putter, and L.M.C. Buydens The bootstrap: a tutorial Chemom Intell Lab Syst 54 2000 35 52
122. E.H. Blackstone Breaking down barriers: helpful breakthrough statistical methods you need to understand better J Thorac Cardiovasc Surg 122 2001 430 439 [Review]
123. I. Pigeot The jackknife and bootstrap in biomedical research-common principles and possible pitfalls Drug Inf J 35 2001 1431 1443
124. G. Jones, M. Wortberg, S.B. Kreissig, B.D. Hammock, and D.M. Rocke Application of the bootstrap to calibration experiments Anal Chem 68 1996 763 770
125. A.H. Briggs, D.E. Wonderling, and C.Z. Mooney Pulling cost-effectiveness analysis up by its bootstraps: a non-parametric approach to confidence interval estimation Health Econ 6 1997 327 340 [Review]
126. A. Desgagné, A.M. Castilloux, J.F. Angers, and J. Lelorier The use of the bootstrap statistical method for the pharmacoeconomic cost analysis of skewed data PharmacoEconomics 13 1998 487 497 [Review]
127. P. Horn, A.J. Pesce, and B. Copeland A robust approach to reference interval estimation and evaluation Clin Chem 44 1998 622 631
128. C.D. Berweger, F. Müller-Plathe, E. Hänseler, and H. Keller Estimating imprecision profiles in biochemical analysis Clin Chim Acta 277 1998 107 125
129. D.G. Pitt, and D.P. Kreutzweiser Applications of computer-intensive statistical methods to environmental research Ecotoxicol Environ Saf 39 1998 78 97 [Review]
130. D.J. Pasta, J.L. Taylor, and J.M. Henning Probabilistic sensitivity analysis incorporating the bootstrap: an example comparing treatments for the eradication of Helicobacter pylori Med Decis Mak 19 1999 353 363
131. C. Couchoud, N. Pozet, M. Labeeuw, and C. Pouteil-Noble Screening early renal failure: cut-off values for serum creatinine as an indicator of renal impairment Kidney Int 55 1999 1878 1884
132. E.I. Lev, I. Tur-Kaspa, I. Ashkenazy, A. Reiner, D. Faraggi, J. Shemer, and Z. Argov Distribution of serum creatine kinase activity in young healthy persons Clin Chim Acta 279 1999 107 115
133. A.W. Partin, L.A. Mangold, D.M. Lamm, P.C. Walsh, J.I. Epstein, and J.D. Pearson Contemporary update of prostate cancer staging nomograms (Partin Tables) for the new millennium Urology 58 2001 843 848
134. F. Mutapi, and A. Roddam p values for pathogens: statistical inference from infectious-disease data Lancet Infect Dis 2 2002 219 230 [Review]
135. C.Z. Mooney, and R.D. Duval Bootstrapping A nonparametric approach to statistical inference 1993 SAGE Publications Newbury Park (CA) 1 72
136. J.L. Simon Resampling: the new statistics 2nd ed. 1997 Resampling Stats, Inc. Arlington (VA) 1 436
137. D.J. Sheskin Handbook of parametric and nonparametric statistical procedures 2nd ed. 2000 Chapman & Hall/CRC Boca Raton 1 982
138. P.I. Good Resampling methods A practical guide to data analysis 2nd ed. 2001 Birkhäuser Boston 1 238
139. W.N. Venables, and B.D. Ripley Modern applied statistics with S 4th ed. 2002 Springer-Verlag New York 1 495
140. P. Armitage, G. Berry, and J.N.S. Matthews Statistical methods in medical research 4th ed. 2002 Blackwell Science Ltd. Oxford 1 817
141. M.R. Chernick, and R.H. Friis Introductory biostatistics for health sciences Modern applications including bootstrap 2003 Wiley-Interscience Hoboken (NJ) 1 406
142. G. van Belle, L.D. Fisher, P.J. Heagerty, and T. Lumley Biostatistics: a methodology for the health sciences 2nd ed. 2004 Wiley Interscience New York 1 871
143. B. Efron The jackknife, the bootstrap and other resampling plans 1982 Society for Industrial and Applied Mathematics Philadelphia 1 92
144. J. Shao, and D. Tu The jackknife and bootstrap 1995 Springer-Verlag New York 1 516