Tests examining skill outcomes in sport: A systematic review of measurement properties and feasibility

Sports Medicine

Volumn 44, Issue 4, 2014, Pages 501-518

  Robertson, Samuel J ^a   Burnett, Angus F ^b,c   Cochrane, Jodie ^c  

^a DEAKIN UNIVERSITY   (Australia)

^b CHINESE UNIVERSITY OF HONG KONG   (Hong Kong)

^c EDITH COWAN UNIVERSITY   (Australia)

Author keywords

[No Author keywords available]

Indexed keywords

ATHLETIC PERFORMANCE; CHECKLIST; HUMAN; MOTOR PERFORMANCE; PHYSIOLOGY; REPRODUCIBILITY; SPORT; TASK PERFORMANCE;

ATHLETIC PERFORMANCE; CHECKLIST; HUMANS; MOTOR SKILLS; REPRODUCIBILITY OF RESULTS; SPORTS; TASK PERFORMANCE AND ANALYSIS;

EID: 84897374281     PISSN: 01121642     EISSN: 11792035     Source Type: Journal    
DOI: 10.1007/s40279-013-0131-0     Document Type: Review

References (106)

1. Currell K, Jeukendrup AE. Validity, reliability and sensitivity of measures of sporting performance. Sports Med. 2008;38(4):297-316.
2. Cale AA. The investigation and analysis of softball skill tests for college women [dissertation]. Maryland: University of Maryland; 1962.
3. Cobb JW. The determination of the merits of selected items for the construction of a baseball skill test for boys of Little League age [dissertation]. Indiana: Indiana University; 1958.
4. Collins DR, Hodges PB. A comprehensive guide to sports skills tests and measurement. Maryland: Rowman & Littlefield Education; 2001.
5. Fein JT. Construction of Skill Tests for Beginning Collegiate Women Fencers [dissertation]. Iowa: University of Iowa; 1964.
6. Safrit MJ. Construction of skill tests for beginning fencers [dissertation]. Madison: University of Wisconsin; 1962.
7. Sopa AM. The construction of an indoor batting skills test for junior high school girls [dissertation]. Madison: University of Wisconsin; 1967.
8. Robertson SR, Burnett AF, Newton RU. Development and validation of the approach-iron skill test for use in golf. Eur J Sport Sci. Epub 2013 Jan 10.
9. Sunderland C, Cooke K, Milne H, et al. The reliability and validity of a field hockey skill test. Int J Sports Med. 2006;27(5):395-400.
10. Russell M, Kingsley M. Influence of exercise on skill proficiency in soccer. Sports Med. 2011;41(7):523-39.
11. Duncan MJ, Taylor S, Lyons M. The effect of caffeine ingestion on field hockey skill performance following physical fatigue. Res Sports Med. 2012;20(1):25-36.
12. Currell K, Conway S, Jeukendrup AE. Carbohydrate ingestion improves performance of a new reliable test of soccer performance. Int J Sport Nutr Exerc Metab. 2009;19(1):34-46.
13. Bottoms LM, Hunter AM, Galloway SD. Effects of carbohydrate ingestion on skill maintenance in squash players. Eur J Sport Sci. 2006;6(3):187-95.
14. Russell M, Benton D, Kingsley M. Influence of carbohydrate supplementation on skill performance during a soccer match simulation. J Sci Med Sport. 2012;15(4):348-54.
15. Russell M, Benton D, Kingsley M. The effects of fatigue on soccer skills performed during a soccer match simulation. Int J Sports Physiol Perform. 2011;6(2):221-33.
16. McKay B, Wulf G. A distal external focus enhances novice dart throwing performance. Int J Sport Exerc Psychol. 2012;10(2):149-56.
17. McCann P, Lavallee D, Lavallee R. The effect of pre-shot routines on golf wedge shot performance. Eur J Sport Sci. 2001;1(5):1-10.
18. Russell M, Rees G, Benton D, et al. An exercise protocol that replicates soccer matchplay. Int J Sports Med. 2011;32(7):511-8.
19. Nicholas CW, Nuttall FE, Williams C. The Loughborough Intermittent Shuttle Test: a field test that simulates the activity pattern of soccer. J Sports Sci. 2000;18(2):97-104.
20. 90): validity and reliability of a 90-minute soccer performance test. J Strength Cond Res. 2010;24(12):3209-18.
21. Robertson SJ, Burnett AF, Newton RU, et al. Development of the Nine-Ball Skills Test to discriminate elite and high-level amateur golfers. J Sports Sci. 2012;30(5):431-7.
22. Gabbett TJ, Georgieff B. The development of a standardized skill assessment for junior volleyball players. Int J Sports Physiol Perform. 2006;1(2):95-107.
23. Pyke F. Introduction. In: Gore C, editor. Physiological tests for elite athletes. Champaign, Illinois: Human Kinetics; 2000. p. xii-xiv.
24. Russell M, Benton D, Kingsley M. Reliability and construct validity of soccer skills tests that measure passing, shooting, and dribbling. J Sports Sci. 2010;28(13):1399-408.
25. Gabbett T, Wake M, Abernethy B. Use of dual task methodology for skill assessment and development: examples from rugby league. J Sports Sci. 2011;29(1):7-18.
26. Brent S, Draper N, Hodgson C, et al. Development of a performance assessment tool for rock climbers. Eur J Sport Sci. 2009;9(3):159-67.
27. Ali A. Measuring soccer skill performance: a review. Scand J Med Sci Sports. 2011;21(2):170-83.
28. Lidor R, CôTé JE, Hackfort D. ISSP position stand: to test or not to test? The use of physical skill tests in talent detection and in early phases of sport development. Int J Sport Exerc Psychol. 2009;7(2):131-46.
29. Ali A, Williams C, Hulse M, et al. Reliability and validity of two tests of soccer skill. J Sports Sci. 2007;25(13):1461-70.
30. Kingsley M, Russell M, Benton D. Authors' response to letter to the editor: "The need for 'representative task design' in evaluating efficacy of skills tests in sport: a comment on Russell, Benton and Kingsley (2010)". J Sports Sci. 2012;30(16):1731-3.
31. Vilar L, Araújo D, Davids K, et al. The need for 'representative task design' in evaluating efficacy of skills tests in sport: a comment on Russell, Benton and Kingsley (2010). J Sports Sci. 2012;30(16):1727-30.
32. Philips E, Davids K, Renshaw I, Portus M. Expert performance in sport and the dynamics of talent development. Sports Med. 2010;40(4):271-83.
33. Vaeyens R, Lenoir M, Williams MA, et al. Talent identification and development programmes in sport: current models and future directions. Sports Med. 2008;38(9):703-14.
34. McNamara A, Collins D. Comment on "Talent identification and promotion programmes of Olympic athletes". J Sports Sci. 2011;29(12):1353-6.
35. Pinder RA, Renshaw I, Davids K. The role of representative design in talent development: a comment on "Talent identification and promotion programmes of Olympic athletes". J Sports Sci. 2013;31(8):803-6.
36. Glazier P. Movement variability in the golf swing: theoretical, methodological and practical issues. Res Q Exerc Sport. 2011;82(2):157-61.
37. Fleisig G, Chu Y, Weber A, et al. Variability in baseball pitching biomechanics among various levels of competition. Sports Biomech. 2009;8(1):10-21.
38. Langdown BL, Bridge M, Li F. Movement variability in the golf swing. Sports Biomech. 2012;11(2):273-87.
39. Betzler NF, Monk SA, Wallace ES, et al. Variability in clubhead presentation characteristics and ball impact location for golfers' drives. J Sports Sci. 2012;30(5):439-48.
40. Bartlett R. Movement variability and its implication for sports scientists and practitioners: an overview. Int J Sports Sci Coach. 2008;3(1):113-24.
41. Ovens A, Smith W. Skill: making sense of a complex concept. J Phys Educ N Z. 2006;39(1):72-84.
42. Smith W. Skill acquisition in physical education: a speculative perspective. Quest. 2011;63(3):265-74.
43. Moher D, Liberati A, Tetzlaff J, et al. Reprint-preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Phys Ther. 2009;89(9):873-80.
44. Mokkink LB, Terwee CB, Patrick DL, et al. The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study. Qual Life Res. 2010;19(4):539-49.
45. Auld ML, Boyd RN, Moseley GL, et al. Tactile assessment in children with cerebral palsy: a clinimetric review. Phys Occup Ther Pediatr. 2011;31(4):413-39.
46. Fliess-Douer O, Vanlandewijck YC, Manor GL, et al. A systematic review of wheelchair skills tests for manual wheelchair users with a spinal cord injury: towards a standardized outcome measure. Clin Rehabil. 2010;24(10):867-86.
47. Helmerhorst HJ, Brage S, Warren J, et al. A systematic review of reliability and objective criterion-related validity of physical activity questionnaires. Int J Behav Nutr Phys Act. 2012;9(1):103-57.
48. Williams MA, McCarthy CJ, Chorti A, et al. A systematic review of reliability and validity studies of methods for measuring active and passive cervical range of motion. J Manip Physiol Ther. 2010;33(2):138-55.
49. Packham T, MacDermid JC, Henry J, et al. A systematic review of psychometric evaluations of outcome assessments for complex regional pain syndrome. Disabil Rehabil. 2012;34(13):1059-69.
50. Dobson F, Morris ME, Baker R, et al. Gait classification in children with cerebral palsy: a systematic review. Gait Posture. 2007;25(1):140-52.
51. Howe TE, Dawson LJ, Syme G, et al. Evaluation of outcome measures for use in clinical practice for adults with musculoskeletal conditions of the knee: a systematic review. Man Ther. 2012;17(4):100-18.
52. Mirkov D, Nedeljkovic A, Kukolj M, et al. Evaluation of the reliability of soccer-specific field tests. J Strength Cond Res. 2008;22(4):1046-50.
53. Knudson DV, Morrison CS. Qualitative analysis of human movement. 2nd ed. Illinois: Human Kinetics; 2002.
54. Streiner DL, Norman GR. Health measurement scales: a practical guide to their development and use. 3rd ed. Oxford: Oxford University Press; 2005.
55. Portney LG, Watkins MP. Foundations of clinical research: applications to practice. 3rd ed. New Jersey: Pearson/Prentice Hall; 2008.
56. Westen D, Rosenthal R. Quantifying construct validity: two simple measures. J Pers Soc Psychol. 2003;84(3):608-18.
57. Squires A. A valid step in the process: a commentary on Beckstead. Int J Nurs Stud. 2009;46(9):1284-5.
58. Braeken AP, Kempen GI, Eekers D, et al. The usefulness and feasibility of a screening instrument to identify psychosocial problems in patients receiving curative radiotherapy: a process evaluation. BMC Cancer. 2011;11(1):479-90.
59. Gabel CP, Melloh M, Burkett B, et al. Lower limb functional index: development and clinimetric propertie. Phys Ther. 2012;92(1):98-110.
60. Shea BJ, Grimshaw JM, Wells GA, et al. Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol. 2007;7(1):10.
61. Gardner MJ, Machin D, Campbell MJ, et al. Statistical checklists. In: Altman DG, Machin D, Bryant TN et al., editors. Statistics with confidence: confidence intervals and statistical guidelines. 2nd ed. Bristol: BMJ Books; 2000. p. 191-201.
62. Scientific Advisory Committee of the Medical Outcomes Trust. Assessing health status and quality-of-life instruments: attributes and review criteria. Qual Life Res. 2002;11(3):193-205.
63. Hill BE, Williams G, Bialocerkowski AE. Clinimetric evaluation of questionnaires used to assess activity after traumatic brachial plexus injury in adults: a systematic review. Arch Phys Med Rehabil. 2011;92(12):2082-9.
64. Solway S, Brooks D, Lacasse Y, et al. A qualitative systematic overview of the measurement properties of functional walk tests used in the cardiorespiratory domain. Chest. 2001;119(1):256-70.
65. Lohr KN, Aaronson NK, Alonso J, et al. Evaluating quality-of-life and health status instruments: development of scientific review criteria. Clin Ther. 1996;18(5):979-92.
66. Campion MA. Article review checklist: a criterion checklist for reviewing research articles in applied psychology. Pers Psychol. 2006;46(3):705-18.
67. Valderas JM, Ferrer M, Mendivil J, et al. Development of EMPRO: a tool for the standardized assessment of patient-reported outcome measures. Value Health. 2008;11(4):700-8.
68. Hopkins WG. Measures of reliability in sports medicine and science. Sports Med. 2000;30(1):1-15.
69. Beck TW. The importance of a priori sample size estimation in strength and conditioning research. J Strength Cond Res. 2013;27(8):2323-37.
70. Nevill AM, Holder RL, Cooper SM. Statistics, truth and error reduction in sport and exercise sciences. Eur J Sport Sci. 2007;7(1):9-14.
71. Cohen J. A. A coefficient of agreement for nominal scales. Educ Psychol Meas. 1960;20(1):37-46.
72. Haynes SN, Richard DC, Kubany ES. Content validity in psychological assessment: a functional approach to concepts and methods. Psychol Assess. 1995;7(3):238-47.
73. Beckstead JW. Content validity is naught. Int J Nurs Stud. 2009;46(9):1274-83.
74. Stinson JN, Kavanagh T, Yamada J, et al. Systematic review of the psychometric properties, interpretability of self-report pain measures for use in clinical trials in children and adolescents. Pain. 2006;125(1 & 2):143-57.
75. Ravens-Sieberer U, Auquier P, Erhart M, et al. The KIDSCREEN-27 quality of life measure for children and adolescents: psychometric results from a cross-cultural survey in 13 European countries. Qual Life Res. 2007;16(8):1347-56.
76. Munro BH. Statistical methods for health care research. 5th ed. Philadelphia (USA): Lippincott, Williams & Wilkins: 1999.
77. Van Saane N, Sluiter JK, Verbeek JH, et al. Reliability and validity of instruments assessing job satisfaction - a systematic review. Occup Med. 2003;53(3):191-200.
78. Beckerman H, Roebroeck ME, Lankhorst GJ, et al. Smallest real difference, a link between reproducibility and responsiveness. Qual Life Res. 2001;10(7):571-8.
79. Guyatt G, Walter S, Norman G. Measuring change over time: assessing the usefulness of evaluative instruments. J Chron Dis. 1987;40(2):171-8.
80. Hopkins WG. Probabilities of clinical or practical significance. Sportscience [serial online]. 2002. 6. http://sportsci.org/jour/0201/wghprob. htm. Accessed 11 Dec 2012.
81. Crossley KM, Bennell KL, Cowan SM, et al. Analysis of outcome measures for persons with patellofemoral pain: which are reliable and valid? Arch Phys Med Rehabil. 2004;85(5):815-22.
82. Yilla AB, Sherrill C. Validating the Beck battery of quad rugby skill tests. Adapt Phys Act Q. 1996;15(2):155-67.
83. Porter JM, Landin D, Hebert EP, et al. The effects of three levels of contextual interference on performance outcomes and movement patterns in golf skills. Int J Sports Sci Coach. 2007;2(3):243-55.
84. Lam ET, Zhang JJ. The development and validation of a racquetball skills test battery for young adult beginners. Meas Phys Educ Exerc Sci. 2002;6(2):95-126.
85. Bock-Jonathon BB, Venter RE, Bressan ES. A comparison between skill and decision-making ability of netball players at club level: pilot study. S Afr J Res Sport Phys Educ Rec. 2007;29(1):29-38.
86. Downs SB, Wood TW. Validating a special olympics volleyball skills assessment test. Adapt Phys Act Q. 1996;13(2):166-79.
87. De Groot S, Balvers IJ, Kouwenhoven SM, et al. Validity and reliability of tests determining performance-related components of wheelchair basketball. J Sports Sci. 2012;30(9):879-87.
88. Ali A, Foskett A, Gant N. Validation of a soccer skill test for use with females. Int J Sports Med. 2008;29(11):917-21.
89. Royal KA, Farrow D, Mujika I, et al. The effects of fatigue on decision making and shooting skill performance in water polo players. J Sports Sci. 2006;24(8):807-15.
90. Vergauwen L, Spaepen AJ, Lefevre J, et al. Evaluation of stroke performance in tennis. Med Sci Sports Exerc. 1998;30(8):1281-8.
91. Lemmink KA, Elferink-Gemser MT, Visscher C. Evaluation of the reliability of two field hockey specific sprint and dribble tests in young field hockey players. Br J Sports Med. 2004;38(2):138-42.
92. Bartlett J, Smith L, Davis K, et al. Development of a valid volleyball skills test battery. J Phys Educ Rec Dance. 1991;62(2):19-21.
93. Atkinson G, Nevill AM. Statistical methods for assessing measurement error (reliability) in variables relevant to sports medicine. Sports Med. 1998;26(4):217-38.
94. Atkinson G, Reilly T. Circadian variation in sports performance. Sports Med. 1996;21(4):292-312.
95. Beaton DE, Wright JG, Katz JN. Development of the QuickDASH: comparison of three item-reduction approaches. J Bone Joint Surg Am. 2005;87(5):1038-46.
96. Kilkens OJ, Post MW, Dallmeijer AJ, et al. Wheelchair skill tests: a systematic review. Clin Rehab. 2003;17(4):418-30.
97. Faber MJ, Bosscher RJ, van Wieringen PC. Clinimetric properties of the performance-orientated mobility assessment. Phys Ther. 2006;86(7):944-54.
98. Stevens B, Gibbins S. Clinical utility and clinical significance in the assessment and management of pain in vulnerable infants. Clin Perinatol. 2002;29(3):459-68.
99. Robinson JM, Cook JL, Purdam C, et al. The VISA-A questionnaire: a valid and reliable index of the clinical severity of Achilles tendinopathy. Br J Sports Med. 2001;35(5):335-41.
100. Baumgartner TA, Jackson AS. Measurement for evaluation in physical education and exercise science. 5th ed. Michigan: Brown & Benchmark; 1998.
101. Law MC, MacDermid J. Evidence-based rehabilitation: A guide to practice, Slack Incorporated, 2007 Outcome measures rating form. CanChild Centre for Childhood Disability Research. 2nd ed. Thorofare (NJ); 2004. p. 367.
102. de Boer MR, Moll AC, de Vet HC, et al. Psychometric properties of vision-related quality of life questionnaires: a systematic review. Ophthalmic Physiol Opt. 2004;24(4):257-73.
103. Landis J, Koch G. The measurement of observer agreement of categorical data. Biometrics. 1977;33(1):159-74.
104. Pietrobon R, Coeytaux RR, Carey TS, et al. Standard scales for measurement of functional outcome for cervical pain or dysfunction: a systematic review. Spine. 2002;27(5):515-22.
105. Copay AG, Subach BR, Glassman SD, et al. Understanding the minimum clinically important difference: a review of concepts and methods. Spine. 2007;7(5):541-6.
106. Streiner DL. A checklist for evaluating the usefulness of rating scales. Can J Psychiatry. 1993;38(2):140-8.