Construction of a paper-and-pencil Test of Basic Scientific Literacy based on selected literacy goals recommended by the American Association for the Advancement of Science

Public Understanding of Science

Volumn 5, Issue 4, 1996, Pages 331-359

  Laugksch, Rüdiger C ^a   Spargo, Peter E ^a  

^a NONE

Author keywords

[No Author keywords available]

Indexed keywords

EID: 0002001527     PISSN: 09636625     EISSN: None     Source Type: Journal    
DOI: 10.1088/0963-6625/5/4/003     Document Type: Article

References (158)

1. See, for example, Arons, A. B., 1983, Achieving wider scientific literacy. Daedalus, 112(2), 91-122;
2. Bloch, E., 1986, Basic research and economic health - the coming challenge. Science, 232(4750), 595-599;
3. Hurd, P. DeH., 1989, Science education and the nation's economy. This Year in School Science. Scientific Literacy, edited by A. B. Champagne, B. E. Lovitts and B. J. Callinger (Washington, DC: AAA), pp. 15-40;
4. Prewitt, K., 1983, Scientific illiteracy and democratic theory. Daedalus, 112(2), 49-64;
5. Royal Society, 1985, The Public Understanding of Science (London: Royal Society);
6. Shortland, M., 1988, Advocating science: literacy and public understanding. Impact of Science on Society, 152, 305-316;
7. Walberg, H. J., 1983, Scientific literacy and economic productivity in international perspective. Daedalus, 112(2), 1-28.
8. For a contrary viewpoint, see Chapman, B., 1991, The overselling of science. School Science Review, 72(260), 47-63;
9. Shamos, M., 1988, The lesson every child need not Ieam. The Sciences, 28(4), 14-20.
10. American Association for the Advancement of Science, 1989, Science for all Americans (Washington, DC: AAAS).
11. National Science Teachers Association, 1992, Scope, Sequence and Coordination of Secondary School Science, Vol. 1: The Content Core (Washington, DC: National Science Teachers Association).
12. The methods and procedures used in the development of the scientific literacy test were outlined in Laugksch, R. C., and Spargo, P. E., 1993, Development of a scientific literacy test based on AAAS literacy goals: methods and procedures. Paper presented at the 1993 International Conference on the Public Understanding of Science and Technology, Chicago Academy of Sciences, Chicago, IL.
13. Laugksch, R. C., and Spargo, P. E., 1996, Development of a pool of scientific literacy test-items based on selected AAAS literacy goals. Science Education, 80(2), 121-143.
14. Berk, R. A., 1984, A Guide to Criterion-referenced Test Construction (Baltimore: Johns Hopkins University Press).
15. DeBoer, G. E., 1991, A History of Ideas in Science Education (New York: Teachers College Press).
16. Biker, D. R., 1991, A summary of research in science education - 1989, Scientific literacy. Science Education, 75(3), 330-333;
17. DeBoer, G. E., 1991, A History of Ideas in Science Education (New York: Teachers College Press);
18. Garfield, E., 1988, Science literacy, Part 2: Major research areas and recommendations for the future. Current Contents, 32(20), 3-11;
19. Jenkins, E. W., 1990, Scientific literacy and school science education. School Science Review, 71(256), 43-51;
20. Jenkins, E. W., 1994, Scientific literacy. The International Encyclopedia of Education: Vol. 9 (2nd edn), edited by T. Husen and T. N. Postlethwaite (Oxford: Pergamon Press), pp.5345-5350;
21. Roberts, D. A., 1983, Scientific Literacy: Towards a Balance for Setting Goals for School Science Programs (Ottawa, Canada: Minister of Supply and Services).
22. Roberts, D. A., 1983, Scientific Literacy: Towards a Balance for Setting Goals for School Science Programs (Ottawa, Canada: Minister of Supply and Services).
23. Jenkins, E. W., 1990, Scientific literacy and school science education. School Science Review, 71(256), 43-51;
24. Jenkins, E. W., 1992, School science education: towards a reconstruction. Journal of Curriculum Studies, 24(3), 229-246 .
25. See, for example, Shortland, M., 1988, Advocating science: literacy and public understanding. Impact of Science on Society, 152, 305-316.
26. Champagne, A. B., and Lovitts, B. E., 1989, Scientific literacy: a concept in search of definition. This Year in School Science, Scientific Literacy, edited by A. B. Champagne, B. E. Lovitts and B. J. Callinger (Washington, DC: AAAS), pp. 1-14.
27. Miller, J. D., 1983, Scientific literacy: a conceptual and empirical review. Daedalus, 112(2), 29-48.
28. For example, Arons, A. B., 1983, Achieving wider scientific literacy. Daedalus, 112(2), 91-122.
29. See, for example, Brickhouse, N. W., Ebert-May, D., and Wier, B. A., 1989, Scientific literacy: perspectives of school administrators, teachers, students, and scientists from an urban mid-Atlantic community. This Year in School Science. Scientific Literacy, edited by A. B. Champagne, B. E. Lovitts and B. J. Callinger (Washington, DC: AAAS), pp.157-176;
30. Laetsch, W. M., 1987, A basis for better understanding of science. Communicating Science to the Public, edited by D. Evered and M. O'Connor (London: Wiley), pp.1-10.
31. Miller, J. D., 1989, Scientific literacy. Paper presented at the annual meeting of the American Association for the Advancement of Science, San Francisco, CA.
32. Assessment of students' perceptions in this area did not commence until the 1950s, but since then a large body of literature has been established in this field. See Lederman, N. G., 1992, Students' and teachers' conceptions of the nature of science: a review of the research. Journal of Research in Science Teaching, 29(4), 331-359;
33. Meichtiy, Y. J., 1993, The impact of science curricula on student views about the nature of science. Journal of Research in Science Teaching, 30(5), 429-443.
34. Cooley, W. W., and Klopfer, L. E., 1961, Test on Understanding Science. Form W (Princeton, NJ: Educational Testing Service).
35. Rubba, P. A., and Anderson, H. O., 1978, Development of an instrument to assess secondary school students' understanding of the nature of scientific knowledge. Science Education, 62(4), 449-458.
36. Kimball, M. E., 1968, Understanding the nature of science: a comparison of scientists and science teachers. Journal of Research in Science Teaching, 5, 110-120.
37. Cooley W. W., and Klopfer, L. E., 1961, Test on Understanding Science. Form W (Princeton NJ: Educational Testing Service);
38. Kimball, M. E., 1968, Understanding the nature of science: a comparison of scientists and science teachers. Journal of Research in Science Teaching, 5, 110-120;
39. Rubba, P. A., and Anderson, H. O., 1978, Development of an instrument to assess secondary school students' understanding of the nature of scientific knowledge. Science Education, 62(4), 449-458.
40. See, for example, Carmichael, P., Driver, R., Holding, B., Twigger, D., and Watts, M., 1990, Research on Children's Conceptions in Science: a Bibliography (Leeds: Children's Learning in Science Research Group, Centre for Studies in Science and Mathematics Education, University of Leeds, UK);
41. Pfundt, H., and Duit, R., 1990, Bibliography: Students' Alternative Frameworks and Science Education (3rd edn) (Kiel: IPN);
42. Anonymous, 1989, Conference proceedings, edited paper collections, books and bibliographic sources on children's conceptions in science. International Journal of Science Education, 11(5), 597.
43. The STS movement is outlined, for example, by Yager, R. E., 1993, Science-Technology-Society as reform. School Science and Mathematics, 93(3), 145-151.
44. Aikenhead, G. S., and Ryan, A. G., 1992, The development of a new instrument: 'Views on Science-Technology-Society' (VOSTS). Science Education, 76(5), 477-491.
45. Aikenhead, G. S., and Ryan, A. G., 1992, The development of a new instrument: 'Views on Science-Technology-Society' (VOSTS). Science Education, 76(5), 480.
46. Lord, T. R., and Rauscher, C., 1991, A sampling of basic life science literacy in a college population. The American Biology Teacher, 53(7), 449-424.
47. Cannon, J., and Jinks, J., 1992, A cultural literacy approach to assessing general scientific literacy. School Science and Mathematics, 92(4), 196-200.
48. See Hirsch, E. D., Jr, 1988, Cultural Literacy: What Every American Needs to Know (New York: Vintage Books).
49. Laugksch, R. C., and Spargo, P. E., 1996, Development of a pool of scientific literacy test-items based on selected AAAS literacy goals. Science Education, 80(2), 121-143;
50. Miller, J. D., 1992, Toward a scientific understanding of the public understanding of science and technology. Public Understanding of Science, 1(1), 23-26.
51. Miller, J. D., 1987, Scientific literacy in the United States. Communicating Science to the Public, edited by D. Evered and M. O'Connor (London: Wiley), pp.19-40;
52. Miller, J. D., 1992, Toward a scientific understanding of the public understanding of science and technology. Public Understanding of Science, 1(1), 23-26.
53. Durant, J. R., Evans, G. A., and Thomas, G. P., 1989. The public understanding of science. Nature, 340(6228), 11-14;
54. Durant, J. R., Evans, G. A., and Thomas, G. P., 1992. The public understanding of science in Britain: the role of medicine in the popular representations of science. Public Understanding of Science, 1(2), 161-182.
55. INRA (Europe) and Report International, 1993, Europeans, Science and Technology. Report No. EUR15461 (Brussels: Commission of the European Communities, Directorate-General XIII)
56. Zhongliang Zhang and Jiansheng Zhang, 1993, A survey of public scientific literacy in China. Public Understanding of Science, 2(1), 21-38;
57. Ge. T., Liu, W., and Li, D., 1995. Public scientific literacy in China and international comparisons. Paper presented at the 1995 International Conference on Public Understanding of Science, Beijing, China.
58. Office of the Prime Minister of Japan, Public Relations Office, 1991, Opinion Survey on Science, Technology, and Society, translated by T. Welch (Washington, DC: Science Resources Studies Division, National Science Foundation),
59. cited by Zhongliang Zhang and Jiansheng Zhang, 1993. A survey of public scientific literacy in China. Public Understanding of Science, 2(1), 21-38.
60. Fjæstad, B., 1994, The 1994 Swedish survey on public understanding of science. Paper presented at the 1994 International Conference on the Public Understanding of Science and Technology, Science Museum, London. UK.
61. See references contained in Einsiedel. E. F., 1994, Mental maps of science: knowledge and attitudes among Canadian adults. International Journal of Public Opinion Research, 6(1), 35-44.
62. Bureau of Industry Economics, 1995, Science Understanding and Awareness: How Australians Rank in International Surveys. Occasional Paper 30 (Canberra: Australian Government Publishing Service).
63. Kim, H.-S., 1995, Does the Korean public understand science and technology? Paper presented at the 1995 International Conference on Public Understanding of Science, Beijing, China.
64. Miller, J. D., 1992, Toward a scientific understanding of the public understanding of science and technology. Public Understanding of Science, 1(1), 23-26.
65. Miller, J. D., 1993, Theory and measurement in the public understanding of science: a rejoinder to Bauer and Schoon. Public Understanding of Science. 2(3), 235-243.
66. See, for example, INRA (Europe) and Report International, 1993, Europeans, Science and Technology. Report No. EUR15461 (Brussels: Commission of the European Communities, Directorate-General XIII);
67. National Science Board, 1993, Science & Engineering Indicators - 1993 (NSB 93-1) (Washington, DC: US Government Printing Office).
68. Durant, J. R., Evans, G. A., and Thomas, G. P., 1989, The public understanding of science. Nature, 340(6228), 11-14.
69. Miller, J. D., 1993, Theory and measurement in the public understanding of science: a rejoinder to Bauer and Schoon. Public Understanding of Science, 2(3), 235-243.
70. For details of these questions, see INRA (Europe) and Report International, 1993, Europeans, Science and Technology. Report No. EUR15461 (Belgium: Commission of the European Communities, Directorate-General XIII);
71. National Science Board, 1993, Science & Engineering Indicators - 1993 (NSB 93-1) (Washington, DC: US Government Printing Office).
72. National Science Board, 1989, Science & Engineering Indicators - 1989 (NSB 89-1) (Washington, DC: US Government Printing Office);
73. National Science Board, 1991, Science & Engineering Indicators - 1991 (NSB 91-1) (Washington, DC: US Government Printing Office).
74. Doran, R. L., Lawrenz, F., and Helgeson, S., 1994, Research on assessment in science. Handbook of Research on Science Teaching and Learning, edited by D. L. Gabel (Toronto: Macmillan), p.394.
75. See, for example, Champagne, A. B., and Newell, S. T., 1992, Directions for research and development: alternative methods of assessing scientific literacy. Journal of Research in Science Teaching, 29(8), 841-860.
76. The approach to test students entering tertiary education has been chosen for practical and for conceptual reasons. Firstly, for a number of reasons it is less of a logistical tour de force to test South African students at tertiary educational institutions than at high-school. Secondly, and more importantly, high-school leavers at tertiary educational institutions represent a self-selected group of 'successful' high-school leavers. It is therefore not unreasonable to believe that this latter group is more likely to exhibit successful outcomes of secondary education, such as, for example, being minimally scientifically literate, than less 'successful' high-school leavers who do not enter tertiary education.
77. For example, it has been suggested on the basis of research that the importance of science topic knowledge in reasoning should not be underestimated. See Linn, M. C., Clement, C., Pulos, S., and Sullivan, P., 1989, Scientific reasoning during adolescence: the influence of instruction in science knowledge and reasoning strategies. Journal of Research in Science Teaching, 26(2), 171-187.
78. As Walberg writes, '. . . without . . . basic knowledge, or literacy . . . [students] cannot be expected to go on to the higher reaches of thought' (p.5); Walberg, H. J., 1983, Scientific literacy and economic productivity in international perspective. Daedalus, 112(2), 1-28.
79. For a discussion of criterion-referenced tests see, for example, Hambleton, R. K., and Rogers, H. J., 1991, Advances in criterion-referenced measurement. Advances in Educational and Psychological Testing: Theory and Applications, edited by R. K. Hambleton and J. N. Zaal (Boston, MA: Kluwer Academic), pp.3-42;
80. Nitko, A. J., 1984, Defining 'criterion-referenced test'. A Guide to Criterion-Referenced Test Construction, edited by R. A. Berk (Baltimore: Johns Hopkins University Press), pp.8-28.
81. Hambleton, R. K., and Rogers, H. J., 1991, Advances in criterion-referenced measurement. Advances in Educational and Psychological Testing: Theory and Applications, edited by R. K. Hambleton and J. N. Zaal (Boston, MA: Kluwer Academic), pp.3-42.
82. In norm referenced-tests, test scores are referenced internally, that is performance is interpreted with reference to a 'norm' group. In criterion-referenced tests, test scores are referenced externally, that is performance is interpreted with reference to clear attainment 'targets' for content specifications. See Hambleton, R. K., and Rogers, H. J., 1991, Advances in criterion-referenced measurement. Advances in Educational and Psychological Testing: Theory and Applications, edited by R. K. Hambleton and J. N. Zaal (Boston, MA: Kluwer Academic), pp.3-42.
83. Ebel, R. L., and Frisbie, D. A., 1991, Essentials of Educational Measurement (5th edn) (Englewood Cliffs, NJ: Prentice Hall).
84. These strategies are summarized in this paper in the section on item validity. Additional details are given in Laugksch, R. C., and Spargo, P. E., 1996, Development of a pool of scientific literacy test-items based on selected AAAS literacy goals. Science Education, 80(2), 121-143.
85. Doran, R. L., Lawrenz, F., and Helgeson, S., 1994, Research on assessment in science. Handbook of Research on Science Teaching and Learning, edited by D. L. Gabel (Toronto: Macmillan), pp.388-442.
86. See, for example, Champagne, A. B., and Newell, S. T., 1992, Directions for research and development: alternative methods of assessing scientific literacy. Journal of Research in Science Teaching, 29(8), 841-860;
87. Doran, R. L., Lawrenz, F., and Helgeson, S., 1994, Research on assessment in science. Handbook of Research on Science Teaching and Learning, edited by D. L. Gabel (Toronto: Macmillan), pp.388-442;
88. Shavelson, R. J. (Guest Ed.), 1994, Performance assessment. International Journal of Educational Research, 21(3), 235-350.
89. The following chapters in SFAA were used in constructing the item pools for Miller's dimensions of scientific literacy: 'The Nature of Science' for the nature-of-science dimension; 'The Physical Environment', 'The Living Environment' and 'The Human Organism' for the science-content knowledge dimension; and 'The Nature of Technology' for the impact-of-science-and-technology-on-society dimension. The science content areas covered by the various SFAA chapters corresponding to the science-content dimension are the earth/space and physical/chemical sciences, the life sciences, and the health sciences, respectively. For further details see Laugksch, R. C., and Spargo, P. E., 1996, Development of a pool of scientific literacy test-items based on selected AAAS literacy goals. Science Education, 80(2), 121-143.
90. Berk, R. A., 1984, Conducting the item analysis. A Guide to Criterion-referenced Test Construction, edited by R. A. Berk (Baltimore: Johns Hopkins University Press), pp.97-143.
91. South African technikons are tertiary educational institutions where the emphasis in teaching and research is placed more on the practical, rather than theoretical, aspects of numerous varied disciplines. Technikons are now widely regarded as equivalent in status to a university.
92. 2 analyses because of the students' very low total frequency (< 15).
93. There were no significant differences in the composition of the four pilot test-forms with respect to population group and gender for each of the 'instructed' and 'uninstructed' groups (unpublished data). Hence the data of the four pilot test-forms were combined for each criterion group.
94. Wedman, I., 1973, Theoretical Problems in Construction of Criterion-referenced Tests. Educational Reports, Umea No. 3 (Umea: School of Education, Umea University, Sweden);
95. Wedman, I., 1974, On the evaluation of criterion-referenced tests. Contemporary Issues in Educational Testing, edited by H. F. Crombag and D. N. M. de Gruijter (The Hague: Mouton);
96. both cited by Berk, R. A., 1984, Conducting the item analysis. A Guide to Criterion-referenced Test Construction, edited by R. A. Berk (Baltimore: Johns Hopkins University Press), pp.97-143.
97. Laugksch, R. C., and Spargo, P. E., 1996, Development of a pool of scientific literacy test-items based on selected AAAS literacy goals. Science Education, 80(2), 121-143. For further details of item-objective congruence see the section in this paper on item validity.
98. Hambleton, R. K., 1984, Validating the test scores. A Guide to Criterion-referenced Test Construction, edited by R. A. Berk (Baltimore: Johns Hopkins University Press), pp. 199-230.
99. Berk, R. A., 1984. Conducting the item analysis. A Guide to Criterion-referenced Test Construction, edited by R. A. Berk (Baltimore: Johns Hopkins University Press), pp.97-143.
100. Once Fellows had identified key ideas of 'undisputed significance to science' in the entire SFAA chapter allocated to each of them, the Fellows were asked to place each idea into the following three categories of importance: 'very important', 'important', and 'relatively less important'. For each item, the number of ratings in the three categories were summed; the item's final importance rating was determined by the category with the highest number of 'votes'. Tied rankings resulted in the middle, that is 'important', category being chosen for that item.
101. The difficulty of finding students enrolled at tertiary educational institutions in South Africa who have taken no science subject at all in senior high-school was pointed out earlier.
102. Hambleton, R. K., 1984, Validating the test scores. A Guide to Criterion-referenced Test Construction, edited by R. A. Berk (Baltimore: Johns Hopkins University Press), pp.199-230;
103. Zeller, R. A., 1994, Validity. The International Encyclopedia of Education: Vol. II (2nd edn), edited by T. Husen and T. N. Postlethwaite (Oxford: Pergamon Press), pp.6569-6576.
104. Shepard, L. A., 1993, Evaluating test validity. Review of Educational Research, 19, 405-450.
105. This figure is an overestimate, as some reviewers, consultants, and advisors participated in more than one stage of the development of SFAA. See American Association for the Advancement of Science, 1989, Science for All Americans (Washington, DC: AAAS);
106. Laugksch, R. C., and Spargo, P. E., 1996, Development of a pool of scientific literacy test-items based on selected AAAS literacy goals. Science Education, 80(2), 121-143.
107. Hambleton, R. K., 1984, Validating the test scores. A Guide to Criterion-referenced Test Construction, edited by R. A. Berk (Baltimore: Johns Hopkins University Press), pp.199-230.
108. Berk, R. A., 1984, Selecting the index of reliability. A Guide to Criterion-referenced Test Construction, edited by R. A. Berk (Baltimore: Johns Hopkins University Press), pp.231-266;
109. Hambleton, R. K., 1984, Validating the test scores. A Guide to Criterion-referenced Test Construction, edited by R. A. Berk (Baltimore: Johns Hopkins University Press), pp.199-230.
110. Further details of the judgemental review procedure are given in Laugksch, R. C., and Spargo, P. E., 1996, Development of a pool of scientific literacy test-items based on selected AAAS literacy goals. Science Education, 80(2), 121-143.
111. Zeller, R. A., 1994, Validity. The International Encyclopedia of Education: Vol. 11 (2nd edn), edited by T. Husen and T. N. Postlethwaite (Oxford: Pergamon Press), p.6572.
112. American Association for the Advancement of Science, 1993, Benchmarks for Science Literacy (New York: Oxford University Press).
113. American Association for the Advancement of Science, 1993, Benchmarks for Science Literacy (New York: Oxford University Press);
114. Ahlgren, A., 1993, Creating benchmarks for science education. Educational Leadership, 50(5), 46-49.
115. Shepard, L. A., 1984, Setting performance standards. A Guide to Criterion-referenced Test Construction, edited by R. A. Berk (Baltimore: Johns Hopkins University Press), pp.169-198.
116. This professional science association has recently extended its scope and has been renamed the Southern African Institute of Ecologists and Environmental Scientists.
117. Shepard, L. A., 1984, Setting performance standards. A Guide to Criterion-referenced Test Construction, edited by R. A. Berk (Baltimore: Johns Hopkins University Press), pp.169-198;
118. Berk, R. A., 1986, A consumer's guide to setting performance standards on criterion-referenced tests. Review of Educational Research, 56(1), 137-172.
119. Angoff, W. H., 1971, Scales, norms, and equivalent scores. Educational Measurement (2nd edn), edited by R. L. Thomdike (Washington, DC: American Council on Education), pp.508-600;
120. cited by Shepard, L. A., 1984, Setting performance standards. A Guide to Criterion-referenced Test Construction, edited by R. A. Berk (Baltimore: Johns Hopkins University Press), pp.169-198.
121. Scientific literacy is defined as 'a desired level of depth and breadth of scientific understanding appropriate to the interests and needs of the person being taught, set within the context of the developmental, educational, economic and political needs and interests of a country at a given point in time'. This definition is adapted from Champagne, A. B., and Lovitts, B. E., 1989, Scientific literacy: a concept in search of definition. This Year in School Science. Scientific Literacy, edited by A. B. Champagne. B. E. Lovitts and B. J. Callinger (Washington, DC: AAAS), p. 12.
122. In almost every content area there were some test-items for which the probability assigned by different judges were observed to vary widely. The median is less sensitive than the mean to large fluctuations in the value of a variable, and consequently the median rather than the mean was used in the determination of the cut-off score per content area. See Zar, J. H., 1984, Biostatistical Analysis (2nd edn) (Englewood Cliffs, NJ: Prentice-Hall).
123. The lower end-point of the two-sided 95 per cent confidence interval around the median overall cut-off score for each area, rounded downwards to the nearest integer, was used in the calculation of the performance standard. For details of the calculation of the confidence interval around the median see Zar, J. H., 1984, Biostatistical Analysis (2nd edn) (Englewood Cliffs, NJ: Prentice-Hall).
124. 20. See Ebel, R. L., and Frisbie, D. A., 1991, Essentials of Educational Measurement (5th edn) (Englewood Cliffs, NJ: Prentice Hall).
125. This method is described by Zieky, M. J., and Livingston, S. A., 1977, Manual for Setting Standards on the Basic Skills Assessment Tests (Princeton, NJ: Educational testing Service);
126. cited by Shepard, L. A., 1984, Setting performance standards. A Guide to Criterion-referenced Test Construction, edited by R. A. Berk (Baltimore: Johns Hopkins University Press), pp.169-198.
127. 2 analysis, and hence the Fisher's Exact Test was used. See Everett, B. S., 1992, The Analysis of Contingency Tables (2nd edn) (London: Chapman & Hall).
128. In this paper, the Yates continuity correction is used for the analysis of all 2x2 contingency tables.
129. The term 'Black' refers to persons classified as African, Coloured or Indian.
130. The majority of African students are English Second Language speakers, and this difference may indicate that the TBSL is potentially not language-robust.
131. Berk, R. A., 1984, Selecting the index of reliability. A Guide to Criterion-referenced Test Construction, edited by R. A. Berk (Baltimore: Johns Hopkins University Press), pp.231-266.
132. Hambleton, R. K., and Novick, M. R., 1973, Toward an integration of theory and method for criterion-referenced tests. Journal of Educational Measurement, 10, 159-170.
133. 20 = k/k - 1 [1- Σp(1 - p)/V] where k is the number of test-items, p is the proportion of students answering a question correctly, and V is the variance of total lest scores. For further details see Livingston, S. A., 1988, Reliability of test results. Educational Research, Methodology, and Measurement, edited by J. P. Keeves (Oxford: Pergamon Press), pp.386-392.
134. Taken together, however, the three subtests measure scientific literacy.
135. Thomdike, R. L., 1990, Reliability. The International Encyclopedia of Educational Evaluation, edited by H. J. Walberg and G. D. Haertel (Oxford: Pergamon Press), pp.260-273.
136. Berk, R. A., 1984, Selecting the index of reliability. A Guide to Criterion-referenced Test Construction, edited by R. A. Berk (Baltimore: Johns Hopkins University Press), pp.231-266;
137. Carmines, E. G., and Zeller, R. A., 1979, Reliability and Validity Assessment. Quantitative Applications in the Social Sciences Series No. 17 (Beverly Hills, CA: Sage).
138. For further details see Berk, R. A., 1984, Selecting the index of reliability. A Guide to Criterion-referenced Test Construction, edited by R. A. Berk (Baltimore: Johns Hopkins University Press), pp.231-266.
139. 2 is the variance of the test scores. See Berk, R. A., 1984, Selecting the index of reliability. A Guide to Criterion-referenced Test Construction, edited by R. A. Berk (Baltimore: Johns Hopkins University Press), pp.231-266;
140. Lord, F. M., and Novick, M. R., 1968, Statistical theories of Mental Test Scores (Reading, MA: Addison-Wesley), p.251.
141. Unpublished data.
142. Carmines, E. G., and Zeller, R. A., 1979, Reliability and Validity Assessment. Quantitative Applications in the Social Sciences Series No. 17 (Beverly Hills, CA: Sage).
143. The measurement coefficient of variation expresses an estimate of the 'true' standard deviation of a subtest score (i.e., the standard error of measurement) as a percentage of the relevant mean subtest score.
144. Berk, R. A., 1984. Selecting the index of reliability. A Guide to Criterion-referenced Test Construction, edited by R. A. Berk (Baltimore: Johns Hopkins University Press), pp.231-266;
145. Subkoviak, M. J., 1984. Estimating the reliability of master-nonmastery classification. A Guide to Criterion-referenced Test Construction, edited by R. A. Berk (Baltimore: Johns Hopkins University Press), pp.267-291.
146. Berk, R. A., 1984, Selecting the index of reliability. A Guide to Criterion-referenced Test Construction, edited by R. A. Berk (Baltimore: Johns Hopkins University Press), pp.231-266;
147. Hambleton, R. K., and Rogers, H. J., 1991, Advances in criterion-referenced measurement. Advances in Educational and Psychological Testing: Theory and Applications, edited by R. K. Hambleton and J. N. Zaal (Boston, MA: Kluwer Academic), pp.3-42.
148. Peng, C.-Y., and Subkoviak, M. J., 1980, A note on Huynh's normal approximation procedure for estimating criterion-referenced reliability. Journal of Educational Measurement, 17, 359-368;
149. Huynh, H., 1976, On the reliability of decisions in domain-referenced testing. Journal of Educational Measurement, 13, 253-264,
150. cited by Subkoviak, M. J., 1984, Estimating the reliability of master-nonmastery classification. A Guide to Criterion-referenced Test Construction, edited by R. A. Berk (Baltimore: Johns Hopkins University Press), pp.267-291.
151. Subkoviak, M. J., 1984, Estimating the reliability of master-nonmastery classification. A Guide to Criterion-referenced Test Construction, edited by R. A. Berk (Baltimore: Johns Hopkins University Press), pp.267-291.
152. 0 occurs only if all individuals are consistently classified.
153. Berk, R. A., 1984, Selecting the index of reliability. A Guide to Criterion-referenced Test Construction, edited by R. A. Berk (Baltimore: Johns Hopkins University Press), pp.231-266;
154. Hambleton, R. K., and Rogers, H. J., 1991, Advances in criterion-referenced measurement. Advances in Educational and Psychological Testing: Theory and Applications, edited by R. K. Hambleton and J. N. Zaal (Boston, MA: Kluwer Academic), pp.3-42.
155. Participating students at the five tertiary educational institutions came from a variety of educational and socio-economic backgrounds (unpublished data), and were registered for first degrees and diplomas in commerce and management, in the human and natural sciences, and in engineering.
156. Plans are afoot to further refine this test. Envisaged work includes factor analysis of the test-items, as well as interviewing respondents to explore, firstly, how they understood the questions, and, secondly, what they had in mind when they gave their answers.
157. See, for example, Laetsch (1987) and Shortland (1988) for a discussion of arguments for the promotion of scientific literacy. Laetsch, W. M., 1987. A basis for better understanding of science. Communicating Science to the Public, edited by D. Evered and M. O'Connor (London: Wiley), pp.1-10;
158. Shortland, M., 1988, Advocating scientific literacy: literacy and public understanding. Impact of Science on Society, 152, 305-316.