Exploring the relationship between experienced students' preference for open- and closed-book examinations, approaches to learning and achievement

Educational Research and Evaluation

Volumn 19, Issue 4, 2013, Pages 271-296

  Karagiannopoulou, Evangelia ^a   Milienos, Fotios S ^a  

^a UNIVERSITY OF IOANNINA   (Greece)

Author keywords

achievement; approaches to learning; ASSIST subscales; open and closed book examination; preference

Indexed keywords

EID: 84876971123     PISSN: 13803611     EISSN: 17444187     Source Type: Journal    
DOI: 10.1080/13803611.2013.765691     Document Type: Article

References (120)

1. Anderson, B., Lee, S., Simpson, M. and Stein, S. 2011. Study orchestrations in distance learning: Identifying dissonance and its implications for distance educators. The International Review of Research in Open and Distance Learning, 12: 1-17.
2. Baeten, M., Dochy, F. and Struyven, K. 2008. Students' approaches to learning and assessment preferences in a portfolio-based learning environment. Instructional Science, 36: 359-374. doi:10.1007/s11251-008-9060-y
3. Baeten, M., Kyndt, E., Struyven, K. and Dochy, F. 2010. Using student-centred learning environments to stimulate deep approaches to learning: Factors encouraging or discouraging their effectiveness. Educational Psychology Review, 5: 243-260. doi:10.1016/j.edurev.2010.06.001
4. Baillie, C. and Toohey, S. 1997. The "power test": Its impact on student learning in a material science course for engineering students. Assessment & Evaluation in Higher Education, 22: 33-48. http://dx.doi.org/10.1080/0260293970220103, (doi:10.1080/0260293970220103)
5. Betteridge, D. 1971. Open-book exams. Education in Chemistry, 8: 68-69.
6. Biggs, J. B. 1987. Student approaches to learning and studying., Hawthorn, Victoria, Australia: Australian Council for Educational Research (ACER.
7. Biggs, J. B. 1999. What the student does: Teaching for enhanced learning. Higher Education Research & Development, 18: 57-75. http://dx.doi.org/10.1080/0729436990180105, (doi:10.1080/0729436990180105)
8. Biggs, J. B. and Tang, C. 2011. Teaching for quality learning at university, 4, Buckingham, UK: Open University Press.
9. Birenbaum, M. 2007. Assessment and instruction preferences and their relationship with test anxiety and learning strategies. Higher Education, 53: 749-768. (doi:10.1007/s10734-005-4843-4)
10. Broyles, I. L, Cyr, P. R. and Korsen, N. 2005. Open book tests: Assessment of academic learning in clerkships. Medical Teacher, 27: 456-462. doi:10.1080/01421590500097075
11. Burnett, P. C. and Proctor, R. M. 2002. Elementary school students' learner self-concept, academic self-concepts and approaches to learning. Educational Psychology in Practice: Theory, research and practice in educational psychology, 18: 325-333. doi:10.1080/0266736022000022020
12. Byrne, B. M. 2010. Structural equation modeling with AMOS: Basic concepts, applications and programming, 2, New York, NY: Routledge/Taylor and Francis.
13. Byrne, M., Flood, B. and Willis, P. 1999. Approaches to learning: Irish students of accounting. Irish Accounting Review, 6: 1-29.
14. Byrne, M., Flood, B. and Willis, P. 2004a. Using the student learning framework to explore the variation in academic performance of European business students. Journal of Further and Higher Education, 28: 67-78. doi:10.1080/0309877032000161823
15. Byrne, M., Flood, B. and Willis, P. 2004b. Validation of the Approaches and Study Skills Inventory for Students (ASSIST) using accounting students in the USA and Ireland: A research note. Accounting Education: an International Journal, 13: 449-459. doi:10.1080/0963928042000306792
16. Caliński, T. and Harabasz, J. 1974. A dendrite method for cluster analysis. Communications in Statistics, 3: 1-27. doi:10.1080/03610927408827101
17. Cano, F. 2007. Approaches to learning and study orchestrations in high-school students. European Journal of Psychology of Education, 22: 131-151. doi: 10.1007/BF03173518
18. Cassidy, S. 2008. Approaches to learning and competitive attitude in students in higher education. The Psychology of Education Review, 32: 18-27.
19. Chamorro-Premuzic, T. and Furnham, A. 2008. Personality, intelligence and approaches to learning as predictors of academic performance. Personality and Individual Differences, 44: 1596-1603. doi:10.1016/j.paid.2008.01.003
20. Diseth, A. 2001. Validation of a Norwegian version of the Approaches and Study Skills Inventory for Students (ASSIST): Application of structural equation modelling. Scandinavian Journal of Educational Research, 45: 381-394. doi:10.1080/00313830120096789
21. Diseth, A. 2003. Personality and approaches to learning as predictors of academic achievement. European Journal of Personality, 17: 143-155. doi:10.1002/per.469
22. Diseth, A. 2007. Approaches to learning, course experience and examination grade among undergraduate psychology students: Testing of mediator effects and construct validity. Studies in Higher Education, 32: 373-388. doi:10.1080/03075070701346949
23. Diseth, A. and Martinsen, O. 2003. Approaches to learning, cognitive style, and motives as predictors of academic achievement. Educational Psychology, 23: 195-207. doi:10.1080/01443410303225
24. Diseth, A., Pallesen, S., Brunborg, G. and Larsen, S. 2010. Academic achievement among first semester undergraduate psychology students: The role of course experience, effort, motives and learning strategies. Higher Education, 59: 335-352. doi:10.1007/s10734-009-9251-8
25. Dolmans, D., Wolfhagen, I. and Ginns, P. 2010. Measuring approaches to learning in a problem-based learning context. International Journal of Medical Education, 1: 55-60. doi:10.5116/ijme.4c50.b666
26. Duda, R. O. and Hart, P. E. 1973. Pattern classification and scene analysis, New York, NY: John Wiley and Sons.
27. Duff, A., Boyle, E., Dunleavy, K. and Ferguson, J. 2004. The relationship between personality, approach to learning and academic performance. Personality and Individual Differences, 36: 1907-1920. doi:10.1016/j.paid.2003.08.020
28. Eilertsen, T. O. and Valdermo, O. 2000. Open-book assessment: A contribution to improved learning?. Studies in Educational Evaluation, 26: 91-103. (doi:10.1016/S0191-491X(00)00010-9)
29. Entwistle, N. J. 1997. " Contrasting perspectives on learning ". In The experience of learning. Implications for teaching and studying in higher education, 2, Edited by: Marton, F., Hounsell, D. and Entwistle, N. J. 3-22. Edinburgh, UK: Scottish Academic Press.
30. Entwistle, N. J. 2009. Teaching for understanding at university: Deep approaches and distinctive ways of thinking, Basingstoke, UK: Palgrave Macmillan.
31. Entwistle, N. J. and Entwistle, A. C. 1991. Contrasting forms of understanding for degree examinations: The student experience and its implications. Higher Education, 22: 205-227. doi:10.1007/BF00132288
32. Entwistle, N. J., McCune, V. and Walker, P. 2001. " Conceptions, styles, and approaches within higher education: Analytical abstractions and everyday experience ". In Perspectives on cognitive, learning and thinking styles, Edited by: Sternberg, R. J. and Zhang, L. F. 103-136. Mahwah, NJ: Lawrence Erlbaum Associates.
33. Entwistle, N. J. and Ramsden, P. 1983. Understanding student learning., London, UK: Croom Helm.
34. Entwistle, N. J. and Tait, H. 1990. Approaches to learning, evaluations of teaching, and preferences for contrasting academic environments. Higher Education, 19: 169-194. doi:10.1007/BF00137106
35. Entwistle, N. J., Tait, H. and McCune, V. 2000. Patterns of response to an "Approaches to Studying Inventory across contrasting groups and contexts". European Journal of Psychology of Education, 15: 33-48. doi:10.1007/BF03173165
36. Everitt, B. S., Landau, S., Leese, M. and Stahl, D. 2011. Cluster analysis, Chichester, UK: John Wiley and Sons.
37. Francis, J. C. 1982. A case for open-book examinations. Educational Review, 34: 13-26. doi:10.1080/0013191820340102
38. Furnham, A, Monsen, J. and Ahmetoglu, G. 2009. Typical educational engagement, Big Five personality traits, approaches to learning and cognitive ability predictors of academic performance. British Journal of Educational Psychology, 79: 769-782. doi:10.1348/978185409X412147
39. Gibbs, G. 1999. " Using assessment strategically to change the way students learn ". In Assessment matters in higher education: Choosing and using diverse approaches, Edited by: Brown, S. and Glasner, A. 41-53. Buckingham, UK: Open University Press.
40. Gijbels, D. and Dochy, F. 2006. Students' assessment preferences and approaches to learning: Can formative assessment make a difference?. Educational Studies, 32: 399-409. doi:10.1080/03055690600850354
41. Gijbels, D., Segers, M. and Struyf, E. 2008. Constructivist learning environments and the (im) possibility to change students' perceptions of assessment demands and approaches to learning. Instructional Science, 36: 431-443. (doi:10.1007/s11251-008-9064-7)
42. Gow, L. and Kember, D. 1990. Does higher education promote independent learning?. Higher Education, 19: 307-322. doi:10.1007/BF00133895
43. Hartigan, J. A. 1975. Clustering algorithms, New York, NY: John Wiley and Sons.
44. Heijne-Penniniga, M., Kuks, J., Hofman, A. and Cohen-Schotanus, J. 2008. Influence of open- and closed-book tests on medical students' learning approaches. Medical Education, 42: 967-974. doi:10.1111/j.1365-2923.2008.03125.x
45. Heikkilä, A. and Lonka, K. 2006. Studying in higher education: Students' approaches to learning, self-regulation, and cognitive strategies. Studies in Higher Education, 31: 99-117. doi:10.1080/03075070500392433
46. Honkimäki, S., Tynjälä, P. and Valkonen, S. 2004. University students' study orientations, learning experiences and study success in innovative courses. Studies in Higher Education, 29: 431-449. doi:10.1080/0307507042000236353
47. Horn, J. L. 1965. A rationale and test for the number of factors in factor analysis. Psychometrica, 30: 179-185. doi:10.1007/BF02289447
48. Hughes, J. and Peiris, R. 2006. " ASSISTing CS1 students to learn: Learning approaches and object-oriented programming ". In Proceedings of the 11th Annual SIGCSE Conference on Innovation and Technology in Computer Science Education, Edited by: Goldweber, M. and Salomoni, P. 275-279. New York, NY: ACM.
49. Ioannidou, M. K. 1997. Testing and life-long learning: Open-book and closed-book examination in a university course. Studies in Educational Evaluation, 23: 131-139. (doi:10.1016/S0191-491X(97)00008-4)
50. Johnson, R. A. and Wichern, D. W. 2002. " Upper Saddle River ". In Applied multivariate statistical analysis, NJ: Prentice Hall.
51. Kaiser, H. F. 1960. The application of electronic computers to factor analysis. Educational and Psychological Measurement, 20: 141-151. doi:10.1177/001316446002000116
52. Kaiser, H. F. 1974. An index of factorial simplicity. Psychometrika, 39: 31-36. doi:10.1007/BF02291575
53. Karagiannopoulou, E. 2002. Approaches to study reported by first and fourth year students in a School of Philosophy. Psychology: The Journal of the Hellenic Psychological Society, 11: 269-283. (in Greek).
54. Karagiannopoulou, E. 2006. The experience of revising for essay type examinations: Differences between first and fourth year University students. Higher Education, 51: 329-350. doi:10.1007/s10734-004-6383-8
55. Karagiannopoulou, E. 2010. Effects of classroom learning experiences and examination type on students' learning. Psychology: The Journal of the Hellenic Psychological Society, 17: 325-342.
56. Karagiannopoulou, E. 2011. Revisiting learning and teaching in higher education. A psychodynamic perspective. Psychodynamic practice, 17: 5-21. doi:10.1080/14753634.2011.539349
57. Karagiannopoulou, E. and Christodoulides, P. 2005. The impact of Greek University students' perceptions of their learning environment on approaches to studying and academic outcomes. International Journal of Educational Research, 43: 329-350. doi:10.1016/j.ijer.2006.05.002
58. Karagiannopoulou, E. and Christodoulides, P. 2009. " The Approaches and Study Skills Inventory for Students: Reliability and validity in a Greek university student sample ". In Scientific annals of the Psychological Society of Northern Greece, studies on psychological assessment, Edited by: Brouzos, A. and Misailides, P. Vol. 7, 217-243. Athens, Greece: Ellinika Grammata.
59. Karagiannopoulou, E. and Entwistle, N. 2012. Influences on the development of academic understanding: Approaches to learning, perceptions of assessment, and the "meeting of minds"., Manuscript submitted for publication
60. Karagiannopoulou, E. and Milienos, F. S. 2012. Experiences of the teaching-learning environment and approaches to learning. Testing the structure of the "Experiences of Teaching and Learning Questionnaire" in relation to earlier analyses, Manuscript submitted for publication
61. Kaufman, L. and Rousseeuw, P. J. 1990. Finding groups in data: An introduction to cluster analysis, New York, NY: John Wiley and Sons.
62. Kember, D., Jamieson, Q. W., Pomfret, M. and Wong, E. T.T. 1995. Learning approaches, study time and academic performance. Higher Education, 29: 329-343. doi:10.1007/BF01384497
63. Kline, R. B. 2011. Principles and practice of structural equation modeling, 3, New York, NY: The Guilford Press.
64. Koutselini, M. 1997. Contemporary trends and perspectives of the curricula: Towards a meta-modern paradigm for curriculum. Curriculum Studies, 5: 87-101. doi:10.1080/14681369700200005
65. Kreber, C. 2003. The scholarship of teaching: A comparison of conceptions held by experts and regular academic staff. Higher Education, 46: 93-121. doi:10.1023/A:1024416909866
66. Lawless, C. and Richardson, J. 2002. Approaches to studying and perceptions of academic quality in distance education. Higher Education, 44: 257-282. doi:10.1023/A:1016315114558
67. Lindblom-Ylänne, S. 2003. Broadening an understanding of the phenomenon of dissonance. Studies in Higher Education, 28: 63-77. doi:10.1080/03075070309306
68. Lindblom-Ylänne, S. and Lonka, K. 1999. Individual ways of interacting with the learning environment-are they related to study success?. Learning and Instruction, 9: 1-18. doi:10.1016/S0959-4752(98)00025-5
69. Lindblom-Ylänne, S. and Lonka, K. 2000. Individual study orchestrations of high achieving university students. European Journal of Educational Psychology, 15: 19-32. (doi:10.1007/BF03173164)
70. Lizzio, A., Wilson, K. and Simons, R. 2002. University students' perceptions of the learning environment and academic outcomes: Implications for theory and practice. Studies in Higher Education, 27: 27-52. doi:10.1080/03075070120099359
71. Magolda, B. B. and King, P. M. 2004. Learning partnerships: Theory and practice to educate for self-authorship., Sterling, VA: Stylus.
72. Marton, F. and Säljö, R. 1976. On qualitative differences in learning I-outcome and process. British Journal of Educational Psychology, 46: 4-11. doi:10.1111/j.2044-8279.1976.tb02980.x
73. Marton, F. and Säljö, R. 1997. " Approaches to learning ". In The experience of learning. Implications for teaching and studying in higher education, 2, Edited by: Marton, F., Hounsell, D. and Entwistle, N. J. 39-58. Edinburgh, UK: Scottish Academic Press.
74. McDowell, L. 1995. The impact of innovative assessment on students learning. Innovations in Education and Training International, 32: 302-313. http://dx.doi.org/10.1080/1355800950320402, (doi:10.1080/1355800950320402)
75. Meyer, J. H.F. 1991. Study orchestration: The manifestation, interpretation and consequences of contextualised approaches to studying. Higher Education, 22: 297-316. doi:10.1007/BF00132293
76. Meyer, J. H.F., Parsons, P. and Dunne, T. T. 1990. Individual study orchestrations and their association with learning outcome. Higher Education, 20: 67-89. (doi:10.1007/BF00162205)
77. Milligan, G. W. and Cooper, M. C. 1985. An examination of procedures for determining the number of clusters in a data set. Psychometrika, 50: 159-179. doi:10.1007/BF02294245
78. Newcombe, R. G. 1998. Two-sided confi{ligature}dence intervals for the single proportion; Comparison of several methods. Statistics in Medicine, 17: 857-872. doi:10.1002/(SICI)1097-0258(19980430)17:8<857::AID-SIM777>3.0.CO;2-E
79. Nunnally, J. C. and Bernstein, I. H. 1994. Psychometric theory, 3, New York, NY: McGraw-Hill.
80. Papakonstandinou, G. 2003. Education market., Athens, Greece: Metaixmio.
81. Papinczak, T. 2009. Are deep strategic learners better suited to PBL? A preliminary study. Advances in Health Sciences Education, 14: 337-353. doi:10.1007/s10459-008-9115-5
82. Parpala, A., Lindblom-Ylänne, S., Komulainen, E. and Entwistle, N. 2011. Assessing students' experiences of teaching-learning environments and approaches to learning: Validation of a questionnaire used in different countries and varying contexts. Learning Environments Research., Manuscript accepted for publication
83. Parpala, A., Lindblom-Ylänne, S., Komulainen, E., Litmanen, T. and Hirsto, L. 2010. Student's approaches o learning and their experiences of the teaching-learning environment in different disciplines. British Journal of Educational Psychology, 80: 269-282. doi:10.1348/000709909X476946
84. Pauker, J. D. 1974. Effect of open book examinations on test performance in an undergraduate child psychology course. Teaching of Psychology, 1: 71-73.
85. Ramsden, P. 2003. Learning to teach in higher education, London, UK: Routledge Falmer.
86. Raykov, T. and Marcoulides, G. A. 2006. A first course in structural equation modeling, 2, Mahwah, NJ: Lawrence Erlbaum Associates.
87. Richardson, J. T.E. 2005. Students' perceptions of academic quality and approaches to studying in distance education. British Journal of Educational Psychology, 31: 7-27. doi:10.1080/0141192052000310001
88. Richardson, J. T.E., Gamborg, G. and Hammerberg, G. 2005. Perceived academic quality and approaches to studying at Danish schools of occupational therapy. Scandinavian Journal of Occupational Psychology, 12: 110-117. doi:10.1080/11038120510030898
89. Richardson, J. T.E. and King, E. 1991. Gender differences in the experience of higher education: Quantitative and qualitative approaches. Educational Psychology, 11: 363-382. doi:10.1080/0144341910110311
90. Ross, S. M. 2010. Introductory statistics, 3, Amsterdam, The Netherlands: Academic Press/Elsevier.
91. Sadlo, G. and Richardson, J. T.E. 2003. Approaches to studying and perceptions of the academic environment in students following problem-based and subject-based curricula. Higher Education Research and Development, 22: 253-274. doi:10.1080/0729436032000145130
92. Sambell, K. and McDowell, L. 1998. The construction of the hidden curriculum: Messages and meanings in the assessment of student learning. Assessment and Evaluation in Higher Education, 23: 391-402. doi:10.1080/0260293980230406
93. Sambell, K., McDowell, L. and Brown, S. 1997. "But is it fair?" An exploratory study of student perceptions of the consequential validity of assessment. Studies in Educational Evaluation, 23: 349-371. (doi:10.1016/S0191-491X(97)86215-3)
94. Schumacher, C. F., Butzin, D. W., Finberg, L. and Burg, F. D. 1978. The effect of open- vs. closed-book testing on performance on a multiple-choice examination in pediatrics. Pediatrics, 61: 256-261.
95. Scouller, K. 1998. The influence of assessment method on students' learning approaches: Multiple choice question examination versus assignment essay. Higher Education, 35: 453-472. doi:10.1023/A:1003196224280
96. Segers, M., Gijbels, D. and Thurlings, M. 2008. The relationship between students' perceptions of portfolio assessment practice and their approaches to learning. Educational Studies, 34: 35-44. doi:10.1080/03055690701785269
97. Segers, M., Martens, R. and Van den Bossche, P. 2008. Understanding how a case-based assessment instrument influences student teachers' learning approaches. Teaching and Teacher Education, 24: 1751-1764. (doi:10.1016/j.tate.2008.02.022)
98. Seifert, T. L. 1996. The learning strategies of bilingual versus monolingual students. British Journal of Educational Psychology, 66: 73-82. (doi:10.1111/j.2044-8279.1996.tb01177.x)
99. Sianou-Kyrgiou, H. 2006. Education and social inequalities, Athens, Greece: Metaixmio.
100. Stryven, K., Dochy, F. and Janssens, S. 2003. " Student's perceptions about new modes of assessment in higher education ". In Optimising new modes of assessment: In search of quality and standards, Edited by: Segers, M., Dochy, F. and Cascallar, E. 171-224. Dordrecht, The Netherlands: Kluwer Academic Publishers.
101. Tait, H., Entwistle, N. J. and McCune, V. 1998. " ASSIST: A reconceptualisation of the Approaches to Studying Inventory ". In Improving student learning: Improving students as learners., Edited by: Rust, C. Oxford, UK: Oxford Centre for Staff and Learning Development.
102. Theophilides, C. and Dionysiou, O. 1996. The major functions of the open-book examination at the university level: A factor analytic study. Studies in Educational Evaluation, 22: 157-170. (doi:10.1016/0191-491X(96)00009-0)
103. Theophilides, C. and Koutselini, M. 2000. Study behavior in the closed-book and the open-book examination: A comparative analysis. Educational Research and Evaluation, 6: 379-393. doi:10.1076/edre.6.4.379.6932
104. Tibshirani, R., Walther, G. and Hastie, T. 2001. Estimating the number of clusters in a dataset via the gap statistic. Journal of the Royal Statistical Society, Series B, 63: 411-423. doi:10.1111/1467-9868.00293
105. Trigwell, K. and Prosser, M. 1991. Improving the quality of student learning: The influence of learning context and student approaches to learning on learning outcomes. Higher Education, 22: 251-266. doi:10.1007/BF00132290
106. Valadas, S. and Gonçalves, F. 2008. Academic success and competences for learning in Portuguese higher education students. A quantitative investigation, March-April Paper presented at the 4th International Conference on Higher Education, Barcelona, Spain
107. Valadas, S. C., Gonçalves, F. and Faisca, L. 2010. Approaches to studying in higher education Portuguese students: A Portuguese version of the approaches and study skills inventory for students. Higher Education, 59: 259-275. doi:10.1007/s10734-009-9246-5
108. Vanderburgh, P. 2005. Open-book tests and student-authored exam questions as useful tools to increase critical thinking. Advances in Physiology Education. 29: 183-184. doi:10.1152/advan.00024.2005
109. Van Prooijen, J. and Van der Kloot, W. 2001. Confirmatory analysis of exploratively obtained factor structures. Educational and Psychological Measurement, 61: 777-792. doi:10.1177/00131640121971518
110. Vermetten, Y. J., Vermunt, J. D. and Lodewijks, H. G. 1999. A longitudinal perspective on learning strategies in higher education-different view-points towards development. British Journal of Educational Psychology, 69: 221-242. doi:10.1348/000709999157699
111. Watkins, D. 1982. Identifying the study process dimensions of Australian university students. Australian Journal of Education, 26: 76-85.
112. Watkins, D. 1983. Assessing tertiary study processes. Human Learning, 2: 29-37.
113. Watkins, D. 1986. The approaches to learning of Australian tertiary students: A replication. Higher Education Research & Development, 5: 185-190. doi:10.1080/0729436860050207
114. Watkins, D. 2001. " Correlates of approaches to learning: A cross-cultural meta-analysis ". In Perspective on thinking, learning, and cognitive styles, Edited by: Sternberg, R. J. and Zhang, L. F. 165-195. Mahwah, NJ: Erlbaum.
115. Watkins, D. and Hattie, J. 1985. A longitudinal study of the approaches to learning of Australian tertiary students. Human Learning, 4: 127-141.
116. Wilson, K. L., Smart, R. M. and Watson, R. J. 1996. Gender difference in approach to learning in first-year psychology students. British Journal of Educational Psychology, 66: 59-71. (doi:10.1111/j.2044-8279.1996.tb01176.x)
117. Zeegers, P. 2001. Approaches to learning in science: A longitudinal study. British Journal of Educational Psychology, 71: 115-132. doi:10.1348/000709901158424
118. Zoller, U. and Ben-Chaim, D. 1988. Interaction between examination type, anxiety state, and academic achievement in college science; an action-oriented research. Journal of Research in Science Teaching, 26: 65-77. doi:10.1002/tea.3660260107
119. Zoller, U. and Ben-Chaim, D. 1990. Gender differences in examination-type preferences, test anxiety, and academic achievements in college science education-A case study. Science Education, 74: 597-608. (doi:10.1002/sce.3730740603)
120. Zoller, U., Ben-Chaim, D. and Kamm, S. 1997. Examination-type preferences of college science students and their faculty in Israel and USA: A comparative study. School Science and Mathematics, 97: 3-12. doi:10.1111/j.1949-8594.1997.tb17334.x