Do psychology researchers tell it like it is? A microgenetic analysis of research strategies and self-report accuracy along a continuum of expertise

Instructional Science

Volumn 38, Issue 4, 2010, Pages 395-415

  Feldon, David F ^a  

^a WASHINGTON STATE UNIVERSITY   (United States)

Author keywords

Automaticity; Expertise; Research methodology; Scientific problem solving; Self report

Indexed keywords

EID: 77953960054     PISSN: 00204277     EISSN: None     Source Type: Journal    
DOI: 10.1007/s11251-008-9085-2     Document Type: Article

References (73)

1. Adams, G. B., & White, J. D. (1994). Dissertation research in public administration and cognate fields: An assessment of methods and quality. Public Administration Review, 54(6), 565-576.
2. Allen, R. M., & Casbergue, R. M. (1997). Evolution of novice through expert teachers' recall: Implications for effective reflection on practice. Teaching and Teacher Education, 13(7), 741-755. doi: 10. 1016/S0742-051X(97)00018-8.
3. Anderson, J. R. (1993). Problem solving and learning. The American Psychologist, 48(1), 35-44. doi: 10. 1037/0003-066X. 48. 1. 35.
4. Anderson, J. R. (1995). ACT: A simple theory of complex cognition. The American Psychologist, 51, 355-365. doi: 10. 1037/0003-066X. 51. 4. 355.
5. Anzai, Y., & Yokoyama, T. (1984). Internal models in physics problem solving. Cognition and Instruction, 1, 397-450. doi: 10. 1207/s1532690xci0104_2.
6. Bainbridge, L. (1999). Verbal reports as evidence of the process operator's knowledge. International Journal of Human-Computer Studies, 51, 213-238. doi: 10. 1006/ijhc. 1979. 0307.
7. Baird, R. R. (2001). Experts sometimes show more false recall than novices: A cost of knowing too much. Learning & Individual Differences, 13(4), 349-355. doi: 10. 1016/S1041-6080(03)00018-9.
8. Barrett, L. F., Tugade, M. M., & Engle, R. W. (2004). Individual differences in working memory capacity and dual-process theories of mind. Psychological Bulletin, 130(4), 553-573. doi: 10. 1037/0033-2909. 130. 4. 553.
9. Blessing, S. B., & Anderson, J. R. (1996). How people learn to skip steps. Journal of Experimental Psychology: Learning, Memory, and Cognition, 22(3), 576-598. doi: 10. 1037/0278-7393. 22. 3. 576.
10. Chao, C., & Salvendy, G. (1994). Percentage of procedural knowledge acquired as a function of the number of experts from whom knowledge is acquired for diagnosis, debugging, and interpretation tasks. International Journal of Human-Computer Interaction, 6(3), 221-233.
11. Chen, Z., & Siegler, R. S. (2000). Across the great divide: Bridging the gap between understanding of toddlers' and older children's thinking. Monographs of the Society for Research in Child Development, Serial No. 261.
12. Chi, M. T. H., Glaser, R., & Rees, E. (1982). Expertise in problem solving. In R. J. Sternberg (Ed.), Advances in psychology of human intelligence (Vol. 1, pp. 7-75). Hillsdale, NJ: Erlbaum.
13. Chung, G. K. W. K., de Vries, F. L., Cheak, A. M., Stevens, R. H., & Bewley, W. L. (2002). Cognitive process validation of an online problem solving assessment. Computers in Human Behavior, 18, 669-684.
14. Clark, R. E., Feldon, D., van Merriënboer, J. J. G., Yates, K., & Early, S. (2008). Cognitive task analysis. In J. M. Spector, M. D. Merrill, J. J. G. van Merriënboer, & M. P. Driscoll (Eds.), Handbook of research on educational communications and technology (3rd ed., pp. 577-593). New York: Routledge.
15. Clement, J. (1988). Observed methods for generating analogies in scientific problem solving. Cognitive Science, 12(4), 563-586.
16. Collinson, J. (1998). Professionally trained researchers? Expectations of competence in social science doctoral research training. Higher Education Review, 31(1), 59-67.
17. Cooke, N. J. (1992). Modeling human expertise in expert systems. In R. R. Hoffman (Ed.), The psychology of expertise: Cognitive research and empirical AI (pp. 29-60). Mahwah, NJ: Lawrence Erlbaum Associates.
18. Cooke, N. J., & Breedin, S. D. (1994). Constructing naive theories of motion on the-fly. Memory and Cognition, 22, 474-493.
19. Delamont, S., & Atkinson, P. (2001). Doctoring uncertainty: Mastering craft knowledge. Social Studies of Science, 31(1), 87-107.
20. Dufresne, R. J., Gerace, W. J., Hardiman, P. T., & Mestre, J. P. (1992). Constraining novices to perform expertlike problem analyses: Effects on schema acquisition. Journal of the Learning Sciences, 2(3), 307-331.
21. Dunbar, K. (2000). How scientists think in the real world: Implications for science education. Journal of Applied Developmental Psychology, 21(1), 49-58.
22. Ericsson, K. A., & Charness, N. (1994). Expert performance: Its structure and acquisition. American Psychologist, 49(8), 725-747.
23. Ericsson, K. A., & Simon, H. A. (1993). Protocol analysis: Verbal reports as data (Revised ed.). Cambridge, MA: Bradford.
24. Feldon, D. F. (2007). Implications of research on expertise for curriculum and pedagogy. Educational Psychology Review, 19(2), 91-110.
25. Garcia-Mila, M., & Andersen, C. (2007). Developmental change in notetaking during scientific inquiry. International Journal of Science Education, 29(8), 1035-1058.
26. Hassebrock, F., Johnson, P. E., Bullemer, P., Fox, P. W., & Moller, J. H. (1993). When less is more: Representation and selective memory in expert problem solving. American Journal of Psychology, 106(2), 155-189.
27. Hoffman, R. R., Crandall, B., & Shadbolt, N. (1998). A case study in cognitive task analysis methodology: The critical decision method for the elicitation of expert knowledge. Human Factors, 40, 254-276.
28. Jackson, P. W. (1985). Private lessons in public schools: Remarks on the limits of adaptive instruction. In M. C. Wang & H. J. Walberg (Eds.), Adapting instruction to individual differences (pp. 66-81). Berkeley, CA: McCutchan.
29. Kirschner, P., Sweller, J., & Clark, R. E. (2006). Why minimally guided learning does not work: An analysis of the failure of discovery learning, problem-based learning, experiential learning and inquiry-based learning. Educational Psychologist, 41(2), 75-86.
30. Klahr, D., & Dunbar, K. (1988). Dual space search during scientific reasoning. Cognitive Science, 12(1), 1-55.
31. Klein, G. A., & Calderwood, R. (1996). Investigations of naturalistic decision making and the recognition-primed decision model (Research Note 96-43). Yellow Springs, OH: Klein Associates, Inc. Prepared under contract MDA903-85-C-0327 for U. S. Army Research Institute for the Behavioral and Social Sciences, Alexandria, VA.
32. Lamberti, D. M., & Newsome, S. L. (1989). Presenting abstract versus concrete information in expert systems: What is the impact on user performance? International Journal of Man-Machine Studies, 31(1), 27-45.
33. Larkin, J. H., McDermott, J., Simon, D. P., & Simon, H. A. (1980). Models of competence in solving physics problems. Cognitive Science, 4(4), 317-345.
34. Lohman, D. F. (1986). Predicting mathemathantic effects in the teaching of higher order thinking skills. Educational Psychologist, 21(3), 191-208.
35. Lovett, M. C., & Anderson, J. R. (1996). History of success and current context in problem solving: Combined influences on operator selection. Cognitive Psychology, 31, 168-217.
36. Merrill, M. D. (2002). A pebble-in-the-pond model for instructional design. Performance Improvement, 41(7), 39-44.
37. Moors, A., & De Houwer, J. (2006). Automaticity: A theoretical and conceptual analysis. Psychological Bulletin, 132(2), 297-326.
38. Morgan, D. L., & Morgan, R. K. (2001). Single-subject research design: Bringing science to managed care. American Psychologist, 56(2), 119-127.
39. Newell, A., & Simon, H. A. (1972). Human problem solving. Englewood Cliffs, NJ: Prentice-Hall.
40. Nisbett, R. E., & Wilson, T. D. (1977). Telling more than we can know: Verbal reports on mental processes. Psychological Review, 84, 231-259.
41. Onwuegbuzie, A. J., Slate, J. R., Paterson, F. R. A., Watson, M. H., & Schwartz, R. A. (2000). Factors associated with achievement in educational research courses. Research in the Schools, 7(1), 53-65.
42. Pearson, M., & Brew, A. (2002). Research training and supervision development. Studies in Higher Education, 27(2), 135-150.
43. Radziszewska, B., & Rogoff, B. (1988). Influence of adult and peer collaborators on children's planning skills. Developmental Psychology, 24(6), 840-848.
44. Rikers, R., Schmidt, H. G., & Boshuizen, H. (2000). Knowledge encapsulation and the intermediate effect. Contemporary Educational Psychology, 25(2), 150-166.
45. Ritter, F. E., Nerb, J., & Lehtinen, E. (2007). Getting things in order: Collecting and analyzing data on learning. In F. E. Ritter, J. Nerb, E. Lehtinen, & T. M. O'Shea (Eds.), In order to learn: How the sequences of topics affect learning (pp. 81-92). New York: Oxford University Press.
46. Russo, J. E., Johnson, E. J., & Stephens, D. L. (1989). The validity of verbal protocols. Memory & Cognition, 17(6), 759-769.
47. Schaafstal, A., Schraagen, J. M., & van Berlo, M. (2000). Cognitive task analysis and innovation of training: The case of the structured troubleshooting. Human Factors, 42(1), 75-86.
48. Schmidt, H. G., & Boshuizen, H. P. A. (1993). On the origin of intermediate effects on clinical case recall. Memory and Cognition, 21(3), 338-351.
49. Schmidt, H. G., Boshuizen, H. P. A., & Hobus, P. P. M. (1988). Transitory stages in the development of expertise: The "intermediate effect" in the recall of clinical case representation studies. In Proceedings of the Cognitive Science Society Meeting (pp. 139-145). Hillsdale, NJ: Lawrence Erlbaum Associates.
50. Schneider, W., & Shiffrin, R. M. (1977). Controlled and automatic human information processing: I. Detection, search, and attention. Psychological Review, 84(1), 1-66.
51. Schooler, J. W., Ohlsson, S., & Brooks, K. (1993). Thoughts beyond words: When language overshadows insight. Journal of Experimental Psychology: General, 122(2), 166-183.
52. Schraagen, J. M. C. (1990). How experts solve a novel problem within their domain of expertise (IZF 1990 B-14). Soesterberg, The Netherlands: Netherlands Organization for Applied Scientific Research. Prepared under HDO assignment B89-35 for TNO Institute for Perception, TNO Division of National Defense Research, Soesterberg, The Netherlands.
53. Schraagen, J. M. C. (1993). How experts solve a novel problem in experimental design. Cognitive Science, 17(2), 285-309.
54. Schraagen, J. M. C., Chipman, S. F., & Shute, V. J. (2000). State-of-the-art review of cognitive task analysis techniques. In J. M. C. Schraagen, S. F. Chipman, & V. L. Shalin (Eds.), Cognitive task analysis (pp. 467-487). Mahwah, NJ: Lawrence Erlbaum Associates.
55. Schunn, C. D., & Anderson, J. R. (1999). The generality/specificity of expertise in scientific reasoning. Cognitive Science, 23(3), 337-370.
56. Schunn, C. D., & Anderson, J. R. (2001). Acquiring expertise in science: Explorations of what, when, and how. In K. Crowley, C. D. Schunn, & T. Okada (Eds.), Designing for science: Implications from everyday, classroom, and professional settings (pp. 83-113). Mahwah, NJ: Lawrence Erlbaum Associates.
57. Schwartz, D., & Bransford, J. D. (1998). A time for telling. Cognition and Instruction, 16(4), 475-522.
58. Siegler, R. S. (2006). Microgenetic analyses of learning. In D. Kuhn & R. S. Siegler (Eds.), Handbook of child psychology (6th ed., pp. 464-510). Hoboken, NJ: John Wiley & Sons, Inc.
59. Siegler, R. S., & Crowley, K. (1991). The microgenetic method: A direct means for studying cognitive development. American Psychologist, 46(6), 606-620.
60. Singley, M. K., & Anderson, J. R. (1989). Transfer of cognitive skill. Cambridge, MA: Harvard University Press.
61. Sloutsky, V., & Yarlas, A. (2000). Problem representation in experts and novices: Part 2. Underlying processing mechanisms. In Proceedings of the XXII Annual Conference of the Cognitive Science Society (pp. 475-480). Mahwah, NJ: Erlbaum.
62. Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12, 257-285.
63. Sweller, J. (1999). Instruction design in technical areas. Camberwell, Australia: ACER.
64. Taylor, K. L., & Dionne, J. P. (2000). Accessing problem solving strategy knowledge: The complementary use of concurrent verbal protocols and retrospective debriefing. Journal of Educational Psychology, 92(3), 413-425.
65. Thorley, N., & Stofflet, R. (1996). Representation of the conceptual change model in science teacher education. Science Education, 80, 317-339.
66. van Gog, T., Paas, F., van Merriënboer, J., & Witte, P. (2006). Uncovering the problem-solving process: Cued retrospective reporting versus concurrent and retrospective reporting. Journal of Experimental Psychology: Applied, 11(4), 237-244.
67. Veermans, K., van Joolingen, W., & de Jon, T. (2006). Use of heuristics to facilitate scientific discovery learning in a simulation learning environment in a physics domain. International Journal of Science Education, 28(4), 341-361.
68. Velmahos, G., Toutouzas, K., Sillin, L., Chan, L., Clark, R. E., Theodorou, D., et al. (2004). Cognitive task analysis for teaching technical skills in an animate surgical skills laboratory. The American Journal of Surgery, 187, 114-119.
69. Wigboldus, D. H. J., Dijksterhuis, A., & van Knippenberg, A. (2003). When stereotypes get in the way: Stereotypes obstruct stereotype-inconsistent trait inferences. Journal of Personality and Social Psychology, 84, 470-484.
70. Wilson, T., & Nisbett, R. (1978). The accuracy of verbal reports about the effects of stimuli on evaluations and behavior. Social Psychology, 41(2), 118-131.
71. Yang, S. C. (2003). Reconceptualizing think-aloud methodology: Refining the encoding and categorizing techniques via contextualized perspectives. Computers in Human Behavior, 19, 95-115.
72. Yarlas, A., & Sloutsky, V. (2000). Problem representation in experts and novices: Part 1. Differences in the content of representation. In Proceedings of the XXII Annual Conference of the Cognitive Science Society (pp. 1006-1011). Mahwah, NJ: Erlbaum.
73. Yates, K. A., & Feldon, D. F. (2008). Towards a taxonomy of cognitive task analysis methods for instructional design: Interactions with cognition. Presented at the Annual Meeting of the American Education Research Association, New York, NY, March, 2008.