Differences between novice and expert systems analysts: What do we know and what do we do?

Journal of Management Information Systems

Volumn 15, Issue 1, 1998, Pages 9-50

  Schenk, K D ^a,b   Vitalari, Nicholas P ^b,c   Davis, K Shannon ^a,d,e  

^a NORTH CAROLINA STATE UNIVERSITY   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

^c UNIVERSITY OF MINNESOTA   (United States)

^d UNIVERSITY OF MARYLAND   (United States)

^e VIRGINIA POLYTECHNIC INSTITUTE AND STATE UNIVERSITY   (United States)

Author keywords

Creativity in problem solving; Information requirements; Information systems development; Novice and expert problem solving; Systems analysis; Systems analyst education

Indexed keywords

EID: 0032223732     PISSN: 07421222     EISSN: None     Source Type: Journal    
DOI: 10.1080/07421222.1998.11518195     Document Type: Article

References (107)

1. Adelson, B., and Soloway, E. A model of software design. In M.T.H. Chi, R. Glaser, and M.J. Farr, M.J. (eds.), The Nature of Expertise. Hillsdale, NJ: Erlbaum, 1988, pp. 185-208.
2. Anderson, J.R. Cognitive Psychology and Its Implications. New York: W.H. Freeman, 1985.
3. Arvey, R.D., and Hoyle, J.C. A Cuttman approach to the development of behaviorally-based rating scales for systems analysts and programmer analysts. Journal of Applied Psychology, 59, 1 (February 1974), 61-68.
4. Atwood, M.E.; Turner, A.A.; Ramsey, H.R.; and Hopper, J.N. An exploratory study of the cognitive structures underlying the comprehension of software design problems. Technical Report no. 392, Alexandria, VA, AKI, 1979.
5. Baddeley, A.D. The Psychology of Memory. New York: Basic Books, 1976.
6. Barki, H., and Hartwick, J. Rethinking the concept of user involvement. MIS Quarterly, 13, 1, (1989), 53-63.
7. Bazerman, M.H. Judgment in Managerial Decision Making, 3d ed. New York: John Wiley and Sons, 1994.
8. Bouwman, M.J. Expert vs. novice decision making in accounting: a summary. Accounting, Organizations and Society, 9 (1984), 325-327.
9. Bradley, J. Measuring general cognitive ability in computer programmers and analysts. Proceedings of the First America's Conference on Information Systems, Pittsburgh, PA, August 1995, pp. 202-204.
10. Brooks, R. Towards a theory of the cognitive processes in computer programming. International Journal of Man-Machine Studies, 9 (1977), 737-751.
11. Brooks, R. Studying programmer behavior experimentally: the problems of a proper methodology. Communications of the ACM, 23 (1980), 207-213.
12. Carroll, J.S. Causal attributions in expert parole decisions. Journal of Personality and Social Psychology, 36, 12 (1978), 1501-1511.
13. Chase, W.G., and Ericsson, K.A. Skill and working memory. In G.H. Bower (ed.), The Psychology of Learning and Motivation, vol. 16. New York: Academic Press, 1982, pp. 1-58.
14. Chase, W.G., and Simon, H.A. The mind's eye in chess. In W.G. Chase (ed.), Visual Information Processing. New York: Academic Press, 1973, pp. 215-281.
15. Chi, M.T.H.; Feltovich, P.; and Glaser, R. Categorization and representation of physics problems by experts and novices. Cognitive Science, 5 (1981), 121-152.
16. Chi, M.T.H., and Glaser, R. Problem solving ability. In R.J. Sternberg (ed.), Human Abilities: An Information-Processing Approach. New York: W.H. Freeman, 1985, pp. 227-250.
17. Chi, M.T.H.; Glaser, R.; and Farr, M.J. The Nature of Expertise. Hillsdale, NJ: Lawrence Erlbaum, 1988.
18. Chi, M.T.H.; Glaser, R.; and Rees, E. Expertise in problem solving. In R.J. Sternberg (ed.), Advances in the Psychology of Human Intelligence, vol. 1. Hillsdale, NJ: Lawrence Erlbaum, 1982, pp. 7-75.
19. Cochran, W.G. Some methods for strengthening the common chi-square tests. Biometrics, 10 (1954), 417-451.
20. Cooper, R., and Swanson, B. Management information requirements assessment: the state of the art. Data Base, 11, 2 (1979), 5-16.
21. Cotterman, W.W. Proceedings of the Systems Analysis and Design Workshop: A Foundation for the 1980's. Workshop sponsored by the Institute for Certification of Computer Professionals, Atlanta, 1981.
22. Couger, J.D. Creative Problem Solving and Opportunity Finding. Danvers, MA: Boyd and Fraser, 1995.
23. Couger, J.D. Creativity/Innovation in IS Organizations. Danvers, MA: Boyd and Fraser, 1995.
24. Dagwell, R., and Weber, R. Systems designers' user models: a comparative study and methodological critique. Communications of the ACM, 26, 11 (1983), 987-998.
25. Danos, P.; Holt, D.; and Imhoff, E. Bond rater's use of management forecasts: experiment in expert judgment. Accounting Review, 49, 4 (1984), 547-573.
26. Dee-Lucas, D., and Larkin, J.H. Novice strategies for processing scientific texts. Paper presented at the meeting of the American Educational Research Association, 1984.
27. Ebbinghaus, H. Memory: A Contribution to Experimental Psychology. New York: Columbia University Press, 1913.
28. Elam J.J., and Mead, M. Designing for creativity: considerations for DSS development. Information and Management, 13 (1987), 215-222.
29. Elam J.J., and Mead, M. Can software influence creativity? Information Systems Research, 1, 1 (1990), 1-22.
30. Elstein, A.S.; Shulman, L.A.; and Sprafka, S.A. Medical Problem Solving: An Analysis of Clinical Reasoning. Cambridge, MA: Harvard University Press, 1978.
31. Einhorn, H. Synthesis: accounting and behavioral science. Studies on Human Information Processing, Supplement to the Journal of Accounting Research (1976), 196-206.
32. Ericsson K.A., and Simon, H.A. Protocol Analysis: Verbal Reports as Data. Cambridge, MA: MIT Press, 1984.
33. Evans, J.R. Creative Thinking in the Decision and Management Sciences. Cincinnati, OH: South-Western Publishing, 1991.
34. Finn, K. Making creativity happen: an introduction to the solution machine. The Gemini Group, Bedford, NY, 1993.
35. Flanagan, J.C. The critical incident technique. Psychological Bulletin, 51, 4 (1954), 327-358.
36. Ford, J.K.; Schmitt, N.; Schectman, S.L.; Hults, B.M.; and Doherty, M.L. Process tracing methods: contributions, problems, and neglected research questions. Organizational Behavior and Human Decision Processes, 43 (1989), 75-117.
37. Glaser, R. Education and thinking. American Psychologist, 39 (1984), 93-104.
38. Groen, G.J., and Patel, V.L. The relationship between comprehension and reasoning in medical expertise. In M.T.H. Chi, R. Glaser, and M.J. Farr (eds.), The Nature of Expertise. Hillsdale, NJ: Erlbaum, 1988, pp. 287-310.
39. Guinan, P.J. Specialist-generalist communication competence: a field experiment investigating the communication behavior of information systems developers. Ph.D. dissertation, Indiana University, 1986.
40. Hedberg, B., and Mumford E. The design of computer systems: man's vision of man as an integral part of the system design process. In E. Mumford and H. Sackman (eds.), Human Choice and Computers. Amsterdam: North-Holland, 1975, pp. 31-59.
41. Hogarth, R. Judgment and Choice: The Psychology of Decision. New York: John Wiley and Sons, 1980.
42. Humphreys, P. Use of problem structuring techniques for option generation: a computer choice case study. In P. Humphreys, Seveson, and Vari (eds.), Advances in Psychology. New York: North-Holland, 1983, pp. 251-279.
43. Ives, B., and Olson, M.H. User involvement and MIS success: a review of research. Management Science, 30, 5 (1984), 586-603.
44. Jeffries, R.; Turner, A.A.; Polson, P.; and Atwood, M. The processes involved in designing software. In J.R. Anderson (ed.), Cognitive Skills and Their Acquisition. Hillsdale, NJ: Lawrence Erlbaum, 1981, pp. 255-283.
45. Johnson, P.E. Representation of expert knowledge and skill in the design of instruction. Paper presented at the meeting of the American Educational Research Association, March 1977.
46. Johnson, P.; Duran, A.; Hassebrock, F.; Moller, J.; and Prietula, M. Expertise and error in diagnostic reasoning. Cognitive Science, 5 (1981), 235-283.
47. Kaiser, K., and King, W. The manager-analyst interface in systems development. MIS Quarterly, 6, 1 (1982), 49-59.
48. Keen, P. Information systems and organizational change. Communications of the ACM, 24, 1 (1981), 24-33.
49. Kolodner, J.L. Towards an understanding of the role of experience in the evolution from novice to expert. Journal of Man-Machine Studies, 19 (1983), 497-518.
50. Langle, G. An investigation of the effect of specific knowledge in functional areas of business on information systems analysis and design. Ph.D. dissertation, School of Management, University of Minnesota, 1987.
51. Larkin, J.H.D.; McDermott, D.; Simon, D.P.; and Simon, H.A. Models of competence in solving physics problems. Cognitive Science, 4 (1980), 317-345.
52. Lawrence, J.A. Expertise on the bench: modeling magistrates' judicial decision-making. In M.T.H. Chi, R. Glaser, and M.J. Farr (eds.), The Nature of Expertise. Hillsdale, NJ: Lawrence Erlbaum, 1988, pp. 229-259.
53. Levine, H.G., and Rossmoore, D. Diagnosing the human threats to information technology implementation: a missing factor in systems analysis. Journal of Management Information Systems, 10, 2 (Fall 1993), 55-73.
54. Lobert, B.M.; Mackay, J.M.; and Barr, S.H. Problem-solving and creativity support systems: a framework for research. Unpublished manuscript.
55. Lopes, L.L. Model-based decision and inference in stud poker. Journal of Experimental Psychology, 105 (1976), 217-239.
56. Lord, R.G., and Maher, K.J. Alternative information-processing models and their implications for theory, research, and practice. Academy of Management Review, 15, 1 (1990), 9-28.
57. Lyytinen, K. Different perspectives on information systems: problems and solutions. Computing Surveys, 19, 1 (1987), 5-46.
58. MacCrimmon, K.R., and Wagner, C. Stimulating ideas through creativity software. Management Science, 40, 11 (1994), 1514-1532.
59. Mackay, J.M.; Barr, S.H.; and Kletke, M.G. An empirical investigation of the effects of decision aids on problem-solving processes. Decision Sciences, 23, 3 (1992), 648-672.
60. Mackay, J.M., and Elam, J.J. A comparative study of how experts and novices use a decision aid to solve problems in complex knowledge domains. Information Systems Research, 3, 2 (1992), 150-172.
61. Markus, L. Power, politics and MIS implementation. Communications of the ACM, 26, 6 (1983), 430-444.
62. Massetti, B. An empirical examination of the value of creativity support systems on idea generation. MIS Quarterly, 20, 1 (1996), 83-97.
63. McKeithen, K.B.; Reitman, J.S.; Rueter, H.H.; and Hirtle, S.C. Knowledge organization and skill differences in computer programmers. Cognitive Psychology, 13 (1981), 307-325.
64. Mehle, T. Hypothesis generation in an automobile malfunction inference task. Acta Psychologica, 52 (1982), 87-106.
65. Miller, M.L., and Goldstein, I.P. Planning and debugging in elementary programming. In P.H. Winston and R.H. Brown (eds.), Artificial Intelligence: An MIT Perspective, vol. 1. Cambridge, MA: MIT Press, 1979, pp. 317-338.
66. Mindlink Problem Solver: Corporate Edition. North Pomfret, VT: Mindlink Software, 1995.
67. Misic, M. The skills needed by today's systems analysts. Journal of Systems Management (May-June 1996), 34-40.
68. Neale, M.A., and Northcraft, G.B. Experience, expertise, and decision bias in negotiation: the role of strategic conceptualization. In B. Sheppard, M. Bazerman, and R. Lewicki (eds.), Research on Negotiations in Organizations, vol. 2. Greenwich, CT: JAI Press, 1989.
69. Newell, A. Heuristic programming: ill-structured problems. In J.S. Aronofsky (ed.), Progress in Operations Research, vol. 3. New York: John Wiley and Sons, 1969, pp. 361-414.
70. Newell, A., and Simon, H.A. Human Problem Solving. Englewood Cliffs, NJ: Prentice-Hall, 1972.
71. Nilsson, N.J. Principles of Artificial Intelligence. Palo Alto, CA: Tioga Publishers, 1980.
72. Payne, J.W. Task complexity and contingent processing in decision making: an information search and protocol analysis. Organizational Behavior and Human Performance, 16 (1976), 366-387.
73. Payne, J.W.; Braunstein, M.L.; and Carroll, J.S. Exploring predecisional behavior: an alternative approach to decision research. Organizational Behavior and Human Performance, 22 (1978), 17-44.
74. Pennington, N. Cognitive components of expertise in computer programming: a review of the literature. Technical Report no. 46, University of Michigan, 1982.
75. Phelps, R.H., and Shanteau, J. Livestock judges: how much information can an expert use? Organizational Behavior and Human Performance, 21 (1978), 209-219.
76. Polson, P., and Jeffries, R. Problem solving as search and understanding. In R.J. Sternberg (ed.), Advances in the Psychology of Human Intelligence, vol. 1. Hillsdale, NJ: Lawrence Erlbaum, 1982, pp. 367-411.
77. Pople, H.E. Heuristic methods for imposing structure on ill-structured problems: the structuring of medical diagnostics. In P. Szolovits (ed.), Artificial Intelligence in Medicine. Boulder, CO: Westview Press, 1982, pp. 119-190.
78. Prietula, M., and Dickson, G. Flexible interfaces and the support of physical database design reasoning. In L. Kerschberg (ed.), Expert Database Systems. New York: Benjamin Cummings Publishing, 1987.
79. Prietula, M., and Marchak, F. Expert variance: differences in solving a dynamic engineering problem. In Proceedings of the Seventh Annual Conference of the Cognitive Science Society, Irvine, CA, August 1985, pp. 15-17.
80. Ricchiute, D.N. Presentation mode, task importance, and cue order in experimental research on expert judges. Journal of Applied Psychology, 70, 2 (1985), 367-373.
81. Rockart, J.F. Critical success factors. Harvard Business Review, 57, 2 (1979), 81-91.
82. Salaway, G. An organizational learning approach to information systems development. MIS Quarterly (June 1987), 245-264.
83. Schank, R.C. Dynamic Memory: A Theory of Learning in People and Computers. Cambridge: Cambridge University Press, 1982.
84. Schoenfeld, A.H., and Hermann, D.J. Problem perception and knowledge structure in expert and novice mathematical problem solvers. Journal of Experimental Psychology: Learning, Memory, and Cognition, 8, 5 (1982), 484-494.
85. Shanteau, J., and Phelps, R.H. Judgment and swine: approaches and issues in applied judgment analysis. In M.F. Kaplan and S. Schwartz (eds.), Human Judgment and Decision Processes in Applied Settings. New York: Academic Press, 1977, pp. 255-272.
86. Sherman, S.J., and Corty, E. Cognitive heuristics. In R.S. Wyer and T.K. Srull (eds.), Handbook of Social Cognition, vol. 1. Hillsdale, NJ: Lawrence Erlbaum, 1984, pp. 189-286.
87. Siegel, S. Nonparametric Statistics for the Behavioral Sciences. New York: McGraw-Hill, 1956.
88. Simon, D.P., and Simon, H.A. Individual differences in solving physics problems. In R.S. Siegler (ed.), Children's Thinking: What Develops? Hillsdale, NJ: Lawrence Erlbaum, 1978, pp. 325-348.
89. Simon, H.A., and Ando, A. Aggregation of variables in dynamic systems. Econometrica, 29, 1961, 111-138.
90. Swanson, E.B. Management information systems: appreciation and involvement. Management Science, 21, 2 (1974), 178-188.
91. Tan, M. Establishing mutual understanding in systems design: an empirical study. Journal of Management Information Systems, 10, 1 (Spring 1994), 159-182.
92. Taylor, R.N. Behavioral Decision Making. Glenview, IL: Scott, Foresman, 1984.
93. Todd, P., and Benbasat, I. An experimental investigation of the impact of computer based decision aids on decision making strategies. Information Systems Research, 2, 2 (June 1991), 87-115.
94. Torgerson. T. Product review: IdeaFisher. A1 Today (March-April 1990), 1.
95. Tversky, A., and Kahneman, D. Availability: a heuristic for judging frequency and probability. Cognitive Psychology, 5 (1973), 207-232.
96. Tversky, A., and Kahneman, D. Judgment under uncertainty: heuristics and biases. Science, 185 (1974), 1124-1131.
97. Vessey, I. Expertise in debugging computer programs: a process analysis. Journal of Man-Machine Studies, 23 (1985), 459-494.
98. Vessey, I., and Conger, S. Learning to specify information requirements: the relationship between application and methodology. Journal of Management Information Systems, 10, 2 (Fall 1993), 177-201.
99. Vitalari, N. An investigation of the problem solving behavior of systems analysts. Ph.D. dissertation, School of Management, University of Minnesota, 1981.
100. Vitalari, N. Knowledge as a basis for expertise in systems analysis: an empirical study. MIS Quarterly, 9, 3 (1985), 221-241.
101. Vitalari, N., and Dickson, G. Problem solving for effective systems analysis: an experimental exploration. Communications of the ACM, 26, 11 (1983), 948-956.
102. Voelcker, J. Automating software: proceed with caution. IEEE Spectrum, 25, 7 (1988), 25-27.
103. Wanninger, L.A., and Dickson, G.W. GRIP- Group requirement identification process: the integration of GDSS technology into the systems development life cycle. Proceedings of the Twenty-Fifth Hawaii International Conference on Organizational Computing, IEEE Computer Society, January 1992.
104. Wegner, P. Research Directions in Software Technology. Cambridge, MA: MIT Press, 1979.
105. Whitten, J.L.; Bentley, L.D.; and Barlow, V.M.. Systems Analysis and Design Methods, 2d ed. Homewood, IL: Irwin, 1989.
106. Whitten, J.L.; Bentley, L.D.; and Barlow, V.M.. Systems Analysis and Design Methods, 3d ed. Homewood, IL: Irwin, 1994.
107. Zmud, R.W.; Anthony, W.P.; and Stair, R.M. The use of mental imagery to facilitate information identification in requirements analysis. Journal of Management Information Systems, 9, 1 (Spring 1993), 175-191.