Near-term liability of exploitation: Exploration and exploitation in multistage problems

Organization Science

Volumn 20, Issue 3, 2009, Pages 538-551

  Fang, Christina ^a   Levinthal, Daniel ^b  

^a NEW YORK UNIVERSITY   (United States)

^b UNIVERSITY OF PENNSYLVANIA   (United States)

Author keywords

Exploration and exploitation; Maximization; Multistage problems; Reinforcement learning; Softmax choice rule

Indexed keywords

EID: 70350391648     PISSN: 10477039     EISSN: 15265455     Source Type: Journal    
DOI: 10.1287/orsc.1080.0376     Document Type: Article

References (90)

1. Andrews, K. 1965. The Concept of Corporate Strategy. Irwin Homewood, IL.
2. Argote, L. 1999. Organizational Learning: Creating Retaining and Transferring Knowledge. Springer-Verlag, New York.
3. Bellman, R. 1957. Dynamic Programming. Princeton University Press, Princeton, NJ.
4. Benner, M., M. Tushman. 2002. Process management and technological innovation: A longitudinal study of the photography and paint industries. Admin. Sci. Quart. 47 676-706.
5. Benner, M., M. Tushman. 2003. Exploitation, exploration and process management: The productivity dilemma revisited. Acad. Management Rev. 28(2) 238-257.
6. Berthier, N., M. Rosenstein, A. Barto. 2005. Approximate optimal control as a model for motor learning. Psych. Rev. 112(2) 329-346.
7. Block, Z., I. MacMillan. 1985. Milestones for successful venture planning. Harvard Bus. Rev. 63(5) 184-190.
8. Breeden, J., F. Dionkelacker, A. Huber. 1990. Noise in the modeling and control of dynamical systems. Physical Rev. A 42 5827-5836.
9. Brehmer, B. 1995. Feedback delays in complex dynamic decision tasks. P. Frensch, J. Funke, eds. Complex Problem Solving, The European Perspective. Erlbaum Associates, Hillsdale, NJ.
10. Bruderer, E., J. Singh. 1996. Organizational evolution, learning and selection: A genetic algorithm-based model. Acad. Management J. 39 1322-1329.
11. Camerer, C., T. H. Ho. 1999. Experience-weighted attraction learning in normal form games. Econometrica 67 837-874.
12. Campbell, D. T. 1960. Blind variation and selective retention in creative thought processes. Psych. Rev. 67 380-400.
13. Cyert, R., J. March. 1963. A Behavioral Theory of the Firm. Prentice Hall, Englewood Cliffs, NJ.
14. Daw, N., P. Dayan. 2004. Matchmaking. Science 304 1753-1754.
15. Daw, N., J. O'Doherty, P. Dayan, B. Seymour, R. Dolan. 2006. Cortical substrates for exploratory decisions in humans. Nature 441(15) 876-879.
16. DeGroot, A. D. 1965. Thought and Choice in Chess. Mouton Publishers, The Hague.
17. DeGroot, M. H. 1970. Optimal Statistical Decisions. McGraw-Hill, New York.
18. Denrell, J., C. Fang, D. Levinthal. 2004. Learning from t-mazes to labyrinths: Learning from model-based feedback. Management Sci. 50(10) 1366-1378.
19. Einhorn, H., R. Hogarth. 1981. Behavioral decision theory: Processes of judgment and choice. J. Accounting Res. 19(1) 1-32.
20. Friedman, M. 1953. Essays in Positive Economics. University of Chicago Press, Chicago.
21. Fukunaga, A., G. Rabideau, S. Chien. 2004. Robust local search for spacecraft operations using adaptive noise. Proc. 4th Internat. Workshop Planning Scheduling Space, Darmstadt, Germany.
22. Gans, N., G. Knox, R. Croson. 2007. Simple models of discrete choice and their performance in bandit experiments. Manufacturing Service Oper. Management 9(4) 383-408.
23. Garvin, D. 1991. How the baldridge award really works. Harvard Bus. Rev. 69(6) 80-93.
24. Gavetti, G., D. Levinthal. 2000. Looking forward and looking backward: Cognitive and experiential search. Admin. Sci. Quart. 45(1) 113-137.
25. Gavetti, G., D. Levinthal, J. Rivkin. 2005. Strategy making in novel and complex worlds: The power of analogy. Strategic Management J. 26 691-712.
26. Gibson, F., M. Fichman, D. Plaut. 1997. Learning in dynamic decision tasks: Computational model and empirical evidence. Organ. Behav. Human Decision Processes 71 11-35.
27. Guastello, S. 1995. Chaos, Catastrophe and Human Affairs. Lawrence Erlbaum Associates, Mahwah, NJ.
28. Herriott, S., D. Levinthal, J. G. March. 1985. Learning from experience in organizations. Amer. Econom. Rev., Papers Proc. 97th Annual Meeting 75(2) 298-302.
29. Holland, J. 1975. Adaptation in Natural and Artificial Systems. University of Michigan Press, Ann Arbor, MI.
30. Hsu, F. H. 2002. Behind Deep Blue: Building the Computer that Defeated the World Chess Champion. Princeton University Press, Princeton, NJ.
31. Jackson, E. 1991a. On the control of complex dynamic systems. Physica D 50 341-366.
32. Jackson, E. 1991b. Controls of dynamic flows with attractors. Physical Rev. A 44 4839-4853.
33. Kaelbling, L. 1993. Learning in Embedded Systems. MIT Press, Cambridge, MA.
34. Kahneman, D., A. Tversky. 1973. On the psychology of prediction. Psych. Rev. 80 237-251.
35. Kaplan, C. A., H. A. Simon. 1990. In search of insight. Cognitive Psych. 22 374-419.
36. Kauffman, S. 1993. The Origins of Order. Oxford University Press, New York.
37. Langley, P., R. Jones. 1988. A computational model of scientific insights. R. Sternberg, ed. The Nature of Creativity: Contemporary Psychological Perspectives. Cambridge University Press, New York, 177-201.
38. Lant, T. K. 1994. Computer simulations of organizations as experiential learning systems: Implications for organization theory. K. Carley, M. Prietula, eds. Computational Organization Theory. Lawrence Erlbaum Associates, Hillsdale, NJ.
39. Lant, T., S. Mezias. 1990. Managing discontinuous change: A simulation study of organizational learning and entrepreneurship. Strategic Management J. 11(4) 147-179.
40. Lant, T., S. Mezias. 1992. An organizational learning model of convergence and reorientation. Organ. Sci. 3(1) 47-71.
41. Lave, C., J. March. 1975. An Introduction to Models in Social Sciences. Harper & Row, New York.
42. Lee, D. 2006. Best to go with what you know? Nature 441(15) 822-823.
43. Leifer, E. 1991. Actors as Observers: A Theory of Skill in Social Relationships. Garland, New York.
44. Levinthal, D. 1997. Adaptation on rugged landscapes. Management Sci. 43(7) 934-950.
45. Levinthal, D., J. March. 1981. A model of adaptive organizational search. J. Econom. Behav. Organ. 2 307-333.
46. Levinthal, D., J. March. 1993. The myopia of learning. Strategic Management J. 14 95-112.
47. Levitt, B., J. March. 1988. Organization learning. Annual Rev. Sociol. 14 319-340.
48. Luce, R. 1959. Individual Choice Behavior. John Wiley and Sons, New York.
49. March, J. 1978. Bounded rationality, ambiguity and the engineering of choice. Bell J. Econom. 9(2) 587-608.
50. March, J. 1991. Exploration and exploitation in organization learning. Organ. Sci. 2(1) 71-87.
51. McKelvey, B. 1999. Avoiding complexity catastrophe in co-evolutionary pockets: Strategies for rugged landscape. Organ. Sci. 10(3) 294-321.
52. Minsky, M. 1961. Steps towards artificial intelligence. Proc. Inst. Radio Engineers 49(1) 8-30.
53. Montague, P. R., P. Dayan, T. J. Sejnowski. 1996. A framework for mesence-phalic dopamine systems based on predictive hebbian learning. J. Neuroscience 16 1936-1947.
54. McClure, S., N. Daw, P. Montague. 2003. A computational substrate for incentive salience. Trends in Neurosciences 26 423-428.
55. Newell, A., H. Simon. 1972. Human Problem Solving. Prentice-Hall, Englewood Cliffs, NJ.
56. O'Doherty, J., P. Dayan, J. Schultz, R. Deichmann, K. Friston, R. Dolan. 2004. Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304 452-454.
57. Ohle, F., F. Dinkelacker, A. Huber, M. Welge. 1990. Adapative control of chaotic systems. Technical Report CCER-90-13, Department of Physics, Beckman Institute, University of Illinois, Urbana-Champaign.
58. Padgett, J., C. Ansell. 1993. Robust action and the rise of the medici, 1400-1434. Amer. J. Sociol. 98(6) 1259-1319.
59. Perkins, D. 2000. The Eureka Effect: The Art and Logic of Break-through Thinking. W. W. Norton and Company, New York.
60. Pisano, G. 2006. Can science be a business? Harvard Bus. Rev. (October) 1-12.
61. Poincaré, H. 1929. The Foundations of Science. Science House, New York.
62. Radner, R., M. Rothschild. 1975. On the allocation of effort. J. Econom. Theory 10 358-376.
63. Rivkin, J. W. 2000. Imitation of complex strategies. Management Sci. 46(6) 824-844.
64. Roth, A., I. Erev. 1995. Learning in extensive form games: Experimental data and simple dynamic models in the intermediate term. Games and Econom. Behav. 8 164-212.
65. Samuel, A. 1959. Some studies in machine learning using the game of checkers. IBM J. Res. Development 32 211-229.
66. Samuel, A. 1967. Some studies in machine learning using the game of checkers II-Recent progress. IBM J. Res. Development 11 601-617.
67. Savage, L. 1954. The Foundations of Statistics. John Wiley and Sons, New York.
68. Schultz, W., P. Dayan, P. R. Montague. 1997. A neural substrate of prediction and reward. Science 275 1593-1599.
69. Seabrook, R., Z. Dienes. 2003. Incubation in problem solving as a context effect. Proc. 25th Meeting Cognitive Sci. Soc., Lawrence Erlbaum Associates, Mahwah, NJ.
70. Segal, E. 2004. Incubation in insight problem solving. Creativity Res. J. 16(1) 141-148.
71. Selman, B., H. Kautz. 1993. An empirical study of greedy local search for satisfiability testing. Proc. Amer. Assoc. Artificial Intelligence, 46-51.
72. Selman, B., H. Kautz, B. Cohen. 1994. Noise strategies for improving local search. Proc. Amer. Assoc. Artificial Intelligence, 337-343.
73. Simon, H. 1955. A behavioral model of rational choice. Quart. J. Econom. 69(1) 99-118.
74. Simon, H. A. 1966. Scientific discovery and the psychology of problem solving. R. Colodny, ed. Mind and Cosmos. University of Pittsburgh Press, Pittsburg, 22-40.
75. Simon, H. 1986. Decision making and problem solving. Research Briefings 1986: Report of the Research Briefing Panel on Decision Making and Problem Solving. National Academy of Sciences, National Academy Press, Washington, D.C.
76. Simon, H. 1990. Invariants of human behavior. Annual Rev. Psych. 41 1-19.
77. Simon, H. 1992. Rational decision-making in business organizations. A. Lindbeck, ed. Nobel Lectures, Economics 1969-1980. World Scientific Publishing, Singapore.
78. Simonton, D. 1995. Foresight in insight? A Darwinian answer. R. J. Sternberg, J. E. Davidson, eds. The Nature of Insight.MIT Press, Cambridge, MA, 465-494.
79. Sterman, J. 1989. Misperceptions of feedback in dynamic decision making. Organ. Behav. Human Decision Processes 43(3) 301-328.
80. Sterman, J., N. Repenning, F. Kofman. 1997. Unanticipated side effects of successful quality programs: Exploring a paradox of organizational improvement. Management Sci. 43(4) 503-521.
81. Sutton, J. 1998. Philosophy and Memory Traces: Descartes to Connectionism. Cambridge University Press, Cambridge, UK.
82. Sutton, R., A. Barto. 1981. An adaptive network that constructs and uses an internal model of its world. Cognition Brain Theory 4(3) 217-246.
83. Sutton, R., A. Barto. 1998. Reinforcement Learning: An Introduction. MIT Press, Cambridge, MA.
84. Thagard, P. 1996. Mind: Introduction to Cognitive Science. MIT Press, Cambridge, MA.
85. Thompson, J. D. 1967. Organizations in Action. McGraw-Hill, New York.
86. Thorndike, E. 1898. Animal intelligence: An experimental study of the associative processes in animals. Psych. Rev. Monograph Supplements 8.
87. Wallas, G. 1926. The Art of Thought. Harcourt, New York.
88. Watkins, C. 1989. Learning from Delayed Rewards. Kings' College, Cambridge, UK.
89. Weber, E., S. Shafir, A.-R. Blais, 2004. Predicting risk sensitivity in humans and lower animals: Risk as variance or coefficient of variation. Psych. Rev. 11 430-445.
90. Yaniv, I., D. Meyer. 1987. Activation and metacognition of inaccessible stored information: Potential bases of incubation effects in problem solving. J. Experiment. Psych. Learn., Memory, Cognition 13 187-205.