A conceptual and computational model of moral decision making in human and artificial agents

Topics in Cognitive Science

Volumn 2, Issue 3, 2010, Pages 454-485

  Wallach, Wendell ^a   Franklin, Stan ^b   Allen, Colin ^c  

^a YALE UNIVERSITY   (United States)

^b UNIVERSITY OF MEMPHIS   (United States)

^c INDIANA UNIVERSITY   (United States)

Author keywords

Artificial general intelligence; Artificial intelligence; Global workspace theory; Machine ethics; Machine morality; Moral decision making

Indexed keywords

ARTIFICIAL INTELLIGENCE; COGNITION; COMPUTER SIMULATION; DECISION MAKING; ETHICS; HUMAN; MORALITY; PSYCHOLOGICAL MODEL;

ARTIFICIAL INTELLIGENCE; COGNITION; COMPUTER SIMULATION; DECISION MAKING; HUMANS; MODELS, PSYCHOLOGICAL; MORALS;

EID: 79957657605     PISSN: 17568757     EISSN: 17568765     Source Type: Journal    
DOI: 10.1111/j.1756-8765.2010.01095.x     Document Type: Article

References (104)

1. Allen, C. (2002). Calculated morality: Ethical computing in the limit. In I. Smit & G. Lasker (Eds.), Cognitive, emotive and ethical aspects of decision making and human action (Vol. I, pp. 19-23). Windsor, ON: IIAS.
2. Allen, C., Smit, I., & Wallach, W. (2006). Artificial morality: Top-down, bottom-up and hybrid approaches. Ethics of New Information Technology, 7, 149-155.
3. Allen, C., Varner, G., & Zinser, J. (2000). Prolegomena to any future artificial moral agent. Journal of Experimental and Theoretical Artificial Intelligence, 12, 251-261.
4. Anderson, J. R. (1990). The adaptive character of thought. Hillsdale, NJ: Erlbaum.
5. Anderson, M., & Anderson, S. (Guest Editors). (2006). Machine ethics. IEEE Intelligent Systems, 21(4), 10-11.
6. Anderson, M., Anderson, S., & Armen, C. (2005). Towards machine ethics: Implementing two action-based ethical theories. In M. Anderson, S. Anderson, & C. Armen (Eds.), Machine ethics (pp. 1-16). Technical Report FS-05-06. Menlo Park, CA: AAAI Press.
7. Anderson, M., Anderson, S., & Armen, C. (2006). An approach to computing ethics. IEEE Intelligent Systems, 21(4), 56-63.
8. Baars, B. J. (1988). A cognitive theory of consciousness. Cambridge, England: Cambridge University Press.
9. Baars, B. J. (2002). The conscious access hypothesis: Origins and recent evidence. Trends in Cognitive Sciences, 6, 47-52.
10. Baars, B. J., & Franklin, S. (2003). How conscious experience and working memory interact. Trends in Cognitive Sciences, 7, 166-172.
11. Baddeley, A. (1992). Consciousness and working memory. Consciousness and Cognition, 1, 3-6.
12. Baddeley, A., Conway, M., & Aggleton, J. (2001). Episodic memory. Oxford, England: Oxford University Press.
13. Baddeley, A. D., & Hitch, G. J. (1974). Working memory. In G. A. Bower (Ed.), The psychology of learning and motivation (pp. 47-89). New York: Academic Press.
14. Barsalou, L. W. (1999). Perceptual symbol systems. Behavioral and Brain Sciences, 22, 577-660.
15. Berne, E. (1964). Games people play: The basic handbook of transactional analysis. New York: Ballantine Books.
16. Breazeal, C. (2002). Designing sociable robots. Cambridge, MA: MIT Press.
17. Bringsjord, S., Arkoudas, K., & Bello, P. (2006). Toward a general logicist methodology for engineering ethically correct robots. IEEE Intelligent Systems, 21(4), 38-44.
18. Brooks, R. A. (2002). Flesh and machines. New York: Pantheon Books.
19. Canamero, L. D. (2003). Designing emotions for activity selection in autonomous agents. In R. Trappl, P. Petta, & S. Payr (Eds.), Emotions in humans and artifacts (pp. 115-148). Cambridge, MA: MIT Press.
20. Clarke, R. (1993). Asimov's Laws of Robotics: Implications for Information Technology (1). IEEE Computer, 26(12), 53-61.
21. Clarke, R. (1994). Asimov's Laws of Robotics: Implications for Information Technology (1). IEEE Computer, 27(1), 57-66.
22. Conway, M. A. (2001). Sensory-perceptual episodic memory and its context: Autobiographical memory. Philos Trans R Soc London 13, 356, 1375-1384.
23. Danielson, P. (1992). Artificial morality: Virtuous robots for virtual games. New York: Routledge.
24. Das, P., Kemp, A. H., Liddell, B. J., Brown, K. J., Olivieri, G., Peduto, A., Gordon, E., & Williams, L. A. (2005). Pathways for fear perception: Modulation of amygdala activity by thalamo-cortical systems. NeuroImage, 26, 141-148.
25. Dehaene, S., Sergent, C., & Changeux, J.-P. (2003). A neuronal network model linking subjective reports and objective physiological data during conscious perception. Proceedings of the National Academy of Sciences of the United States of America, 1001, 8520-8525.
26. DeMoss, D. (1998). Aristotle, connectionism, and the morally excellent brain. Proceedings of the 20th world congress of philosophy, The Paideia Archive. Available at: Accessed March 1, 2010.
27. D'Mello, S. K., Ramamurthy, U., Negatu, A., & Franklin, S. (2006). A procedural learning mechanism for novel skill acquisition. In T. Kovacs & J. A. R. Marshall (Eds.), Workshop on motor development: Proceeding of adaptation in artificial and biological systems, AISB'06 (Vol. 1, pp. 184-185). Bristol, England: Society for the Study of Artificial Intelligence and the Simulation of Behaviour.
28. Drescher, G. L. (1991). Made-up minds: A constructivist approach to artificial intelligence. Cambridge, MA: MIT Press.
29. Edelman, G. M. (1987). Neural Darwinism. New York: Basic Books.
30. Ericsson, K. A., & Kintsch, W. (1995). Long-term working memory. Psychological Review, 102, 211-245.
31. Estes, W. K. (1993). Classification and cognition. Oxford, England: Oxford University Press.
32. Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area of cognitive-developmental inquiry. American Psychologist, 34, 906-911.
33. Franklin, S. (2000a). A "consciousness" based architecture for a functioning mind. Paper presented at the Symposium on Designing a Functioning Mind: Artificial Intelligence and the Simulation of Behaviour, April 2000, Birmingham, England.
34. Franklin, S. (2000b). Deliberation and voluntary action in 'conscious' software agents. Neural Network World, 10, 505-521.
35. Franklin, S. (2003). IDA: A conscious artifact? Journal of Consciousness Studies, 10, 47-66.
36. Franklin, S. (2005a). Cognitive robots: Perceptual associative memory and learning. Proceedings of the 14th Annual International Workshop on Robot and Human Interactive Communication (RO-MAN 2005) (pp. 427-433). Nashville, TN.
37. Franklin, S. (2005b). Evolutionary pressures and a stable world for animals and robots: A commentary on Merker. Consciousness and Cognition, 14, 115-118.
38. Franklin, S. (2005c). Perceptual memory and learning: Recognizing, categorizing, and relating. Symposium on developmental robotics: American Association for Artificial Intelligence (AAAI). Palo Alto, CA: Stanford University.
39. Franklin, S., Baars, B. J., Ramamurthy, U., & Ventura, M. (2005). The role of consciousness in memory. Brains, Minds and Media, 1, 1-38.
40. Franklin, S., & Graesser, A. C. (1997). Is it an agent, or just a program?: A taxonomy for autonomous agents. Proceedings of the third international workshop on agent theories, architectures, and languages, intelligent agents III (pp. 21-35). Berlin: Springer-Verlag.
41. Franklin, S., Kelemen, A., & McCauley, L. (1998). IDA: A cognitive agent architecture. IEEE conference on systems, man and cybernetics (pp. 2646-2651). IEEE Press.
42. Franklin, S., & McCauley, L. (2003). Interacting with IDA. In H. Hexmoor, C. Castelfranchi, & R. Falcone (Eds.), Agent autonomy (pp. 159-186). Dordrecht, The Netherlands: Kluwer.
43. Franklin, S., & Patterson, F. G. Jr. (2006). The LIDA architecture: Adding new modes of learning to an intelligent, autonomous, software agent. IDPT-2006 Proceedings (Integrated Design and Process Technology). San Diego, CA: Society for Design and Process Science.
44. Franklin, S., & Ramamurthy, U. (2006). Motivations, values and emotions: Three sides of the same coin. In Proceedings of the sixth international workshop on epigenetic robotics (Vol. 128, pp. 41-48). Paris: Lund University Cognitive Studies.
45. Franklin, S., Ramamurthy, U., D'Mello, S. K., McCauley, L., Negatu, A., Silva, R.L., & Datla, V. (2007, November 9-11). LIDA: A computational model of global workspace theory and developmental learning. Paper presented at the AAAI fall symposium on AI and consciousness: Theoretical foundations and current approaches, Arlington, VA.
46. Freeman, W. J. (1999). How brains make up their minds. London: Weidenfeld and Nicolson General.
47. Friedlander, D., & Franklin, S. (2008). LIDA and a theory of mind. In P. Wang, B. Goertzel, & S. Franklin (Eds.), Artificial general intelligence 2008 (pp. 137-148). Amsterdam: IOS Press.
48. Gadanho, S. C. (2003). Learning behavior-selection by emotions and cognition in a multi-goal robot task. Journal of Machine Learning Research, 4, 385-412.
49. Gibson, J. J. (1979). The ecological approach to visual perception. Mahwah, NJ: Lawrence Erlbaum Associates.
50. Gips, J. (1991). Towards the ethical robot. In K. Ford, C. Glymour, & P. Hayes (Eds.), Android epistemology (pp. 243-252). Cambridge, MA: MIT Press.
51. Glenberg, A. M. (1997). What memory is for. Behavioral and Brain Sciences, 20, 1-19.
52. Goodale, M. A., & Milner, D. (2004). Sight unseen. Oxford, England: Oxford University Press.
53. Grau, C. (2006). There is no 'I' in 'robot': Robots and utilitarianism. IEEE Intelligent Systems, 21(4), 52-55.
54. Guarini, M. (2006). Particularism and classification and reclassification of moral cases. IEEE Intelligent Systems, 21(4), 22-28.
55. Heilman, K. M. (1997). The neurobiology of emotional experience. Journal of Neuropsychiatry and Clinical Neurosciences, 9, 439-448.
56. Hofstadter, D. R., & Mitchell, M. (1995). The copycat project: A model of mental fluidity and analogy-making. In K. J. Holyoak & J. Barnden (Eds.), Advances in connectionist and neural computation theory, Vol. 2: Logical connections (pp. 205-267). New York: Basic Books.
57. Holland, O. (Ed.) (2003). Machine consciousness. New York: Imprint Academic.
58. Jackson, J. V. (1987). Idea for a mind. ACM SIGART Bulletin, 191, 23-26.
59. James, W. (1890). The principles of psychology. Cambridge, MA: Harvard University Press.
60. Johnston, V. S. (1999). Why we feel: The science of human emotions. Reading, MA: Perseus Books.
61. Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996). Reinforcement learning: A survey. Journal of Artificial Intelligence Research, 4, 237-285.
62. Kanerva, P. (1988). Sparse distributed memory. Cambridge, MA: MIT Press.
63. Kruschke, J. K. (2003). Attention in learning. Current Directions in Psychological Science, 12(5), 171-175.
64. Laird, J. E., Newell, A., & Rosenbloom, P. S. (1987). SOAR: An architecture for general intelligence. Artificial Intelligence, 33, 1-64.
65. Massimini, M., Ferrarelli, F., Huber, R., Esser, S. K., Singh, H., & Tononi, G. (2005). Breakdown of cortical effective connectivity during sleep. Science, 309, 2228-2232.
66. McLaren, B. (2006). Computational models of ethical reasoning: Challenges, initial steps, and future directions. IEEE Intelligent Systems, 21(4), 29-37.
67. Merker, B. (2005). The liabilities of mobility: A selection pressure for the transition to consciousness in animal evolution. Consciousness and Cognition, 14, 89-114.
68. Minsky, M. (1985). The society of mind. New York: Simon and Schuster.
69. Mulcahy, N. J., & Call, J. (2006). Apes save tools for future use. Science, 312, 1038-1040.
70. Nadel, L. (1992). Multiple memory systems: What and why. Journal of Cognitive Neuroscience, 4, 179-188.
71. Nadel, L., & Moscovitch, M. (1997). Memory consolidation, retrograde amnesia and the hippocampal complex. Current Opinion in Neurobiology, 7, 217-227.
72. Negatu, A., D'Mello, S. K., & Franklin, S. (2007). Cognitively inspired anticipation and anticipatory learning mechanisms for autonomous agents. In M. V. Butz, O. Sigaud, G. Pezzulo, & G. O. Baldassarre (Eds.), Proceedings of the third workshop on Anticipatory Behavior in Adaptive Learning Systems (ABiALS 2006) (pp. 108-127). Rome: Springer-Verlag.
73. Negatu, A., & Franklin, S. (2002). An action selection mechanism for 'conscious' software agents. Cognitive Science Quarterly, 2, 363-386.
74. Ornstein, R. (1986). Multimind. Boston: Houghton Mifflin.
75. Picard, R. (1997). Affective computing. Cambridge, MA: MIT Press.
76. Powers, T. (2006). Prospects for a Kantian machine. IEEE Intelligent Systems, 21(4), 46-51.
77. Ramamurthy, U., Baars, B. J., D'Mello, S. K., & Franklin, S. (2006). LIDA: A working model of cognition. In D. Fum, F. Del Missier, & A. Stocco (Eds.), Proceedings of the 7th international conference on cognitive modeling (pp. 244-249). Trieste: Edizioni Goliardiche.
78. Ramamurthy, U., D'Mello, S. K., & Franklin, S. (2004). Modified sparse distributed memory as transient episodic memory for cognitive software agents. IEEE international conference on Systems, Man and Cybernetics-SMC2004, The Hague, The Netherlands.
79. Ramamurthy, U., D'Mello, S. K., & Franklin, S. (2005). Role of consciousness in episodic memory processes: Poster. Ninth conference of the Association for the Scientific Study of Consciousness-ASSC9, Pasadena, CA.
80. Scassellati, B. M. (2001). Foundations for a theory of mind for a humanoid robot. PhD thesis, Department of Electrical Engineering and Computer Science, MIT. Available at: Accessed March 1, 2010.
81. Shanahan, M. (2006). A cognitive architecture that combines internal simulation with a global workspace. Consciousness and Cognition, 15, 433-449.
82. Sigman, M., & Dehaene, S. (2006). Dynamics of the central bottleneck: Dual-task and task uncertainty. PLoS Biology, 4(7), e220.
83. Sloman, A. (1998). Damasio, Descartes, alarms and meta-management. Proceedings symposium on cognitive agents: Modeling human cognition. San Diego, CA: IEEE.
84. Sloman, A. (1999). What sort of architecture is required for a human-like agent? In M. Wooldridge & A. S. Rao (Eds.), Foundations of rational agency (pp. 35-52). Dordrecht, The Netherlands: Kluwer Academic Publishers.
85. Smith, J. D., & Washburn, D. A. (2005). Uncertainty monitoring and metacognition by animals. Current Directions in Psychological Science, 14, 19-24.
86. Stahl, B. C. (2002). Can a computer adhere to the categorical imperative? A contemplation of the limits of transcendental ethics in IT. In I. Smit & G. Lasker (Eds.), Cognitive, emotive and ethical aspects of decision making in humans and in artificial intelligence (Vol. I, pp. 13-18). Windsor, ON: IIAS.
87. Stickgold, R., & Walker, M. P. (2005). Memory consolidation and reconsolidation: What is the role of sleep? Trends in Neuroscience, 28, 408-415.
88. Sun, R. (2007). The importance of cognitive architectures: An analysis based on CLARION. Journal of Experimental and Theoretical Artificial Intelligence, 19(2), 159-193.
89. Tarsitano, M. (2006). Route selection by a jumping spider (Portia labiata) during the locomotory phase of a detour. Animal Behavior, 72, 1437-1442.
90. Tulving, E. (1983). Elements of episodic memory. Oxford, England: Clarendon Press.
91. Uchida, N., Kepecs, A., & Mainen, Z. F. (2006). Seeing at a glance, smelling in a whiff: Rapid forms of perceptual decision making. Nature Reviews Neuroscience, 7, 485-491.
92. Varela, F. J., Thompson, E., & Rosch, E. (1991). The embodied mind. Cambridge, MA: MIT Press.
93. Vidnyánszky, Z., & Sohn, W. (2003). Attentional learning: Learning to bias sensory competition [abstract]. Journal of Vision, 3, 174a.
94. Wallach, W., & Allen, C. (2009). Moral machines: Teaching robots right from wrong. New York: Oxford University Press.
95. Wallach, W., Allen, C., & Smit, I. (2008). Machine morality: Bottom-up and top-down approaches for modelling human moral faculties. AI and Society, 22(4), 565-582.
96. Wang, P., Goertzel, B., & Franklin, S. (2008). Artificial general intelligence 2008. Amsterdam: IOS Press.
97. Watt, D. F. (1998). Affect and the limbic system: Some hard problems. Journal of Neuropsychiatry and Clinical Neurosciences, 10, 113-116.
98. Werdenich, D., & Huber, L. (2006). A case of quick problem solving in birds: String pulling in keas, Nestor notabilis. Animal Behaviour, 71, 855-863.
99. Wilcox, S., & Jackson, R. (2002). Jumping spider tricksters: Deceit, predation, and cognition. In M. Bekoff, C. Allen, & G. M. Burghardt (Eds.), The cognitive animal (pp. 27-33). Cambridge, MA: MIT Press.
100. Willis, J., & Todorov, A. (2006). First impressions: Making up your mind after a 100-ms exposure to a face. Psychological Science, 17, 592-599.
101. Yoshida, H., & Smith, L. B. (2003). Known and novel noun extensions: Attention at two levels of abstraction. Child Development, 76(2), 564-577.
102. Yudkowsky, E. (2001). What is friendly AI? Available at: Accessed March 1, 2010.
103. Zacks, J. M., Speer, N. K., Swallow, K. M., Braver, T. S., & Reynolds, J. R. (2007). Event perception: A mind-brain perspective. Psychological Bulletin, 133(2), 273-293.
104. Zhu, J., & Thagard, P. (2002). Emotion and action. Philosophical Psychology, 15, 19-36.