Computer systems and responsibility: A normative look at technological complexity

Ethics and Information Technology

Volumn 7, Issue 2, 2005, Pages 99-107

  Johnson, Deborah G ^a   Powers, Thomas M ^b  

^a UNIVERSITY OF VIRGINIA   (United States)

^b UNIVERSITY OF DELAWARE   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

HARDWARE AND SOFTWARE; HUMAN BEING; MORAL RESPONSIBILITY; SYSTEM COMPLEXITY; SYSTEM DESIGNERS; TECHNOLOGICAL COMPLEXITY;

PHILOSOPHICAL ASPECTS;

COMPUTER SYSTEMS;

EID: 31344458722     PISSN: 13881957     EISSN: 15728439     Source Type: Journal    
DOI: 10.1007/s10676-005-4585-0     Document Type: Article

References (21)

1. The term 'moral patient' is used to refer to beings who are the recipients of action in the sense that they can be harmed or helped and can have their interests impeded or furthered. Human beings are moral patients; it is a contentious matter as to whether other animals or the environment can be moral patients.
2. See Brian Cantwell Smith, "The Limits of Correctness in Computers," CSLI Technical Reports 85-35, Center for the Study of Language and Information, Stanford, CA, 1985;
3. a version is also published as "The Limits of Correctness," ACM SIGCAS Journal Computers and Society, combined (14)4 and (15)1, 2, 3 (1985):18-26.
4. In this paper we use 'technology' to refer to the artifact, though we acknowledge that social and historical studies of technology point to a broader definition of technology as socio-technical systems. Here, however, 'technology' and 'artifact' are interchangeable terms.
5. The multiple meanings of responsibility have been explored by H.L.A. Hart in his Punishment and Responsibility (Oxford University Press: 1968),
6. and by Hart and A.M. Honoré in Causation in the Law (2nd ed. Oxford, Clarendon: 1985). Our account of the various contents of the concept of responsibility is indebted to these influential discussions, but we do not follow them strictly here.
7. The first versions of a rootkit (which was originally a proper name for the first-generation malware) were written for Sun and Linux operating systems and released in 1994. Since that time, versions of rootkits have been developed for other operating systems, including those of Microsoft Windows. For details on the history and computational methodology of rootkits, see David O'Brien, "Recognizing and Recovering from Rootkit Attacks," Sys Admin 5(11), November 1996, 8-20;
8. also accessible at http://www.cs.wright.edu/people/faculty/pmateti/Courses/499/ Fortification/obrien.html
9. Some versions of rootkits used against Microsoft Windows products use compromised versions of Microsoft's own proprietary software to hide their intrusions. These types of rootkits then have the ability to place infected files onto the attacked computer system, to log all keystrokes at the system console, and to hide files and directories on hard drive of the computer. See Bartosz Bobkiewicz, "Hidden Backdoors, Trojan Horses and Rootkit tools in a Windows Environment," (article 1122, January 23 2003) at www.windowssecurity.com.
10. Excerpted from "Security Watch" of September 2004 in Network Magazine India, accessible at http://www.networkmagazineindia.com/200409
11. The arguments are stronger than similar ones charging gun makers with responsibility for harm caused by guns, primarily because there are no legitimate uses of hacking scripts and rootkits. There is also the counter-factual argument that, had the script maker not made the script, the kiddie could not have launched the attack. Finally, it seems plausible that the causal sequence of the attack (though not the causal sequence of the victim's vulnerability) begins with the script maker coding the script.
12. There may be scripts that have another purpose, namely to find intruders. However, this use of scripts would not be necessary were there no scripts.
13. See chapter II, "Causation and Common Sense," in H.L.A. Hart and A.M. Honoré in Causation in the Law (2nd ed. Oxford, Clarendon: 1985)
14. Ibid., p. 26.
15. Our account indicates a way in which a technological artifact can create an abnormal condition, and hence a cause of an untoward event. However, the distinction between normal and abnormal conditions can cut the other way. Hart and Honoré write that "what is taken as normal for the purpose of the distinction between cause and effect is very often an artefact of human habit, custom, or convention. This is so because men have discovered that nature is not only sometimes harmful if we intervene, but is also sometimes harmful unless we intervene, and have developed customary techniques, procedures, and routines to counteract such harm" (Ibid. p. 35). Storm shelters, for instance, are technological artifacts which protect from untoward events, and are thus "normal conditions" in areas prone to violent storms.
16. NASA and the National Academies have been actively pursuing computerized air traffic control for several years. See Heinz Erzberger, "Transforming the NAS: The Next Generation Air Traffic Control System" Proceedings of the International Congress of the Aeronautical Sciences (ICAS) 2004;
17. accessible at http://www.ctas.arc.nasa.gov/publications/papers/erzberger_08_04.pdf;
18. "The Future of Air Traffic Control: Human Factors and Automation (Report of the Special Panel of the National Research Council) National Academies Press, Washington D.C. 1998.
19. In the aforementioned article by Erzberger (a NASA scientist), he writes that "ground-based computer logic ensures that all uplinked trajectories will be mutually conflict-free" (p. 2). Unfortunately, this optimism in computer logic may be misplaced, since there currently are no correctness proofs for computer programs of the complexity required by robust automated air traffic control systems.
20. This is one plausible conclusion from the Congressional testimony of Christopher D. Wickens concerning the aforementioned National Research Council report. Wickens notes that, in emergency situations where human controller intervention is called for, "the controller may be less effective in intervention, if decision making has been highly automated" accessible at http://www7.nationalacademies.org/ocga/testimony/Air_Traffic_Control.asp
21. See M.J. van den Hoven, Towards ethical principles for designing political-administrative information systems, Normalization and the Public Sector 3 (1994): 353-373.