Composing software components: A software-testing perspective

Composing Software Components: A Software-testing Perspective

Volumn , Issue , 2010, Pages 1-368

  Hamlet, Dick ^a  

^a PORTLAND STATE UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84891444900     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1007/978-1-4419-7148-7     Document Type: Book

References (101)

1. Adams, E.N.: Optimizing preventive service of software products. IBM J. Research and Development 28, 2-14 (1984)
2. Addis, W.: Structural Engineering: The Nature of Theory and Design. Ellis Horwood (1991) Although Addis's book is out of print, it is well worth seeking in the library. He holds joint degrees in civil engineering and philosophy and is one of only a handful of writers on 'philosophy of engineering.' His historical studies of structural engineering are rich and detailed. This book is the only one I know that tries to explain what 'engineering theory' is and how it relates to practice. He makes a strong case that for an engineer, 'theory' is the same as 'design rules,' the codified ways that design is to be carried out. The fascinating insight Addis proposes is that good design rules are not necessarily in agreement with reality. If they can be followed easily and include adequate safety factors from past experience, they can be quite at odds with physical laws yet remain in successful use for years, or even centuries. Lorenzo Strigini first put me onto the book.
3. Addis, W.: Creativity and Innovation: The Structural Engineer's Contribution to Design. Architectural Press (2001)
4. Allen, R., Garlan, D.: A formal basis for architectural connection. ACM Trans. on Softw. Eng. Methodology 6, 213-249 (1997)
5. Allen, S.F., Constable, R.L., Eaton, R., Kreitz, C., Lorigo, L.: The NUPRL open logical environment. In: Proceedings 17th International Conference on Automated Deduction, LNAI 1831, pp. 170-176. Springer Verlag (2000)
6. Antoy, S., Hamlet, R.G.: Automatically checking an implementation against its formal specification. IEEE Trans. on Softw. Eng. 26, 55-69 (2000) The implementations of this paper are abstract data types written in C++ and the formalism is that of algebraic equations suitably restricted to eliminate difficulties like non-confluence. Its major contribution is Antoy's idea that the abstraction/representation mapping relating the concrete implementation of a data type to its abstract mathematical structure should be explicitly made part of the implementation. This allows random testing to be completely automated, thus extending the ideas of DAISTS [30] and Mills [29]. The example used for illustratation exposes the weakness of random testing in the presence of state that it may fail to use enough long sequences.
7. Basili, V., Turner, A.: Iterative enhancement: A practical technique for software development. IEEE Trans. on Softw. Eng. 1(4), 390-396 (1975)
8. Boehm, B.W.: Software engineering. IEEE Trans. on Computers pp. 1226-1241 (1976)
9. Boehm, B.W.: A spiral model of software development and enhancement. IEEE Computer pp. 61-72 (1988)
10. Boehm, C., Jacopini, G.: Flow diagrams, Turing machines, and languages with only two formation rules. Comm. of the ACM 9, 366-371 (1966)
11. Boland, P., Singh, H., Cukik, B.: Comparing partition and random testing via majorization and schur functions. IEEE Trans. on Softw. Eng. 29, 88-94 (2003)
12. Bolton, F.: Pure Corba. Sams (2001)
13. Butler, R.W., Finelli, G.B.: The infeasibility of experimental quantification of life-critical software reliability. IEEE Trans. on Softw. Eng. 19(1), 3-12 (1993) The clear conclusion of this well-thought-out paper is that 'ultrareliability,' that is, failure rates below about 10?8/run, cannot be investigated using testing. The number of test points required to obtain confidence in ultrareliabilty is too large to be practical, even under the best possible circumstances. The roughly 107 seconds in a work-year are not enough to conduct the roughly 109 necessary test runs. The paper considers a number of ingenious ways to try to get around this stubborn fact, and shows that each must fail. The technical device used to obtain results is hypothesis testing, which makes the derivations more forbidding than need be. Most of the results could be obtained with less statistical machinery using failure-rate bounds and upper confidence bounds as in Chapter 12.
14. Chen, T.Y., Leung, H., Mak, I.K.: Adaptive random testing. In: Proceedings of the 9th Asian Computing Science Conference, Lecture Notes in Computer Science, vol. 3321, pp. 320-329. Springer-Verlag (2004)
15. Chen, T.Y., Merkel, R.: An upper bound on software testing effectiveness. IEEE Trans. on Softw. Eng. (2008). (to appear)
16. Chen, T.Y., Tse, T.H., Zhou, Z.: Semi-proving: an integrated method based on global symbolic evaluation and metamorphic testing. SIGSOFT Softw. Eng. Notes 27(4), 191-195 (2002)
17. Cobb, R.H., Mills, H.D.: Engineering software under statistical quality control. IEEE Software pp. 44-54 (1990) Although some of the mathematics is garbled in the printed version, this paper makes a good case for prioritizing software failures for attention by frequency of occurrence. The paper popularizes very original work by Adams [1] in which he suggests that many failures are simply not worth fixing-they will probably never occur again. My own experience with operating-system crashes agrees: Bug after bug is fixed, often with great difficulty, but the failure rate of the system hardly improves because there is a large supply of very lowfrequency failures.
18. Cohen, D.M., Dalal, S.R., Fredman, M.L., Patton, G.C.: The aetg system: An approach to testing based on combinatorial design. IEEE Trans. on Softw. Eng. 23(7), 437-444 (1997)
19. Cook, S.A., Reckhow, R.A.: Time-bounded random access machines. Journal of Computer Systems Science 7, 354-375 (1973)
20. Davis, M.: The Undecidable. Raven Press (1965) Davis has collected the papers of Gödel, Church, Turing, and Post written in the 1930s, which define the idea of mechanical or 'rote' computation, and explore its limits. Certainly these papers are among the most important and difficult in the history of mathematical logic. It is impossible to be sure of the effect this work had on the development of electronic computers in the following decade. It certainly did not suggest how real machines should be built- von Neumann's design is nothing like a Turing machine or a Post system, closer in fact to Babbage's mechanical engine-but the idea that from the simplest operations all possible computations can be built may have been important.
21. DeMillo, R.A., Lipton, R.J., Sayward, F.G.: Hints on test data selection: Help for the practicing programmer. Computer 11, 34-41 (1978)
22. Duran, J., Ntafos, S.: An evaluation of random testing. IEEE Trans. on Softw. Eng. 10, 438-444 (1984) Joe Duran was an early and tireless advocate of random testing, whose insights are well expressed in this paper. Its contribution was a way of analyzing subdomain testing (called "partition testing" in this and other papers) by simulating its distributions. It is not an exaggeration to say that this paper opened up a new area, generating many similar publications (some say too many), e.g., [55, 11]. The most recent development comes from T-Y. Chen and his co-workers, who are beginning to find fundamental explanations for the limitations of systematic testing [15], and perhaps will be able to unravel the way in which so-called 'faults' are expressed in a program's input space.
23. Ernst, M., Cockrell, J., Griswold,W.G., Notkin, D.: Dynamically discovering likely program invariants to support program evolution. IEEE Trans. on Softw. Eng. 27, 99-123 (2001)
24. Fagan, M.E.: Design and code inspections to reduce errors in program development. IBM Systems Journal 15(3), 182-211 (1976)
25. Floyd, R.W.: Assigning meanings to programs. In: Proceedings Symposium Applied Mathematics, vol. 19, pp. 19-32. Amer. Math. Soc (1967) Floyd's paper is generally credited with originating program proving, although Turing [95, ?A-] and others had provided earlier examples. I first read the paper as a grad student, and I remember telling my thesis advisor that I was sure Floyd was onto something truly wonderful. I had previously written a paper for a class in which I attempted to describe Algol programs in first-order logic, a paper that received a B+ with the comment that the professor (Buzz Hunt) didn't think it was practical, so I was in a position to appreciate just how well Floyd had done. So far, Hunt appears to have been right.
26. Foster, K.A.: Error sensitive test cases analysis (estca). IEEE Trans. on Softw. Eng. 6(3), 258-264 (1980) Foster's background was in hardware testing, and he was seeking to find for software an analogy to hardware tests for so-called 'stuck-at' faults (in fabrication technology that has now changed). It is a lack of fault models-details of how things might go wrong-that makes software testing very different from hardware testing. For hardware the engineer knows what might fail and tests for it; when it's not found all is well. For software the catalog of possible problems seems unlimited, so when all tests succeed it means almost nothing. The difference is that in hardware manufacture the whole point is to control and confine the possibilities for mistakes, but in software the point is to allow a human programmer as much power as possible.
27. Frankl, P.G.,Weiss, S.N.: An experimental comparison of the effectiveness of branch testing and data flow testing. IEEE Trans. on Softw. Eng. 19(8), 774-787 (1993)
28. Frankl, P.G.,Weyuker, E.J.: An applicable family of data flow testing criteria. IEEE Trans. on Softw. Eng. 14, 1483-1498 (1988)
29. Gannon, J., Hamlet, D., Mills, H.: Theory of modules. IEEE Trans. on Softw. Eng. 13, 820-829 (1987)
30. Gannon, J., Hamlet, R., McMullin, P.: Data abstraction implementation, specification, and testing. ACM Trans. Prog. Lang. and Systems 3, 211-223 (1981)
31. Geller, M.M.: Test data as an aid in proving program correctness. Comm. of the ACM pp. 368-375 (1978) For many years this paper stood alone as a serious attempt to aid program proofs using test data. (Other 'testing/proving' papers were mostly the obverse.) Geller recognized that in special cases a few test points constitute a proof in conjunction with knowledge of the program code. For example, if some part of the code can be seen to be implementing a linear function, it is sufficient to test two points to show that it is the correct line. This kind of analysis is exactly what takes place in the best unit tests. But Geller was able to come up with only a few examples, mostly for the integer data type. The current work in metamorphic testing [16] may be the long-sought generalization of this promising start.
32. Gelprin, D., Hetzel, B.: The growth of software testing. Comm. of the ACM 31(6), 687-695 (1988)
33. Gerhart, S.L., Yelowitz, L.: Observations of fallibility in applications of modern programming methodologies. IEEE Trans. on Softw. Eng. 2(3), 195-207 (1976)
34. Goodenough, J.B., Gerhart, S.L.: Toward a theory of test data selection. IEEE Trans. on Softw. Eng. 1, 156-173 (1975) The tentative title correctly indicates that the authors present only the beginnings of a theory, long on definitions but short on theorems. They assume that programs have neither persistent state nor concurrency. Nevertheless, this paper set the standard for practially all subsequent theoretical work in software testing. It was a revelation because testing was treated using a functional mathematical model, in contrast to other treatments that were merely exposition of practical methods. The exciting prospect that opened up was the possibility of proving results rather than supporting them only with empirical studies. The workshop on Testing And Verification (TAV) first organized by Susan Gerhart and Bill Howden in Ft. Lauderdale, FL in 1979, which eventually evolved into the International Symposium on Software Testing and Analysis (ISSTA), had this paper and Howden's on subdomain testing [64] as prime motivation.
35. Gorton, I., Heineman, G.T., Crnkovic, I., Schmidt, H.W., Stafford, J.A., Szyperski, C., Wallnau, K. (eds.): Component-based software engineering, 9th Int. Symposium, CBSE9, LNCS 4063. Springer (2006)
36. Guy Steele, J., Woods, D., Finkel, R., Crispin, M., Stallman, R., Goodfellow, G.: The Hacker's Dictionary, A guide to the World of Computer Wizards. Harper & Row (1983) This original version, like Algol 60 in Hoare's aphorism, is a great improvement on its successor The New Hacker's Dictionary. It is out of print, but the text is mostly preserved at http://www.dourish.com/ goodies/jargon.html. It is fun to read, particularly for former DEC PDP-10 assembly language programmers (if you did not have the good fortune to be at Stanford or MIT in the glory days). Many entries supply the obscure etymology for jargon in common use today. Some personal favorites are HACK, LOGICAL, MAGIC, RANDOM.
37. Halstead, M.H.: Machine-Independent Computer Programming. Spartan Books, Washington, DC (1962)
38. Hamlet, D.: "determining" tests. In: Proceedings Workshop on Effectiveness of Testing and Proving Methods, pp. 87-93. Avalon, CA (1982)
39. Hamlet, D.: Software component dependability, a subdomain-based theory. Tech. Rep. RSTR-96-999-01, Reliable Software Technologies, Sterling, VA (1996)
40. Hamlet, D.: What can we learn by testing a program? In: Proc. ISSTA, pp. 50-52 (1998)
41. Hamlet, D.: On subdomains: testing, profiles, and components. In: Proceedings ISSTA '00, pp. 71-76. Portland, OR (2000)
42. Hamlet, D.: Continuity in software systems. In: Proceedings ISSTA '02, pp. 196-200. Rome (2002)
43. Hamlet, D.: Invariants and state in testing and formal methods. In: Proceedings Workshop on Program Analysis for Software Tools and Engineering (PASTE), pp. 48-51. Lisbon, Portugal (2005)
44. Hamlet, D.: Subdomain testing of units and systems with state. In: Proceedings ISSTA 2006, pp. 85-96. Portland, ME (2006)
45. Hamlet, D.:When only random testing will do. In: Proceedings First InternationalWorkshop on Random Testing. Portland, ME (2006)
46. Hamlet, D.: Software component composition: subdomain-based testing-theory foundation. J. Software Testing, Verification and Reliability 17, 243-269 (2007) A detailed presentation of the subdomain-based composition theory, but only for the stateless, non-concurrent case. The SYN tools are mentioned only in passing. The paper includes a brief review of related work.
47. Hamlet, D.: Collaboration among software components. In: Proceedings 27th Annual Pacific Northwest Software Quality Conference. Portland, OR (2008)
48. Hamlet, D.: Tools and experiments supporting a testing-based theory of component composition. ACM Trans. on Softw. Eng. Methodology 18 (2009) This publication was originally intended as the experimental companion to the stateless theory [46]. Its primary purpose was to investigate the behavior of measurement and prediction errors as subdomains shrink. But long delays in publication and referee demands allowed it to include cases with state (including a minimal theoretical treatment of state), and other case studies, for example on substituting one component for another. The SYN tools are heavily used, but without any attention to them in their own right.
49. Hamlet, D., Andric, M., Tu, Z.: Experiments with composing component properties. In: K. Wallnau (ed.) Proc. 6th ICSE Workshop on Component-based Software Engineering. Portland, OR (2003). http://www.sei.cmu.edu/pacc
50. Hamlet, D., Mason, D., Woit, D.: Foundational theory of software component reliability. In: Proc. 10th International Symposium on Software Reliability Engineering (ISSRE'99)-Fast Abstracts. Boca Raton, FL (1999)
51. Hamlet, D., Mason, D., Woit, D.: Theory of software reliability based on components. In: Proc. 3rd ICSE Workshop on Component-based Software Engineering [98]
52. Hamlet, D., Mason, D., Woit, D.: Theory of software reliability based on components. In: Proceedings ICSE '01, pp. 361-370. Toronto, Canada (2001) Although papers in the CBSE workshop [51] and ISSTA [41] first presented information on the subdomain theory of component composition that is the subject of the present monograph, its first detailed presentation was at ICSE 2001. (An identical paper was submitted to ICSE 2000 but rejected-the authors like to think this meant it was ahead of its time, but it probably only illustrates the wildly inconsistent reviewing of software engineering papers.) A slightly modified version appears in Lau's collection [53]. There are three serious mistakes in this initial paper: (1) The non-functional property described is reliability rather than run time, which lets in all the issues described in Chapter 12, detracting from the theory; (2) It was not realized that the synthesis of loop approximations is algorithmic, so the paper is weaker than it might have been; (3) The use of a 'transfer matrix' from one component's subdomains to another following in series does not generalize to further compositions, compromising the paper's claim that an arbitrary system can be synthesized. It was the implementation of the SYN tools that exposed these mistakes.
53. Hamlet, D., Mason, D., Woit, D.: Properties of software systems synthesized from components, chap. 6. In: Lau [70] (2004). An updated version of [52].
54. Hamlet, D., Maybee, J.: The Engineering of Software. Addison-Wesley (2001)
55. Hamlet, D., Taylor, R.: Partition testing does not inspire confidence. IEEE Trans. on Softw. Eng. 16, 1402-1411 (1990)
56. Hamlet, R.G.: Introduction to Computation Theory. Intext Educational Publishers (1974)
57. Hamlet, R.G.: Testing programs with the aid of a compiler. IEEE Trans. on Softw. Eng. pp. 279-289 (1977)
58. Hamlet, R.G., Haralick, R.M.: Transportable package software. Software-Practice & Experience pp. 1009-1027 (1980)
59. Harel, D.: Statecharts: A visual formalism for complex systems. Science of Computer Programming 8, 231-274 (1987)
60. Heninger, K.L.: Specifying software requirements for complex systems: new techniques and their applications. IEEE Trans. on Softw. Eng. 6, 2-13 (1980)
61. Hoare, C.A.R.: An axiomatic basis for computer programming. Comm. of the ACM 12, 576-583 (1969)
62. Hoare, C.A.R.: Communicating sequential processes. Comm. of the ACM 21(8), 666-677 (1978)
63. Howden, W.: Methodology for the generation of program test data. IEEE Trans. Computers 24, 554-559 (1975)
64. Howden, W.E.: Reliability of the path analysis testing strategy. IEEE Trans. on Softw. Eng. 2, 208-215 (1976) In the title, "Reliability" does not refer to the engineering measure of probable success, but to the social-science notion that various attempts to judge something will agree with each other, often applied to diagnosis of mental illness. The various attempts are test-point choices from subdomains, and they agree if they all fail or all succeed; then in Howden's terminology such subdomains are "reliable." Despite the unfortunate choice of a technical word (in hindsight), this notion is just what a tester wants from a subdomain breakdown: not to be misled by success; to miss no failures. Using a functional model to define test success, Howden formally defines subdomain testing, then experimentally investigates the path-coverage subdomains for a set of programs containing common beginners' mistakes. His mathematics is sound and precise and the examples give an excellent insight into why subdomains sometimes work for testing and sometimes don't. The workshop on Testing And Verification (TAV) first organized by Susan Gerhart and Bill Howden in Ft. Lauderdale, FL in 1979, which eventually evolved into the International Symposium on Software Testing and Analysis (ISSTA), had this paper and Goodenough and Gerhart's [34] as prime motivation. A short summary and appreciation of Howden's paper appears in a retrospective session of ISSTA 98 [40].
65. Howden, W.E.: Weak mutation testing and completeness of test sets. IEEE Trans. on Softw. Eng. 8, 371-379 (1982)
66. Kleene, S.C.: Introduction to Metamathematics. Elsevier (1980)
67. Knight, J., Cass, A., Fernandez, A., Wika, K.: Testing a safety-critical application. In: Proceedings ISSTA '94, p. 199. Seattle, WA (1994)
68. Knuth, D.E.: Literate programming. The Computer Journal 27(2), 97-111 (1984)
69. Lamport, L.: How to tell a program from an automobile. http://research.microsoft.com /en-us/um/people/lamport/pubs/automobile.pdf (1997)
70. Lau, K.K. (ed.): Case Studies in Computer-based Software Engineering. World Scientific (2004)
71. Leveson, N.G., Cha, S.S., Knight, J.C., Shimeall, T.J.: The use of self checks and voting in software detection: An empirical study. IEEE Trans. on Softw. Eng. 16, 432-443 (1990)
72. Linger, R.C.,Mills, H.D.,Witt, B.I.: Structured programming, theory and practice. Addison-Wesley (1979)
73. Lions, J.L.: ARIANE 5-Flight 501 Failure-Report by the Inquiry Board. European Space Agency (ESA), Paris (1996)
74. Lyu, M. (ed.): Handbook of Software Reliability Engineering. McGraw-Hill, New York (1996)
75. Manna, Z., McCarthy, J.: Properties of programs and partial function logic. In: B. Meltzer, D. Michie (eds.) Machine Intelligence 5, pp. 27-39. Edinburgh University Press (1969)
76. Marick, B.: Experience with the cost of different coverage goals for testing. In: Pacific Northwest Software Quality Conference, pp. 147-164. Portland, OR (1991)
77. Mason, D.: Probabilistic program analysis for software component reliability. Ph.D. thesis, University of Waterloo (2002)
78. Mason, D., Woit, D.: Software system reliability from component reliability. In: Proc. of 1998 Workshop on Software Reliability Engineering (SRE'98). Ottawa, Ontario (1998)
79. Medvidovic, N., Mehta, N., Mikic-Rakic, M.: A family of software architecture implementation frameworks. In: Proc. 3rd IFIP working int. conf. on software architectures (WICSA), pp. 221-235. Montreal, Canada (2002)
80. Meinke, K.: Automated black-box testing of functional correctness using function approximation. In: Proceedings ISSTA '04, pp. 143-153. Boston (2004)
81. Meyer, B.: Object-oriented Software Construction. Prentice Hall (2000)
82. Mills, H., Basili, V., Gannon, J., Hamlet, D.: Principles of Computer Programming: A Mathematical Approach. Allyn and Bacon (1987) Although intended as a textbook at the freshman (!) level, this is the best exposition of Harlan Mills's denotational-semantic calculus for the Pascal programming language. It first defines a minimal language subset ("CF Pascal" for "Character-File") that reduces the detail required in proof examples, then treats (most of) the full language. Two features of the formalism recommend it over (say) Hoare's axiomatic treatment: (1) The mathematical basis is functions rather than relations, and (2) The rules for proving loops are much easier to use than assertions. (The loop function must still be guessed, but checking it is more straightforward than the proof that Hoare logic requires.) The 'box' notation was introduced in this book; it is an adaptation of Kleene's [66] notational way of distinuishing a program (Gödel number) from the (partial recursive) function it computes. (Horrors! Most computer science papers confuse the two notationally.) As befits a textbook, there are numerous exercises. Mills probably did not invent his calculus, but the extensive development in this text was new. At the same time the book was published, Mills and Hamlet wrote a summary for Computing Reviews. That paper was assigned to the database editor (!), who first "had difficulty finding referees," and never completed handling the paper. The junior author did not at that time grasp how unprofessional this treatment is (or even that such treatment was a possibility!) and was too diffident to complain effectively. The paper remains in limbo.
83. Musa, J., Iannino, A., Okumoto, K.: Software Reliability. McGraw-Hill, New York (1990)
84. Musa, J.D.: Operational profiles in software-reliability engineering. IEEE Software 10, 14-32 (1993)
85. Myers, G.J.: The Art of Software Testing. Wiley-Interscience, New York, NY (1979)
86. Ntafos, S.: On required element testing. IEEE Trans. on Softw. Eng. 10, 795-803 (1984)
87. Ostrand, T.J., Balcer, M.J.: The category-partition method for specifying and generating functional tests. Comm. of the ACM 16, 676-686 (1988)
88. Parnas, D.: On a "Buzzword": Hierarchical structure. In: Proc. IFIP Congress, pp. 336-339. North-Holland Publishing Co. (1974)
89. Parnas, D.L.: On the criteria to be used in decomposing systems into modules. Comm. of the ACM (1972) Parnas is best known for his deep insights into software engineering and their expression in revealing concrete examples; there is no better illustration of these strengths than this famous paper. Its example is the design for a program for KWIC indexing, which now also serves the unexpected purpose of keeping this neglected variant of index-making alive. To the modern student, it seems that Parnas is just presenting object-oriented design in an unfamiliar way; this is chronologically reversed, since O-O was barely a glimmer in Alan Kay's and O-J Dahl's eyes when Parnas wrote. His ideas came from software engineering, not programming languages; he was mostly thinking of operating systems written in assembly language.
90. Petroski, H.: To Engineer is Human: The Role of Failure in Successful Design. St. Martin's Press, New York, NY (1985)
91. Sethi, R.: A case study in specifying the semantics of a programming language. In: Proc. POPL 1980, pp. 117-130 (1980)
92. Spivey, J.M.: The Z Notation: a reference manual, 2 edn. Prentice Hall (1992)
93. Szyperski, C.: Component Software, 2nd edn. Addison-Wesley (2002) Almost every citation to this much-cited monograph is pointing to its definition of software component, and justly so, for Szyperski was among the first to think deeply about this concept. Perhaps the close reasoning and careful abstraction that characterize the book were necessary to come to framing a suitable definition, but the general reader will find it hard going.
94. Turing, A.: On computable numbers, with an application to the entscheidungsproblem. London Mathematical Society pp. 230-265 (1936-37) Reprinted, including the correction (1937), in Davis's fine collection [20, ?A-]. Davis's annotation of the paper is right on: "...read this paper for its general sweep, ignoring the petty technical details." But of course the sweep would not be there if the author had not been convinced that the details (hardly 'petty' unless all programming is, which Davis may very well believe) supported it. A modern proof of Turing's result is often component-based [56], in that a number of simple Turing machines are devised to solve various bookkeeping problems, then combined into a 'system' that is a universal machine. It is probably safe to say that none of the many 'implementations' of the universal machine is likely to be correct-there are too many details, and none has been tested! But carrying through the exercise is absolutely convincing. One sees clearly that it can be done, even as one makes mistakes trying to do it. The modern reader has difficultly keeping in mind just how little was established about computability in 1936, requiring Turing to feel his way cautiously to each application.
95. Turing, A.: Checking a large routine. In: Report of a conference on high speed automatic calculating machines (EDSAC inaugral conference), pp. 67-69 (1949) This short paper is from Turing's second career as a programmer on the early machines. In it he anticipates some of what Floyd [25, ?A-] would publish almost 20 years later. Mathematicians (his colleagues in his first career) lamented Turing's loss to the profession: What good work he might have done if he hadn't been seduced by those digital computers! Reprinted with corrections and annotations in "An early program proof by Alan Turing," L. Morris and C. B. Jones, Ann. Hist. Computing 6 (2) pp 129-143 (1984), and in The early British computer conferences, MIT Press, 1989, pp 70-72.
96. Van Zandt, T.: Pstricks: Postscript macros for generic tex, user's guide. ctan.org/tex-archive/graphics/pstricks/base/doc/pst-user.pdf (1993)
97. Vincenti, W.G.: What Engineers Know and How They Know It. Johns Hopkins University Press (1993) This collection of chapter examples from aircraft engineering provides an invaluable insight into how engineers think about and solve problems. One particularly notable theme is the transition from pure guesswork in design to experimentation that determines the best combination of parameters. Before this is possible there must be enough theory to delineate the parameters to be monitored in experiments.
98. Wallnau, K.: http://www.sei.cmu.edu/pacc (links to CBSE proceedings)
99. Weiser, M.: Programmers use slices when debugging. Comm. of the ACM 25(7), 446-452 (1982)
100. White, L.J., Cohen, E.I.: A domain strategy for computer program testing. IEEE Trans. on Softw. Eng. 6(3), 247-257 (1980)
101. Woit, D.,Mason, D.: Software component independence. In: Proc. 3rd IEEE High-Assurance Systems Engineering Symposium (HASE'98). Washington, DC (1998)