Factor-covering designs for testing software

Technometrics

Volumn 40, Issue 3, 1998, Pages 234-243

  Dalal, Siddhartha R ^a   Mallows, Colin L ^b  

^a TELCORDIA TECHNOLOGIES   (United States)

^b AT AND T LABS RESEARCH   (United States)

Author keywords

Equivalence partitioning; Orthogonal arrays; Random balance; Supersaturated designs

Indexed keywords

COMPUTER SOFTWARE SELECTION AND EVALUATION; DESIGN; TESTING;

EQUIVALENCE PARTITIONING; ORTHOGONAL ARRAYS; RANDOM BALANCE; SUPERSATURATED DESIGNS;

COMPUTER SOFTWARE;

EID: 0032139384     PISSN: 00401706     EISSN: 15372723     Source Type: Journal    
DOI: 10.1080/00401706.1998.10485524     Document Type: Article

References (48)

1. Agrawal, V., and Dey, A. (1982), “A Note on Orthogonal Main Effect Plans for Asymmetrical Factorials,”Sankhyä, Ser. B, 44, 278-282.
2. Assmus, E. F., and Key, J. D. (1992), “Hadamard Matrices and Their Designs: A Coding Theoretic Approach,”Transactions of the American Mathematical Society, 330, 269-293.
3. Beizer, B. (1995), Black-Box Testing, New York: Wiley.
4. Booth, E. H., and Cox, D. R. (1962), “Some Systematic Supersaturated Experimental Designs,”Technometrics, 4, 489-495.
5. Box, G. E. P., and Tyssedal, J. (1996), “Projective Properties of Certain Orthogonal Arrays,”Biometrika,83, 950-955.
6. Brownlie, R., Prowse, J., and Phadke, M. S. (1992), “Robust Testing of AT& T PMX/StarMAIL Using OATS,”AT& T Technical Journal,71, 41-47.
7. Burr, K., and Young, W. (1997), “Test Acceleration (Automatic Efficient Testcase Generation),” unpublished paper presented at Nortel Design Forum, Ottawa, December 3-5.
8. Burroughs, K., Jain, A., and Erickson, R. L. (1994), “Improved Quality of Protocol Testing Through Techniques of Experimental Design,” Super-comm/ICC '94 (IEEE International Conference on Communications), New York: IEEE, pp. 745-752.
9. Cohen, D. M., Dalai, S. R., Fredman, M., and Patton, G. C. (1997), “The AETG System: An Approach to Testing Based on Combinatorial Design,”IEEE Transactions of Software Engineering,23, 437-443.
10. Cohen, D. M., Dalai, S. R., Kajla, A., and Patton, G. C. (1994), “The Automatic Efficient Test Generator (AETG) Systems,” in Proceedings of the 5th International Symposium on Software Reliability Engineering, Los Alamitos, CA: IEEE Computer Society Press, pp. 305-309.
11. Cohen, D. M., Dalai, S. R., Parelius, J., and Patton, G. C. (1996), “The Combinatorial Approach to Automatic Test Generation,”IEEE Software, 13, 83-89.
12. Cohen, D. M., and Fredman, M. (1996), “New Techniques for Designing Qualitatively Independent Systems,” Technical Report DCS-96-114, Rutgers University, Computer Science Dept.
13. Dalai, S. R., Horgan, J. R., and Kettenring, J. R. (1994), Reliable Software and Communication II, Controlling the Software Development Process, IEEE Journal of Selected Areas in Communications,12, 33-39.
14. Dalai, S. R., and Mallows, C. L. (1988), “When Should One Stop Testing Software?”Journal of the American Statistical Association,83, 872-879.
15. Dalai, S. R., and McIntosh, A. M. (1994), “When to Stop Testing for Large Software Systems With Changing Code,”IEEE Transactions on Software Engineering,20, 318-323.
16. Dalai, S. R., and Patton G. C. (1993), “Automatic Efficient Test Generator (AETG): A Test Generation System for Screen Testing, Protocol Verification and Feature Interactions Testing,” technical memo, Bellcore, Morristown, NJ.
17. Dey, A., and Ramakrishna, G. V. S. (1977), “A Note on Orthogonal Main Effect Plans,”Technometrics,19, 511-512.
18. Dunietz, I. S., Ehrlich, W. K., and Mallows, C. L. (1997), “Experience With Covering Designs for Testing Software,” in Proceedings of the Statistical Computing Section, American Statistical Association, pp. 37-40.
19. Dunietz, I. S., Ehrlich, W. K., Szablak, B. D., Mallows, C. L., and Ian-nino, A. (1997), “Applying Systematic t-factor Covering Experimental Designs to Software Testing,” unpublished manuscript.
20. The Economist (1997), “The Millennium Bug,” October 4, 25-29.
21. Gargano, L., Korner, J., and Vaccaro, U. (1993), “Sperner Capacities,”Graphs and Combinatorics,9, 31-46.
22. Godbole, A. P., Skipper, D. E., and Sunley, R. A. (1995), “t-Covering Arrays: Upper Bounds and Approximations,” Preprint, Michigan Technological University, Mathematics Dept.
23. Humphry, W. S. (1989), Managing the Software Process, Reading, MA: Addison-Wesley.
24. Jorgensen, P. (1995), Software Testing, A Craftsman’s Approach, New York: CRC Press.
25. Katona, G. O. H. (1973), “Two Applications (for Search Theory and Truth Functions) of Sperner Type Theorems,”Periodica Mathematica Hun-garica,1-2, 19-26.
26. Kleitman, D. J., and Spencer, J. (1973), “Families of fc-Independent Sets,”Discrete Mathematics,6, 255-262.
27. Korner, J., and Lucertini, M. (1994), “Compressing Inconsistent Data,”IEEE Transactions on Information Theory,40, 706-715.
28. Lin, D. K. J. (1993), “A New Class of Supersaturated Designs,”Technometrics,35, 28-31.
29. Lin, D. K. J. (1995), “Generating Systematic Supersaturated Designs,”Technometrics,37, 213-225.
30. Mallows, C. L. (1997), “Covering Designs in Random Environments,” in The Practice of Data Analysis: Essays in Honor of J. W. Tukey, eds., D. R. Brillinger, L. T. Fernholz, and S. Morganthaler, Princeton, NJ: Princeton University Press, pp. 235-246.
31. Mandl, R. (1985), “Orthogonal Latin Squares: An Application of Experimental Design to Compiler Testing,”Communications of the ACM,28, 1054-1058.
32. Myers, G. J. (1979), The Art of Software Testing, New York: Wiley.
33. Mills, W., and Mullin, R. (1992), “A Survey on Coverings and Packings,” in Contemporary Design Theory: A Collection of Surveys, eds. J. Dinitz and D. Stinson, New York: Wiley, pp. 371-299.
34. Nair, V. N., James, D. A., Ehrlich, W. K., and Zevallos, J. (in press), “A Statistical Assessment of Some Software Testing Strategies and Application of Experimental Design Techniques,” Statistica Sinica, 8.
35. National Research Council (1996), “Statistical Software Engineering,” report by a panel on Statistical Methods in Software Engineering, October 11-12.
36. Plwowarski, P., Obha, M., and Caruse, J. (1993), “Coverage Measurement Experience During Function Test,” in Proceedings of the 15th International Conference on Software Engineering, Los Alamitos, CA: IEEE Computer Society Press, pp. 287-303.
37. Renyi, A. (1971), Foundations of Probability, New York: Wiley.
38. Roux, G. (1987), “fc-Propriétés dans des Tableaux de n Colonnes; Cas Particulier de la fc-Surjectivité et Dica/de la fc-Permutivite,” unpublished Ph.D. dissertation, University of Paris.
39. Satterthwaite, F. (1959), “Random Balance Experimentation” (with discussion), Technometrics, 1, 111-137.
40. Sherwood, G. B. (1994), “Effective Testing of Factor Combinations,” in Proceedings of the Third International Conference on Software Testing, Analysis and Review, Jacksonville, FL: Software Quality Engineering.
41. Singpurwalla, N. D. (1991), “Determining an Optimal Time Interval for Testing and Debugging Software,”IEEE Transactions on Software Engineering,17, 313-319.
42. Sloane, N. J. A. (1993), “Covering Arrays and Intersecting Codes,”Journal of Combinatorial Designs,1, 51-63.
43. Taguchi, G. (1986), Introduction to Quality Engineering, White Plains, NY: Kraus.
44. Tatsumi, K., Watanabe, S., Takeuchi, Y., and Shimokawa, H. (1987), “Conceptual Support for Test Case Design,” in Proceedings of the 11th Computer Software and Applications Conference (Compsac 87) Piscataway, NJ: IEEE, pp. 285-290.
45. Wang, J. C., and Wu, C. F. J. (1991), “An Approach to the Construction of Asymmetrical Orthogonal Arrays,”Journal of the American Statistical Association,8, 450-456.
46. Williams, A. L., and Probert, R. L. (1996), “A Practical Strategy for Testing Pairwise Coverage at Network Interfaces,” in Proceedings of the 7th International Symposium on Software Reliability Engineering, Los Alamitos, CA: IEEE Computer Society Press.
47. Wu, C. F. J. (1993), “Construction of Supersaturated Designs Through Partially Aliased Interactions,”Biometrica,80, 661-669.
48. Zeitler, D. (1991), “Statistical Methodology for Software Systems Testing,” in Proceedings of the 23rd Interface Symposium, Fairfax, VA: The Interface Foundation of North America, pp. 110-113.