Centroidal voronoi tessellations-a new approach to random testing

IEEE Transactions on Software Engineering

Volumn 39, Issue 2, 2013, Pages 163-183

  Shahbazi, Ali ^a   Tappenden, Andrew F ^b   Miller, James ^a  

^a UNIVERSITY OF ALBERTA   (Canada)

^b KING S UNIVERSITY COLLEGE   (Canada)

Author keywords

Adaptive random testing; centroidal Voronoi tessellation; P measure; random testing; software testing; test case generation; test strategies

Indexed keywords

ADAPTIVE RANDOM TESTING; CENTROIDAL VORONOI TESSELLATION; P-MEASURE; RANDOM TESTING; TEST CASE GENERATION; TEST STRATEGIES;

ALGORITHMS; OBJECT RECOGNITION; TESTING; VARIABLE SPEED TRANSMISSIONS;

SOFTWARE TESTING;

EID: 84873302084     PISSN: 00985589     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSE.2012.18     Document Type: Article

References (68)

1. V. Agrawal, "When to Use Random Testing," IEEE Trans. Computers, vol. 27, no. 11, pp. 1054-1055, Nov. 1978.
2. J.W. Duran and S.C. Ntafos, "An Evaluation of Random Testing," IEEE Trans. Software Eng., vol. 10, no. 4, pp. 438-444, July 1984.
3. P. Loo and W. Tsai, "Random Testing Revisited," Information and Software Technology, vol. 30, no. 7, pp. 402-417, 1988.
4. P. Schneck, "Comment on 'When to Use Random Testing'," IEEE Trans. Computers, vol. 28, no. 8, pp. 580-581, Aug. 1979.
5. D.R. Slutz, "Massive Stochastic Testing of SQl," Proc. 24th Int'l Conf. Very Large Data Bases, pp. 618-622, 1998.
6. H. Bati, L. Giakoumakis, S. Herbert, and A. Surna, "A Genetic Approach for Random Testing of Database Systems," Proc. 33th Int'l Conf. Very Large Data Bases, pp. 1243-1251, 2007.
7. T. Yoshikawa, K. Shimura, and T. Ozawa, "Random Program Generator for Java Jit Compiler Test System," Proc. Third Int'l Conf. Quality Software, pp. 20-23, 2003.
8. C. Pacheco, S.K. Lahiri, and T. Ball, "Finding Errors in .Net with Feedback-Directed Random Testing," Proc. Int'l Symp. Software Testing and Analysis, pp. 87-96, 2008.
9. P. Godefroid, A. Kiezun, and M.Y. Levin, "Grammar-Based Whitebox Fuzzing," Proc. ACM SIGPLAN Conf. Programming Language Design and Implementation, pp. 206-215, 2008.
10. J.E. Forrester and B.P. Miller, "An Empirical Study of the Robustness of Windows NT Applications Using Random Testing," Proc. Fourth Conf. USENIX Windows Systems Symp., vol. 4, pp. 6-6, 2000.
11. B.P. Miller, G. Cooksey, and F. Moore, "An Empirical Study of the Robustness of Macos Applications Using Random Testing," Proc. First Int'l Workshop Random Testing, pp. 46-54, 2006. (Pubitemid 46645425)
12. S. Lipner and M. Howard, "The Trustworthy Computing Security Development Lifecycle Document (SDL)," http://msdn.microsoft. com/en-us/library/ms995349.aspx, 2005.
13. P. Godefroid, "Random Testing for Security: Blackbox vs. Whitebox Fuzzing," Proc. Second Int'l Workshop Random Testing: Co-Located with IEEE/ACM 22nd Int'l Conf. Automated Software Eng., p. 1, 2007.
14. M. Marre and A. Bertolino, "Using Spanning Sets for Coverage Testing," IEEE Trans. Software Eng., vol. 29, no. 11, pp. 974-984, Nov. 2003.
15. A. Bertolino and M. Marre, "Automatic Generation of Path Covers Based on the Control Flow Analysis of Computer Programs," IEEE Trans. Software Eng., vol. 20, no. 12, pp. 885-899, Dec. 1994.
16. L. White and E. Cohen, "A Domain Strategy for Computer Program Testing," IEEE Trans. Software Eng., vol. 6, no. 3, pp. 247-257, May 1980. (Pubitemid 10485841)
17. P. Ammann and J. Knight, "Data Diversity: An Approach to Software Fault Tolerance," IEEE Trans. Computers, vol. 37, no. 4, pp. 418-425, Apr. 1988. (Pubitemid 18638799)
18. G.B. Finelli, "NASA Software Failure Characterization Experiments," Reliability Eng. and System Safety, vol. 32, nos. 1/2, pp. 155-169, 1991.
19. P. Bishop, "The Variation of Software Survival Time for Different Operational Input Profiles (or Why You Can Wait a Long Time for a Big Bug to Fail)," 23rd Int'l Symp. Fault-Tolerant Computing Digest of Papers, pp. 98-107, june 1993.
20. C. Schneckenburger and J. Mayer, "Towards the Determination of Typical Failure Patterns," Proc. Fourth Int'l Workshop Software Quality Assurance: in Conjunction with the Sixth ESEC/FSE Joint Meeting, pp. 90-93, 2007. (Pubitemid 351438754)
21. F.T. Chan, T.Y. Chen, I.K. Mak, and Y.T. Yu, "Proportional Sampling Strategy: Guidelines for Software Testing Practitioners," Information and Software Technology, vol. 38, no. 12, pp. 775-782, 1996. (Pubitemid 126371400)
22. C. Jones, "Software Quality in 2011: A Survey of the State of the Art," http://www.asq509.org/ht/a/GetDocumentAction/id/62711, 2011.
23. T. Chen, D.H. Huang, T. Tse, and Z. Yang, "An Innovative Approach to Tackling the Boundary Effect in Adaptive Random Testing," Proc. 40th Ann. Hawaii Int'l Conf. System Sciences, p. 262a, 2007.
24. F.-C. Kuo, "An Indepth Study of Mirror Adaptive Random Testing," Proc. Ninth Int'l Conf. Quality Software, pp. 51-58, 2009.
25. T. Chen, H. Leung, and I. Mak, "Adaptive Random Testing," Proc. Ninth Asian Computing Science Conf.: Advances in Computer Science, M. Maher, ed., pp. 3156-3157, 2005.
26. T. Chen and R. Merkel, "Efficient and Effective Random Testing Using the Voronoi Diagram," Proc. Australian Software Eng. Conf., pp. 300-308, 2006.
27. T. Chen, R. Merkel, P. Wong, and G. Eddy, "Adaptive Random Testing through Dynamic Partitioning," Proc. Fourth Int'l Conf. Quality Software, pp. 79-86, 2004. (Pubitemid 40276400)
28. T.Y. Chen, T.H. Tse, and Y.T. Yu, "Proportional Sampling Strategy: A Compendium and Some Insights," J. Systems and Software, vol. 58, no. 1, pp. 65-81, 2001. (Pubitemid 32746268)
29. T. Chen and D. Huang, "Adaptive Random Testing by Localization," Proc. 11th Asia-Pacific Software Eng. Conf., pp. 292-298, 2004. (Pubitemid 40700519)
30. T. Chen, D. Huang, and Z. Zhou, "Adaptive Random Testing Through Iterative Partitioning," Proc. 11th Ada-Europe Int'l Conf. Reliable Software Technologies, L. Pinho and M. González Harbour, eds., pp. 155-166, 2006. (Pubitemid 44071093)
31. J. Mayer, "Lattice-Based Adaptive Random Testing," Proc. IEEE/ACM 20th Int'l Conf. Automated Software Eng., pp. 333-336, 2005.
32. J. Mayer, "Adaptive Random Testing by Bisection and Localization," Proc. Fifth Int'l Conf. Formal Approaches to Software Testing, W. Grieskamp and C. Weise, eds., pp. 72-86, 2006. (Pubitemid 44165378)
33. A. Tappenden and J. Miller, "A Novel Evolutionary Approach for Adaptive Random Testing," IEEE Trans. Reliability, vol. 58, no. 4, pp. 619-633, Dec. 2009.
34. I. Ciupa, A. Leitner, M. Oriol, and B. Meyer, "Artoo: Adaptive Random Testing for Object-Oriented Software," Proc. ACM/IEEE 30th Int'l Conf. Software Eng., pp. 71-80, May 2008.
35. T.Y. Chen and R. Merkel, "Quasi-Random Testing," IEEE Trans. Reliability, vol. 56, no. 3, pp. 562-568, Sept. 2007. (Pubitemid 47399764)
36. H. Chi and E.L. Jones, "Computational Investigations of Quasirandom Sequences in Generating Test Cases for Specification-Based Tests," Proc. 38th Conf. Winter Simulation, pp. 975-980, 2006.
37. K.P. Chan, T.Y. Chen, F.-C. Kuo, and D. Towey, "A Revisit of Adaptive Random Testing by Restriction," Proc. 28th Ann. Int'l Computer Software and Applications Conf., vol. 1, pp. 78-85, 2004. (Pubitemid 40673833)
38. P. LEcuyer and C. Lemieux, "Recent Advances in Randomized Quasi-Monte Carlo Methods," Modeling Uncertainty, pp. 419-474, Springer, 2005.
39. M. Li, X. Chen, X. Li, B. Ma, and P. Vitanyi, "The Similarity Metric," IEEE Trans. Information Theory, vol. 50, no. 12, pp. 3250-3264, Dec. 2004.
40. M. Li and P. Vitanyi, An Introduction to Kolmogorov Complexity and Its Applications. Springer-Verlag, 2008.
41. K. Chan, T. Chen, and D. Towey, "Restricted Random Testing," Proc. Seventh Int'l Conf. Software Quality, J. Kontio and R. Conradi, eds., pp. 321-330, 2006.
42. T.Y. Chen, F.-C. Kuo, and H. Liu, "Adaptive Random Testing Based on Distribution Metrics," J. Systems and Software, vol. 82, no. 9, pp. 1419-1433, 2009.
43. J. Mayer and C. Schneckenburger, "An Empirical Analysis and Comparison of Random Testing Techniques," Proc. ACM/IEEE Int'l Symp. Empirical Software Eng., pp. 105-114, 2006. (Pubitemid 46634446)
44. K.P. Chan, T. Chen, and D. Towey, "Forgetting Test Cases," Proc. 30th Ann. Int'l Computer Software and Applications Conf., vol. 1, pp. 485-494, Sept. 2006.
45. I.M. Sobol, "Uniformly Distributed Sequences with Additional Uniformity Properties," J. Computational Math. and Math. Physics, vol. 16, pp. 236-242, 1976.
46. J.H. Halton, "Algorithm 247: Radical-Inverse Quasi-Random Point Sequence," Comm. ACM, vol. 7, pp. 701-702, Dec. 1964.
47. H. Niederreiter, "Low-Discrepancy and Low-Dispersion Sequences," J. Number Theory, vol. 30, no. 1, pp. 51-70, 1988.
48. H. Faure, "Discrépance de Suites Associées à un Systeme de numération (en Dimension Un)," Bull. Soc. Math. France, vol. 109, no. 2, pp. 143-182, 1981.
49. P. Peart, "The Dispersion of the Hammersley Sequence in the Unit Square," Monatshefte für Mathematik, vol. 94, pp. 249-261, 1982.
50. C. Schlier, "On Scrambled Halton Sequences," Applied Numerical Math., vol. 58, no. 10, pp. 1467-1478, 2008.
51. B.L. Fox, "Algorithm 647: Implementation and Relative Efficiency of Quasirandom Sequence Generators," ACM Trans. Math. Software, vol. 12, pp. 362-376, Dec. 1986.
52. H. Everett, D. Lazard, S. Lazard, and M. Safey El Din, "The Voronoi Diagram of Three Lines," Discrete and Computational Geometry, vol. 42, pp. 94-130, 2009.
53. Q. Du, V. Faber, and M. Gunzburger, "Centroidal Voronoi Tessellations: Applications and Algorithms," SIAM Rev., vol. 41, no. 4, pp. 637-676, 1999. (Pubitemid 30536738)
54. L. Ju, Q. Du, and M. Gunzburger, "Probabilistic Methods for Centroidal Voronoi Tessellations and Their Parallel Implementations," Parallel Computing, vol. 28, no. 10, pp. 1477-1500, 2002.
55. A. Okabe, B. Boots, K. Sugihara, and S.N. Chiu, Spatial Tessellations: Concepts and Applications of Voronoi Diagrams, second ed. John Wiley & Sons, Inc., 2008.
56. N. Beckmann, H.-P. Kriegel, R. Schneider, and B. Seeger, "The r-Tree: An Efficient and Robust Access Method for Points and Rectangles," SIGMOD Record, vol. 19, pp. 322-331, May 1990.
57. A. Arcuri and L. Briand, "Adaptive Random Testing: An Illusion of Effectiveness?" Proc. Int'l Symp. Software Testing and Analysis, pp. 265-275, 2011.
58. T.Y. Chen, F.-C. Kuo, and R. Merkel, "On the Statistical Properties of Testing Effectiveness Measures," J. Systems and Software, vol. 79, no. 5, pp. 591-601, 2006.
59. C. Pacheco, S. Lahiri, M. Ernst, and T. Ball, "Feedback-Directed Random Test Generation," Proc. 29th Int'l Conf. Software Eng., pp. 75-84, May 2007. (Pubitemid 47430643)
60. P. Godefroid, N. Klarlund, and K. Sen, "DART: Directed Automated Random Testing," Proc. ACM SIGPLAN Conf. Programming Language Design and Implementation, pp. 213-223, 2005.
61. Y.-S. Ma, J. Offutt, and Y.R. Kwon, "Mujava: An Automated Class Mutation System," Software Testing, Verification, and Reliability, vol. 15, no. 2, pp. 97-133, 2005. (Pubitemid 40858060)
62. J. Cohen, "A Power Primer," Psychological Bull., vol. 112, no. 1, pp. 155-159, 1992.
63. J. Cohen, Statistical Power Analysis for the Behavioral Sciences. Lawrence Erlbaum, 1988.
64. L.A. Becker, "Effect Size (ES)," no. 1993, http://web.uccs.edu/ lbecker/Psy590/es.htm, 2000.
65. M.J. Meyer, "Martingale Java Stochastic Library," http://martingale.berlios.de/Martingale.html, 2012.
66. K. Morse Jr, "Compression Tools Compared," Linux J., vol. 2005, no. 137, pp. 62-66, 2005.
67. T. Chen, F. Kuo, and Z. Zhou, "On Favourable Conditions for Adaptive Random Testing," Int'l J. Software Eng. and Knowledge Eng., vol. 17, no. 6, pp. 805-825, 2007. (Pubitemid 351206402)
68. D. Salomon, Data Compression: The Complete Reference, vol. 10. Springer-Verlag, 2007.