Finding failures from passed test cases: Improving the pattern classification approach to the testing of mesh simplification programs

Software Testing Verification and Reliability

Volumn 20, Issue 2, 2010, Pages 89-120

  Chan, W K ^a   Ho, Jeffrey C F ^b   Tse, T H ^c  

^a CITY UNIVERSITY OF HONG KONG   (Hong Kong)

^b Wwwins Consulting Hong Kong Limited   (Hong Kong)

^c UNIVERSITY OF HONG KONG   (Hong Kong)

Author keywords

Classification; Mesh simplification; Metamorphic testing; Non testable software; Test oracle problem; Testing methodology

Indexed keywords

DESIGN TRADES; FAILURE DETECTION; INTEGRATED APPROACH; MESH SIMPLIFICATIONS; METAMORPHIC TESTING; PATTERN CLASSIFICATION; POLYGONAL MODELS; REFERENCE MODELS; SOFTWARE TEST; STATISTICAL PATTERN CLASSIFICATION; TEST CASE; TEST ORACLES; TESTING METHODOLOGY;

CLASSIFIERS; COMPUTER SOFTWARE; MODELS; TESTING;

THREE DIMENSIONAL;

EID: 77952749425     PISSN: 09600833     EISSN: 10991689     Source Type: Journal    
DOI: 10.1002/stvr.408     Document Type: Article

References (62)

1. Moore EB, Poliakov AV, Lincoln P, Brinkley JF. MindSeer: A portable and extensible tool for visualization of structural and functional neuroimaging data. BMC Bioinformatics 2007; 8:389.
2. Ahmed MN, Yamany SM, Mohamed N, Farag AA, Moriarty T. A modified fuzzy c-means algorithm for bias field estimation and segmentation of MRI data. IEEE Transactions on Medical Imaging 2002; 21(3):193-199.
3. Web3D Consortium. X3D International Specification Standards. Available at: http://www.web3d.org/x3d/(Last accessed: 27 June 2008).
4. Stanford University. The Stanford 3D Scanning Repository. Available at: http://graphics.stanford.edu/data/3Dscanrep/(Last accessed: 27 June 2008).
5. ISO/IEC. The MPEG Standards. Moving Picture Experts Group. Available at: http://www.chiariglione.org/mpeg/standards.htm (Last accessed: 15 June 2008).
6. Lindstrom P, Turk G. Imagine-driven simplification. ACM Transactions on Graphics 2000; 19(3):204-241.
7. Cook RL, Halstead J, Planck M, Ryu D. Stochastic simplification of aggregate detail. ACM Transactions on Graphics 2007; 26(3), Article no. 79.
8. Yoon SE, Lauterbach C, Monocha D. Rlods: Fast LOD-based ray tracing of massive models. The Visual Computer 2006; 22(9):772-784.
9. DeCoro C, Tatarchuk N. Real-time mesh simplification using the GPU. Proceedings of the 2007 Symposium on Interactive 3D Graphics and Games. ACM Press: New York, NY, 2007; 161-166.
10. Cignoni P, Rocchini C, Impoco G. A comparison of mesh simplification algorithms. Computers and Graphics 1998; 22(1):37-54.
11. Fahn C, Chen H, Shiau Y. Polygonal mesh simplification with face color and boundary edge preservation using quadric error metric. Proceedings of the Fourth IEEE International Symposium on Multimedia Software Engineering (MSE 2002). IEEE Computer Society Press: Los Alamitos, CA, 2002; 174-181.
12. Luebke DP, Reddy M, Cohen JD, Varshney A, Watson B, Huebner R. Level of Detail for 3D Graphics. Morgan Kaufmann: San Francisco, CA, 2003.
13. Zhu Y. Uniform remeshing with an adaptive domain: A new scheme for view-dependent level-of-detail rendering of meshes. IEEE Transactions on Visualization and Computer Graphics 2005; 11(3):306-316.
14. Luebke DP. A developer's survey of polygonal simplification algorithms. IEEE Computer Graphics and Applications 2001; 21(3):24-35.
15. Weyuker EJ. On testing non-testable programs. The Computer Journal 1982; 25(4):465-470.
16. Chan WK, Cheung SC, Ho JCF, Tse TH. Reference models and automatic oracles for the testing of mesh simplification software for graphics rendering. Proceedings of the 30th Annual International Computer Software and Applications Conference (COMPSAC 2006), vol. 1. IEEE Computer Society Press: Los Alamitos, CA, 2006; 429-438.
17. Chan WK, Cheung SC, Ho JCF, Tse TH. PAT: A pattern classification approach to automatic reference oracles for the testing of mesh simplification programs. Journal of Systems and Software 2008; DOI: 10.1016/j.jss.2008.07.019.
18. Duda RO, Hart PE, Stork DG. Pattern Classification. Wiley: New York, NY, 2000.
19. Chen TY, Cheung SC, Yiu SM. Metamorphic testing: A new approach for generating next test cases. Technical Report HKUST-CS98-01, Department of Computer Science, Hong Kong University of Science and Technology, Hong Kong, 1998.
20. Berstel J, Reghizzi SC, Roussel G, San Pietro P. A scalable formal method for design and automatic checking of user interfaces. ACM Transactions on Software Engineering and Methodology 2005; 14(2):124-167.
21. d'Ausbourg B, Seguin C, Durrieu G, Roch P. Helping the automated validation process of user interfaces systems. Proceedings of the 20th International Conference on Software Engineering (ICSE 1998). IEEE Computer Society Press: Los Alamitos, CA, 1998; 219-228.
22. Memon AM, Pollack ME, Soffa ML. Automated test oracles for GUIs. Proceedings of the Eighth ACM SIGSOFT International Symposium on Foundations of Software Engineering (SIGSOFT 2000/FSE-8). ACM Press: New York, NY, 2000; 30-39.
23. Sun Y, Jones EL. Specification-driven automated testing of GUI-based Java programs. Proceedings of the 42nd Annual Southeast Regional Conference (ACM-SE 42). ACM Press: New York, NY, 2004; 140-145.
24. Memon A, Banerjee I, Nagarajan A. What test oracle should I use for effective GUI testing? Proceedings of the 18th IEEE International Conference on Automated Software Engineering (ASE 2003). IEEE Computer Society Press: Los Alamitos, CA, 2003; 164-173.
25. Segal M, Akeley K. The OpenGL Graphics System: A Specification, version 2.0. Silicon Graphics, Mountain View, CA, 2004.
26. Bierbaum A, Hartling P, Cruz-Neira C. Automated testing of virtual reality application interfaces. Proceedings of the Eurographics Workshop on Virtual Environments (EGVE 2003). ACM Press: New York, NY, 2003; 107-114.
27. Cheung SC, Chanson ST, Xu Z. Applying generic timing tests for distributed multimedia software systems. IEEE Transactions on Reliability 2004; 53(3):329-341.
28. Mayer J. On testing image processing applications with statistical methods. Software Engineering 2005 (SE 2005) (Lecture Notes in Informatics). Gesellschaft für Informatik: Bonn, 2005; 69-78.
29. Mayer J, Guderlei R. An empirical study on the selection of good metamorphic relations. Proceedings of the 30th Annual International Computer Software and Applications Conference (COMPSAC 2006), vol. 1. IEEE Computer Society: Los Alamitos, CA, 2006; 475-484.
30. Hu P, Zhang Z, Chan WK, Tse TH. An empirical comparison between direct and indirect test result checking approaches. Proceedings of the Third International Workshop on Software Quality Assurance (SOQUA 2006) (in conjunction with the 14th ACM SIGSOFT International Symposium on Foundations of Software Engineering (SIGSOFT 2006/FSE-14)). ACM Press: New York, NY, 2006; 6-13.
31. Ostrand T, Anodide A, Foster H, Goradia T. A visual test development environment for GUI systems. Proceedings of the 1998 ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 1998). ACM SIGSOFT Software Engineering Notes 1998; 23(2):82-92.
32. Dillon LK, Ramakrishna YS. Generating oracles from your favorite temporal logic specifications. Proceedings of the Fourth ACM SIGSOFT Symposium on Foundations of Software Engineering (SIGSOFT '96/FSE-4). ACM SIGSOFT Software Engineering Notes 1996; 21(6):106-117.
33. Baresi L, Denaro G, Mainetti L, Paolini P. Assertions to better specify the Amazon bug. Proceedings of the 14th International Conference on Software Engineering and Knowledge Engineering (SEKE 2002). ACM Press: New York, NY, 2002; 585-592.
34. Meyer B. Eiffel: The Language. Prentice-Hall: New York, NY, 1992.
35. Peters DK, Parnas DL. Using test oracles generated from program documentation. IEEE Transactions on Software Engineering 1998; 24(3):161-173.
36. Binder RV. Testing Object-oriented Systems: Models, Patterns, and Tools. Addison-Wesley: Reading, MA, 2000.
37. Last M, Friedman M, Kandel A. The data mining approach to automated software testing. Proceedings of the Ninth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 2003). ACM Press: New York, NY, 2003; 388-396.
38. Vanmali M, Last M, Kandel A. Using a neural network in the software testing process. International Journal of Intelligent Systems 2002; 17(1):45-62.
39. Podgurski A, Leon D, Francis P, Masri W, Minch M, Sun J, Wang B. Automated support for classifying software failure reports. Proceedings of the 25th International Conference on Software Engineering (ICSE 2003). IEEE Computer Society Press: Los Alamitos, CA, 2003; 465-475.
40. Francis P, Leon D, Minch M, Podgurski A. Tree-based methods for classifying software failures. Proceedings of the 15th International Symposium on Software Reliability Engineering (ISSRE 2004). IEEE Computer Society Press: Los Alamitos, CA, 2004; 451-462.
41. Bowring JF, Rehg JM, Harrold MJ. Active learning for automatic classification of software behavior. Proceedings of the 2004 ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2004). ACM SIGSOFT Software Engineering Notes 2004; 29(4):195-205.
42. Chan WK, Cheng MY, Cheung SC, Tse TH. Automatic goal-oriented classification of failure behaviors for testing XML-based multimedia software applications: An experimental case study. Journal of Systems and Software 2009; 79(5):602-612.
43. Wikipedia. Binary classification. Available at: http://en.wikipedia.org/ wiki/Binaryclassification (Last accessed: 30 June 2008).
44. Andrews JH, Briand LC, Labiche Y. Is mutation an appropriate tool for testing experiments? Proceedings of the 27th International Conference on Software Engineering (ICSE 2005). ACM Press: New York, NY, 2005; 402-411.
45. Offutt AJ, Lee A, Rothermel G, Untch RH, Zapf C. An experimental determination of sufficient mutant operators. ACM Transactions on Software Engineering and Methodology 1996; 5(2):99-118.
46. DeMillo RA, Lipton RJ, Sayward FG. Hints on test data selection: Help for the practicing programmer. IEEE Computer 1978; 11(4):34-41.
47. Rusinkiewicz S, Levoy M. Efficient variants of the ICP algorithm. Proceedings of the Third International Conference on 3D Digital Imaging and Modeling (3DIM 2001). IEEE Computer Society Press: Los Alamitos, CA, 2001; 145-152.
48. Hierons RM. Avoiding coincidental correctness in boundary value analysis. ACM Transactions on Software Engineering and Methodology 2006; 15(3):227-241.
49. Chan WK, Chen TY, Lu H, Tse TH, Yau SS. Integration testing of context-sensitive middleware-based applications: A metamorphic approach. International Journal of Software Engineering and Knowledge Engineering 2006; 16(5):677-703.
50. Melax S. A simple, fast, and effective polygon reduction algorithm. Game Developer Magazine, November 1998; 44-49.
51. Garland M, Heckbert P. Surface simplification using quadric error metrics. Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH 1997). ACM Press: New York, NY, 1997; 209-216.
52. Gonzalez RC, Woods RE. Digital Image Processing. Prentice-Hall: Englewood Cliffs, NJ, 2002.
53. Ma Y-S, Offutt AJ, Kwon Y-R. MuJava: An automated class mutation system. Software Testing, Verification and Reliability 2005; 15(2):97-133.
54. Hosmer DW, Lemeshow S. Applied Logistic Regression. Wiley: New York, NY, 2004.
55. Kohavi R, Provost F. Glossary of terms. Machine Learning 1998; 30(2/3):271-274.
56. Wikipedia. Specificity tests. Available at: http://en.wikipedia.org/wiki/ Specificity%28tests%29 (Last accessed: 30 June 2008).
57. Grinde RB, Cavalier TM. A new algorithm for the two-polygon containment problem. Computers and Operations Research 1997; 24(3):231-251.
58. Shulmeyer GG, McCabe TJ. The Pareto principle applied to software quality assurance. Handbook of Software Quality Assurance (3rd edn). Prentice-Hall: Upper Saddle River, NJ, 1998; 291-328.
59. Kapoor K. Formal analysis of coupling hypothesis for logical faults. Innovations in Systems and Software Engineering 2006; 2(2):80-87.
60. Lau MF, Yu YT. An extended fault class hierarchy for specification-based testing. ACM Transactions on Software Engineering and Methodology 2005; 14(3):247-276.
61. Kapoor K, Bowen JP. Test conditions for fault classes in Boolean specifications. ACM Transactions on Software Engineering and Methodology 2007; 16(3):1-12.
62. Chan WK, Ho JCF, Tse TH. Piping classification to metamorphic testing: An empirical study towards better effectiveness for the identification of failures in mesh simplification programs. Proceedings of the 31st Annual International Computer Software and Applications Conference (COMPSAC 2007), vol. 1. IEEE Computer Society Press: Los Alamitos, CA, 2007; 397-404.