Oracles are hardly attain'd, and hardly understood: Confessions of software testing researchers

Proceedings of the International Symposium on the Physical and Failure Analysis of Integrated Circuits, IPFA

Volumn , Issue , 2013, Pages 245-252

  Chan, W K ^a   Tse, T H ^b  

^a CITY UNIVERSITY OF HONG KONG   (Hong Kong)

^b UNIVERSITY OF HONG KONG   (Hong Kong)

Author keywords

metamorphic testing; object equivalence and nonequivalence; pattern classifier; test harness; Test oracle

Indexed keywords

METAMORPHIC TESTING; OBJECT EQUIVALENCE AND NONEQUIVALENCE; PATTERN CLASSIFIER; TEST HARNESS; TEST ORACLES;

TESTING;

SOFTWARE TESTING;

EID: 84885660194     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/QSIC.2013.16     Document Type: Conference Paper

References (33)

1. G. Banavar and A. Bernstein, "Software infrastructure and design challenges for ubiquitous computing applications," Communications of the ACM, vol. 45, no. 12, pp. 92-96, 2002.
2. P. Bellavista, A. Corradi, R. Montanari, and C. Stefanelli, "Contextaware middleware for resource management in the wireless Internet," IEEE Transactions on Software Engineering, vol. 29, no. 12, pp. 1086-1099, 2003.
3. A.T.S. Chan and S.-N. Chuang, "MobiPADS: a reflective middleware for context-aware mobile computing," IEEE Transactions on Software Engineering, vol. 29, no. 12, pp. 1072-1085, 2003.
4. W.K. Chan, T.Y. Chen, H. Lu, T.H. Tse, and S.S. Yau, "Integration testing of context-sensitive middleware-based applications: a metamorphic approach," International Journal of Software Engineering and Knowledge Engineering, vol. 16, no. 5, pp. 677-703, 2006.
5. W.K. Chan, S.C. Cheung, J.C.F. Ho, and T.H. Tse, "PAT: a pattern classification approach to automatic reference oracles for the testing of mesh simplification programs," Journal of Systems and Software, vol. 82, no. 3, pp. 422-434, 2009.
6. W.K. Chan, S.C. Cheung, and K.R.P.H. Leung, "A metamorphic testing approach for online testing of service-oriented software applications," International Journal of Web Services Research, vol. 4, no. 2, pp. 60-80, 2007.
7. W.K. Chan, J.C.F. Ho, and T.H. Tse, "Finding failures from passed test cases: improving the pattern classification approach to the testing of mesh simplification programs," Software Testing, Verification and Reliability, vol. 20, no. 2, pp. 89-120, 2010.
8. S.C. Chapra and R.P. Canale, Numerical Methods for Engineers, McGraw-Hill, 2006.
9. H.Y. Chen, T.H. Tse, F.T. Chan, and T.Y. Chen, "In black and white: an integrated approach to class-level testing of object-oriented programs," ACM Transactions on Software Engineering and Methodology, vol. 7, no. 3, pp. 250-295, 1998.
10. H.Y. Chen, T.H. Tse, and T.Y. Chen, "TACCLE: a methodology for object-oriented software testing at the class and cluster levels," ACM Transactions on Software Engineering and Methodology, vol. 10, no. 1, pp. 56-109, 2001.
11. T.Y. Chen, S.C. Cheung, and S.M. Yiu, "Metamorphic testing: a new approach for generating next test cases," Department of Computer Science, Hong Kong University of Science and Technology, Hong Kong, Technical Report HKUST-CS98-01, 1998.
12. T.Y. Chen, J. Feng, and T.H. Tse, "Metamorphic testing of programs on partial differential equations: a case study," Proceedings of the 26th Annual International Computer Software and Applications Conference (COMPSAC 02), IEEE Computer Society, 2002, pp. 327-333.
13. T.Y. Chen, T.H. Tse, and Z.Q. Zhou, "Semi-proving: an integrated method for program proving, testing, and debugging," IEEE Transactions on Software Engineering, vol. 37, no. 1, pp. 109-125, 2011.
14. W.J. Cody, Jr. and W. Waite, Software Manual for the Elementary Functions, Prentice Hall, 1980.
15. C.F. Gerald and P.O. Wheatley, Applied Numerical Analysis, Addison-Wesley, 1999.
16. B. Jiang, K. Zhai, W.K. Chan, T.H. Tse, and Z. Zhang, "On the adoption of MC/DC and control-flow adequacy for a tight integration of program testing and statistical fault localization," Information and Software Technology, vol. 55, no. 5, pp. 897-917, 2013.
17. Z. Lai, S.C. Cheung, and W.K. Chan, "Inter-context control-flow and data-flow test adequacy criteria for nesC applications," Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of Software Engineering (SIGSOFT 08/FSE-16), ACM, 2008, pp. 94-104.
18. H. Lu, W.K. Chan, and T.H. Tse, "Testing context-aware middlewarecentric programs: a data flow approach and an RFID-based experimentation," Proceedings of the 14th ACM SIGSOFT International Symposium on Foundations of Software Engineering (SIGSOFT 06/FSE-14), ACM, 2006, pp. 242-252.
19. H. Lu, W.K. Chan, and T.H. Tse, "Testing pervasive software in the presence of context inconsistency resolution services," Proceedings of the 30th International Conference on Software Engineering (ICSE 08), ACM, 2008, pp. 61-70.
20. C. Mascolo, L. Capra, S. Zachariadis, and W. Emmerich, "Xmiddle: a data-sharing middleware for mobile computing," Wireless Personal Communications, vol. 21, no. 1, pp. 77-103, 2002.
21. L. Mei, W.K. Chan, and T.H. Tse, "Data flow testing of serviceoriented workflow applications," Proceedings of the 30th International Conference on Software Engineering (ICSE 08), ACM, 2008, pp. 371-380.
22. L. Mei, W.K. Chan, and T.H. Tse, "Data flow testing of service choreography," Proceedings of the 7th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT International Symposium on Foundations of Software Engineering (ESEC 09/FSE-17), ACM, 2009, pp. 151-160.
23. H.J. Nock, G. Iyengar, and C. Neti, "Multimodal interfaces that flex, adapt, and persist: multimodal processing by finding common cause," Communications of the ACM, vol. 47, no. 1, pp. 51-56, 2004.
24. D.S. Rosenblum, C. Mascolo, M.Z. Kwiatkowska, D. Ghica, M. Ryan, N. Dulay, and E. Lupu, "UbiVal: fundamental approaches to validation of ubiquitous computing applications and infrastructures," University of Birmingham, University College London and Imperial College London, UK, Project Proposal, EPSRC Project GR/D076625/01, 2006-2010.
25. M. Sama, D.S. Rosenblum, Z. Wang, and S.G. Elbaum, "Model-based fault detection in context-aware adaptive applications," Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of Software Engineering (SIGSOFT 08/FSE-16), ACM, 2008, pp. 261-271.
26. J.C. Strikwerda, Finite Difference Schemes And Partial Differential Equations, Society for Industrial and Applied Mathematics, 2004.
27. T.H. Tse, F.C.M. Lau, W.K. Chan, P.C.K. Liu, and C.K.F. Luk, "Testing object-oriented industrial software without precise oracles or results," Communications of the ACM, vol. 50, no. 8, pp. 78-85, 2007.
28. T.H. Tse, S.S. Yau, W.K. Chan, H. Lu, and T.Y. Chen, "Testing context-sensitive middleware-based software applications," Proceedings of the 28th Annual International Computer Software and Applications Conference (COMPSAC 04), vol. 1, IEEE Computer Society, 2004, pp. 458-465.
29. E.J. Weyuker, "On testing non-testable programs," The Computer Journal, vol. 25, no. 4, pp. 465-470, 1982.
30. C. Xu, S.C. Cheung, W.K. Chan, and C. Ye, "Partial constraint checking for context consistency in pervasive computing," ACM Transactions on Software Engineering and Methodology, vol. 19, no. 3, p. article no. 9, 2010.
31. S.S. Yau, F. Karim, Y. Wang, B. Wang, and S.K.S. Gupta, "Reconfigurable context-sensitive middleware for pervasive computing," IEEE Pervasive Computing, vol. 1, no. 3, pp. 33-40, 2002.
32. F. Zambonelli, N.R. Jennings, and M. Wooldridge, "Developing multiagent systems: The Gaia methodology," ACM Transactions on Software Engineering and Methodology, vol. 12, no. 3, pp. 317-370, 2003.
33. Z.Q. Zhou, S. Zhang, M. Hagenbuchner, T.H. Tse, F.-C. Kuo, and T.Y. Chen, "Automated functional testing of online search services," Software Testing, Verification and Reliability, vol. 22, no. 4, pp. 221-243, 2012