A general framework for comparing automatic testing techniques of Android mobile apps

Journal of Systems and Software

Volumn 125, Issue , 2017, Pages 322-343

  Amalfitano, Domenico ^a   Amatucci, Nicola ^a   Memon, Atif M ^b   Tramontana, Porfirio ^a   Fasolino, Anna Rita ^a  

^a UNIVERSITY OF NAPLES FEDERICO II   (Italy)

^b Institute of Advanced Computer Science   (United States)

Author keywords

Android testing; Automated testing; Comparing android testing techniques; Event based testing; Mobile apps testing; Random testing

Indexed keywords

AUTOMATIC TESTING; QUALITY ASSURANCE; QUALITY CONTROL;

AUTOMATED TESTING; EVENT-BASED; MOBILE APPS; RANDOM TESTING; TESTING TECHNIQUE;

ANDROID (OPERATING SYSTEM);

EID: 85008239831     PISSN: 01641212     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.jss.2016.12.017     Document Type: Article

References (50)

1. Android Instrumentation, 2012. http://developer.android.com/tools/testing/testing_android.html.
2. Amalfitano, D., Amatucci, N., Fasolino, A.R., Tramontana, P., Agrippin: A novel search based testing technique for android applications. Proceedings of the 3rd International Workshop on Software Development Lifecycle for Mobile, 2015, ACM, New York, NY, USA, 5–12, 10.1145/2804345.2804348.
3. Amalfitano, D., Amatucci, N., Fasolino, A.R., Tramontana, P., A conceptual framework for the comparison of fully automated gui testing techniques. Proceedings of the 30th IEEE/ACM International Conference on Automated Software Engineering Workshop (ASEW), 2015, 50–57, 10.1109/ASEW.2015.19.
4. Amalfitano, D., Amatucci, N., Fasolino, A.R., Tramontana, P., Kowalczyk, E., Memon, A.M., Exploiting the saturation effect in automatic random testing of android applications. Proceedings of the 2nd ACM International Conference on Mobile Software Engineering and Systems (MOBILESoft), 2015, 33–43, 10.1109/MobileSoft.2015.11.
5. Amalfitano, D., Fasolino, A., Tramontana, P., De Carmine, S., Imparato, G., A toolset for gui testing of android applications. Proceedings of the 28th IEEE International Conference on Software Maintenance (ICSM), 2012, 650–653, 10.1109/ICSM.2012.6405345.
6. Amalfitano, D., Fasolino, A.R., Tramontana, P., A gui crawling-based technique for android mobile application testing. Proceedings of the 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops (ICSTW), 2011, 252–261, 10.1109/ICSTW.2011.77.
7. Amalfitano, D., Fasolino, A.R., Tramontana, P., De Carmine, S., Memon, A.M., Using gui ripping for automated testing of android applications. Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering, 2012, ACM, New York, NY, USA, 258–261, 10.1145/2351676.2351717.
8. Amalfitano, D., Fasolino, A.R., Tramontana, P., Ta, B.D., Memon, A.M., Mobiguitar: automated model-based testing of mobile apps. IEEE Softw. 32:5 (2015), 53–59, 10.1109/MS.2014.55.
9. Anand, S., Naik, M., Harrold, M. J., Yang, H., 2012. Automated concolic testing of smartphone apps, 59:1–59:11. 10.1145/2393596.2393666.
10. Angluin, D., Learning regular sets from queries and counterexamples. Inf. Comput. 75:2 (1987), 87–106, 10.1016/0890-5401(87)90052-6.
11. Azim, T., Neamtiu, I., Targeted and depth-first exploration for systematic testing of android apps. SIGPLAN Not. 48:10 (2013), 641–660, 10.1145/2544173.2509549.
12. Bae, G., Rothermel, G., Bae, D.-H., On the relative strengths of model-based and dynamic event extraction-based gui testing techniques: An empirical study. Proceedings of the 2012 IEEE 23rd International Symposium on Software Reliability Engineering (ISSRE), 2012, 181–190, 10.1109/ISSRE.2012.18.
13. Belli, F., Beyazit, M., Memon, A., Testing is an event-centric activity. Proceedings of the 2012 IEEE Sixth International Conference on Software Security and Reliability Companion, 2012, IEEE Computer Society, Washington, DC, USA, 198–206, 10.1109/SERE-C.2012.24.
14. Cai, X., Lyu, M.R., The effect of code coverage on fault detection under different testing profiles. SIGSOFT Softw. Eng. Notes 30:4 (2005), 1–7, 10.1145/1082983.1083288.
15. Chen, T.Y., Kuo, F.-C., Merkel, R.G., Tse, T.H., Adaptive random testing: the art of test case diversity. J. Syst. Softw. 83:1 (2010), 60–66, 10.1016/j.jss.2009.02.022.
16. Chen, T.Y., Leung, H., Mak, I.K., Adaptive random testing. Proceedings of the 9th Asian Computing Science Conference on Advances in Computer Science, 2004, Springer-Verlag, Berlin, Heidelberg, 320–329, 10.1007/978-3-540-30502-6_23.
17. Choi, W., Necula, G., Sen, K., Guided gui testing of android apps with minimal restart and approximate learning. Proceedings of the 2013 ACM SIGPLAN International Conference on Object Oriented Programming Systems Languages & Applications, 2013, ACM, New York, NY, USA, 623–640, 10.1145/2509136.2509552.
18. Choudhary, S.R., Gorla, A., Orso, A., Automated test input generation for android: Are we there yet? (e). Proceedings of 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE), 2015, 429–440, 10.1109/ASE.2015.89.
19. Emma: a free java code coverage tool. http://emma.sourceforge.net/index.html.
20. Espresso, 2012. http://developer.android.com/training/testing/ui-testing/espresso-testing.html.
21. Gomez, L., Neamtiu, I., Azim, T., Millstein, T., Reran: Timing- and touch-sensitive record and replay for android. Proceedings of 2013 35th International Conference on Software Engineering (ICSE), 2013, 72–81, 10.1109/ICSE.2013.6606553.
22. Griebe, T., Gruhn, V., A model-based approach to test automation for context-aware mobile applications. Proceedings of the 29th Annual ACM Symposium on Applied Computing, 2014, ACM, New York, NY, USA, 420–427, 10.1145/2554850.2554942.
23. Hao, S., Liu, B., Nath, S., Halfond, W.G., Govindan, R., Puma: Programmable ui-automation for large-scale dynamic analysis of mobile apps. Proceedings of the 12th Annual International Conference on Mobile Systems, Applications, and Services, 2014, ACM, New York, NY, USA, 204–217, 10.1145/2594368.2594390.
24. Hicinbothom, J.H., Zachary, W.W., A tool for automatically generating transcripts of human-computer interaction. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 37, 1993, 1042, 10.1177/154193129303701514.
25. Hu, C., Neamtiu, I., Automating gui testing for android applications. Proceedings of the 6th International Workshop on Automation of Software Test, 2011, ACM, New York, NY, USA, 77–83, 10.1145/1982595.1982612.
26. Hu, G., Yuan, X., Tang, Y., Yang, J., Efficiently, effectively detecting mobile app bugs with appdoctor. Proceedings of the Ninth European Conference on Computer Systems, 2014, ACM, New York, NY, USA, 18:1–18:15, 10.1145/2592798.2592813.
27. Jensen, C.S., Prasad, M.R., Møller, A., Automated testing with targeted event sequence generation. Proceedings of the 2013 International Symposium on Software Testing and Analysis, 2013, ACM, New York, NY, USA, 67–77, 10.1145/2483760.2483777.
28. Lin, Y.-D., Chu, E.-H., Yu, S.-C., Lai, Y.-C., Improving the accuracy of automated gui testing for embedded systems. IEEE Software 31:1 (2014), 39–45, 10.1109/MS.2013.100.
29. Lin, Y.-D., Rojas, J., Chu, E.-H., Lai, Y.-C., On the accuracy, efficiency, and reusability of automated test oracles for android devices. IEEE Trans. Softw. Eng. 40:10 (2014), 957–970, 10.1109/TSE.2014.2331982.
30. Liu, C.-H., Lu, C.-Y., Cheng, S.-J., Chang, K.-Y., Hsiao, Y.-C., Chu, W.-M., Capture-replay testing for android applications. Proceedings of 2014 International Symposium on Computer, Consumer and Control (IS3C), 2014, 1129–1132, 10.1109/IS3C.2014.293.
31. Liu, Z., Gao, X., Long, X., Adaptive random testing of mobile application. Proceedings of 2010 2nd International Conference on Computer Engineering and Technology (ICCET), 2, 2010, V2–297–V2–301, 10.1109/ICCET.2010.5485442.
32. Machiry, A., Tahiliani, R., Naik, M., Dynodroid: An input generation system for android apps. Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering, 2013, ACM, New York, NY, USA, 224–234, 10.1145/2491411.2491450.
33. Mahmood, R., Mirzaei, N., Malek, S., Evodroid: Segmented evolutionary testing of android apps. Proceedings of the 22Nd ACM SIGSOFT International Symposium on Foundations of Software Engineering, 2014, ACM, New York, NY, USA, 599–609, 10.1145/2635868.2635896.
34. Maiya, P., Kanade, A., Majumdar, R., Race detection for android applications. SIGPLAN Not. 49:6 (2014), 316–325, 10.1145/2666356.2594311.
35. Monkey. http://developer.android.com/tools/help/monkey.html.
36. Monkeyrunner. http://developer.android.com/tools/index.html.
37. Morgado, I.C., Paiva, A.C., The impact tool: Testing ui patterns on mobile applications. Proceedings of 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE), 2015, 876–881, 10.1109/ASE.2015.96.
38. Nguyen, C.D., Marchetto, A., Tonella, P., Combining model-based and combinatorial testing for effective test case generation. Proceedings of the 2012 International Symposium on Software Testing and Analysis, 2012, ACM, New York, NY, USA, 100–110, 10.1145/2338965.2336765.
39. Robotium, 2012. http://code.google.com/p/robotium/.
40. Salva, S., Zafimiharisoa, S., Apset, an android application security testing tool for detecting intent-based vulnerabilities. Int. J. Softw. Tools Technol. Trans. 17:2 (2015), 201–221, 10.1007/s10009-014-0303-8.
41. Sasnauskas, R., Regehr, J., Intent fuzzer: Crafting intents of death. Proceedings of the 2014 Joint International Workshop on Dynamic Analysis (WODA) and Software and System Performance Testing, Debugging, and Analytics (PERTEA), 2014, ACM, New York, NY, USA, 1–5, 10.1145/2632168.2632169.
42. Takala, T., Katara, M., Harty, J., Experiences of system-level model-based gui testing of an android application. Proceedings of the 2011 Fourth IEEE International Conference on Software Testing, Verification and Validation, 2011, IEEE Computer Society, Washington, DC, USA, 377–386, 10.1109/ICST.2011.11.
43. Utting, M., Pretschner, A., Legeard, B., A taxonomy of model-based testing approaches. Software Testing, Verification and Reliability 22:5 (2012), 297–312, 10.1002/stvr.456.
44. Wang, P., Liang, B., You, W., Li, J., Shi, W., Automatic android gui traversal with high coverage. Proceedings of the 2014 Fourth International Conference on Communication Systems and Network Technologies, 2014, IEEE Computer Society, Washington, DC, USA, 1161–1166, 10.1109/CSNT.2014.236.
45. Wen, H., Lin, C., Hsieh, T., Yang, C., PATS: A parallel GUI testing framework for android applications. Proceedings of 39th IEEE Annual Computer Software and Applications Conference, COMPSAC 2015, Taichung, Taiwan, July 1–5, 2015, 2, 2015, 210–215, 10.1109/COMPSAC.2015.80.
46. Wohlin, C., Runeson, P., Höst, M., Ohlsson, M.C., Regnell, B., Wesslén, A., Experimentation in Software Engineering: An Introduction. 2000, Kluwer Academic Publishers, Norwell, MA, USA.
47. Yang, W., Prasad, M.R., Xie, T., A grey-box approach for automated gui-model generation of mobile applications. Proceedings of the 16th International Conference on Fundamental Approaches to Software Engineering, 2013, Springer-Verlag, Berlin, Heidelberg, 250–265, 10.1007/978-3-642-37057-1_19.
48. Ye, H., Cheng, S., Zhang, L., Jiang, F., Droidfuzzer: Fuzzing the android apps with intent-filter tag. Proceedings of International Conference on Advances in Mobile Computing & Multimedia, 2013, ACM, New York, NY, USA, 68:68–68:74, 10.1145/2536853.2536881.
49. Zaeem, R.N., Prasad, M.R., Khurshid, S., Automated generation of oracles for testing user-interaction features of mobile apps. Proceedings of the 2014 IEEE International Conference on Software Testing, Verification, and Validation, 2014, IEEE Computer Society, Washington, DC, USA, 183–192, 10.1109/ICST.2014.31.
50. Zhauniarovich, Y., Philippov, A., Gadyatskaya, O., Crispo, B., Massacci, F., Towards black box testing of android apps. Proceedings of the 2015 10th International Conference on Availability, Reliability and Security (ARES), 2015, 501–510, 10.1109/ARES.2015.70.