The GISMOE challenge: Constructing the pareto program surface using genetic programming to find better programs (Keynote Paper)

2012 27th IEEE/ACM International Conference on Automated Software Engineering, ASE 2012 - Proceedings

Volumn , Issue , 2012, Pages 1-14

  Harman, Mark ^a   Langdon, William B ^a   Jia, Yue ^a   White, David R ^b   Arcuri, Andrea ^c   Clark, John A ^d  

^a UNIVERSITY COLLEGE LONDON   (United Kingdom)

^b UNIVERSITY OF GLASGOW   (United Kingdom)

^c SIMULA RESEARCH LABORATORY   (Norway)

^d UNIVERSITY OF YORK   (United Kingdom)

Author keywords

Compilation; GeneticProgramming; Non functional properties Pareto Surface; SBSE; Search Based Optimization

Indexed keywords

ALL-AT-ONCE; COMPILATION; COMPILER TECHNOLOGY; FUNCTIONAL REQUIREMENT; GAINING INSIGHTS; INPUT PROGRAMS; NON FUNCTIONAL PROPERTIES; NON-FUNCTIONAL; PARETO SURFACE; PROGRAM IMPLEMENTATION; PROGRAMMING ENVIRONMENT; SBSE; SEARCH-BASED; SEARCH-BASED SOFTWARE ENGINEERING; SOFTWARE DESIGNERS; SOFTWARE DEVELOPMENT ENVIRONMENT; TRADE OFF; SEARCH BASED OPTIMIZATIONS;

BANDWIDTH; GENETIC PROGRAMMING; PARETO PRINCIPLE; SOFTWARE DESIGN; COMMERCE;

COMMERCE; GENETIC ALGORITHMS;

EID: 84866900182     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2351676.2351678     Document Type: Conference Paper

References (98)

1. K. Adamopoulos, M. Harman, and R. M. Hierons. Mutation testing using genetic algorithms: A co-evolution approach. In Genetic and Evolutionary Computation Conference (GECCO 2004), LNCS 3103, pages 1338-1349, Seattle, Washington, USA, June 2004. Springer.
2. W. Afzal and R. Torkar. On the application of genetic programming for software engineering predictive modeling: A systematic review. Expert Systems Applications, 38(9):11984-11997, 2011.
3. W. Afzal, R. Torkar, and R. Feldt. A systematic review of search-based testing for non-functional system properties. Information and Software Technology, 51(6):957-976, 2009.
4. W. Afzal, R. Torkar, R. Feldt, and G. Wikstrand. Search-based prediction of fault-slip-through in large software projects. In Second International Symposium on Search Based Software Engineering (SSBSE 2010), pages 79-88, Benevento, Italy, 7-9 Sept. 2010.
5. S. Ali, L. C. Briand, H. Hemmati, and R. K. Panesar-Walawege. A systematic review of the application and empirical investigation of search-based test-case generation. IEEE Transactions on Software Engineering, pages 742-762, 2010.
6. N. Alshahwan and M. Harman. Automated web application testing using search based software engineering. In 26th IEEE/ACM International Conference on Automated Software Engineering (ASE 2011), pages 3 - 12, Lawrence, Kansas, USA, 6th - 10th November 2011.
7. A. Arcuri, D. R. White, J. A. Clark, and X. Yao. Multi-objective improvement of software using co-evolution and smart seeding. In 7th International Conference on Simulated Evolution and Learning (SEAL 2008), pages 61-70, Melbourne, Australia, December 2008. Springer.
8. A. Arcuri and X. Yao. A Novel Co-evolutionary Approach to Automatic Software Bug Fixing. In Proceedings of the IEEE Congress on Evolutionary Computation (CEC '08), pages 162-168, Hongkong, China, 1-6 June 2008. IEEE Computer Society.
9. A. Arcuri and X. Yao. Co-evolutionary automatic programming for software development. Information Sciences, 2010. To appear. Available on line from Elsevier.
10. A. Baars, M. Harman, Y. Hassoun, K. Lakhotia, P. McMinn, P. Tonella, and T. Vos. Symbolic search-based testing. In 26th IEEE/ACM International Conference on Automated Software Engineering (ASE 2011), pages 53 - 62, Lawrence, Kansas, USA, 6th - 10th November 2011.
11. P. Baker, M. Harman, K. Steinḧofel, and A. Skaliotis. Search based approaches to component selection and prioritization for the next release problem. In 22nd International Conference on Software Maintenance (ICSM 06), pages 176-185, Philadelphia, Pennsylvania, USA, Sept. 2006.
12. L. Beckman, A. Haraldson, O. Oskarsson,, and E. Sandewall. A partial evaluator, and its use as a programming tool. Artificial Intelligence, 7(4):319-357, 1976.
13. B. Beizer. Software Testing Techniques. Van Nostrand Reinhold, 1990.
14. D. Bjørner, A. P. Ershov, and N. D. Jones. Partial evaluation and mixed computation. North-Holland, 1987.
15. C. Cadar, P. Godefroid, S. Khurshid, C. S. P̌ašareanu, K. Sen, N. Tillmann, and W. Visser. Symbolic execution for software testing in practice: preliminary assessment. In 33rd International Conference on Software Engineering (ICSE'11), pages 1066-1071, New York, NY, USA, 2011. ACM.
16. C. Cadar, P. Pietzuch, and A. L. Wolf. Multiplicity computing: a vision of software engineering for next-generation computing platform applications. In G.-C. Roman and K. J. Sullivan, editors, Workshop on Future of Software Engineering Research (FoSER 2010), pages 81-86. ACM, 2010.
17. J. Clark, J. J. Dolado, M. Harman, R. M. Hierons, B. Jones, M. Lumkin, B. Mitchell, S. Mancoridis, K. Rees, M. Roper, and M. Shepperd. Reformulating software engineering as a search problem. IEE Proceedings - Software, 150(3):161-175, 2003.
18. P. C. Clements. Managing variability for software product lines: Working with variability mechanisms. In 10th International Conference on Software Product Lines (SPLC 2006), pages 207-208, Baltimore, Maryland, USA, 2006. IEEE Computer Society.
19. S. L. Cornford, M. S. Feather, J. R. Dunphy, J. Salcedo, and T. Menzies. Optimizing Spacecraft Design - Optimization Engine Development: Progress and Plans. In Proceedings of the IEEE Aerospace Conference, pages 3681-3690, Big Sky, Montana, March 2003.
20. J. Darlington and R. M. Burstall. A system which automatically improves programs. Acta Informatica, 6:41-60, 1976.
21. P. T. Devanbu. GENOA: A customizable language- and front-end independent code analyzer. In T. Montgomery, L. A. Clarke, and C. Ghezzi, editors, 14th International Conference on Software Engineering (ICSE '92), pages 307-317, Melbourne, Australia, May 1992. ACM Press.
22. M. D. Ernst. Dynamically Discovering Likely Program Invariants. PhD Thesis, University of Washington, 2000.
23. M. D. Ernst, J. Cockrell, W. G. Griswold, and D. Notkin. Dynamically discovering likely program invariants to support program evolution. IEEE Transactions on Software Engineering, 27(2):1-25, Feb. 2001.
24. D. Fatiregun, M. Harman, and R. Hierons. Evolving transformation sequences using genetic algorithms. In 4th International Workshop on Source Code Analysis and Manipulation (SCAM 04), pages 65-74, Los Alamitos, California, USA, Sept. 2004. IEEE Computer Society Press.
25. R. Feldt. Generating diverse software versions with genetic programming: and experimental study. IEE Proceedings - Software, 145(6):228-312, 1998.
26. R. Feldt. Genetic programming as an explorative tool in early software development phases. In 1st International Workshop on Soft Computing Applied to Software Engineering, pages 11-20, University of Limerick, Ireland, 12-14 Apr. 1999. Limerick University Press.
27. A. J. Field and P. G. Harrison. Functional Programming. Addison-Wesley, Wokingham, 1988.
28. A. Finkelstein, M. Harman, A. Mansouri, J. Ren, and Y. Zhang. A search based approach to fairness analysis in requirements assignments to aid negotiation, mediation and decision making. Requirements Engineering, 14(4):231-245, 2009.
29. G. Fraser and A. Arcuri. Evosuite: automatic test suite generation for object-oriented software. In 8th European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering (ESEC/FSE '11), pages 416-419. ACM, September 5th - 9th 2011.
30. F. G. Freitas and J. T. Souza. Ten years of search based software engineering: A bibliometric analysis. In 3rd International Symposium on Search based Software Engineering (SSBSE 2011), pages 18-32, 10th - 12th September 2011.
31. Z. P. Fry, B. Landau, and W. Weimer. A human study of patch maintainability. In International Symposium on Software Testing and Analysis (ISSTA'12), Minneapolis, Minnesota, USA, July 2012. To appear.
32. P. Funes, E. Bonabeau, J. Herve, and Y. Morieux. Interactive multi-participant task allocation. In Proceedings of the 2004 IEEE Congress on Evolutionary Computation, pages 1699-1705, Portland, Oregon, 20-23 June 2004. IEEE Press.
33. M. Gabel and Z. Su. A study of the uniqueness of source code. In 18th ACM SIGSOFT international symposium on foundations of software engineering (FSE 2010), pages 147-156, Santa Fe, New Mexico, USA, 7-11 Nov. 2010. ACM.
34. P. Godefroid, N. Klarlund, and K. Sen. DART: directed automated random testing. In V. Sarkar and M. W. Hall, editors, Proceedings of the ACM SIGPLAN 2005 Conference on Programming Language Design and Implementation, Chicago, IL, USA, June 12-15, 2005, pages 213-223. ACM, 2005.
35. M. Goedicke, C. K̈ollmann, and U. Zdun. Designing runtime variation points in product line architectures: three cases. Science of Computer Programming, 53(3):353 - 380, 2004.
36. C. L. Goues, M. Dewey-Vogt, S. Forrest, and W. Weimer. A systematic study of automated program repair: Fixing 55 out of 105 bugs for $8 each. In International Conference on Software Engineering (ICSE 2012), Zurich, Switzerland, 2012.
37. M. Harman. The current state and future of search based software engineering. In L. Briand and A. Wolf, editors, Future of Software Engineering 2007, pages 342-357, Los Alamitos, California, USA, 2007. IEEE Computer Society Press.
38. M. Harman. Search based software engineering for program comprehension. In 15th International Conference on Program Comprehension (ICPC 07), pages 3-13, Banff, Canada, 2007. IEEE Computer Society Press.
39. M. Harman. Open problems in testability transformation. In 1st International Workshop on Search Based Testing (SBT 2008), Lillehammer, Norway, 2008.
40. M. Harman. Automated patching techniques: The fix is in: technical perspective. Communications of the ACM, 53(5):108, 2010.
41. M. Harman. The relationship between search based software engineering and predictive modeling. In 6th International Conference on Predictive Models in Software Engineering (PROMISE 2010), Timisoara, Romania, 2010.
42. M. Harman. Software engineering meets evolutionary computation. IEEE Computer, 44(10):31-39, Oct. 2011.
43. M. Harman, E. Burke, J. A. Clark, and X. Yao. Dynamic adaptive search based software engineering. In 6th IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM 2012), Lund, Sweden, September 2012.
44. M. Harman, Y. Jia, and B. Langdon. Strong higher order mutation-based test data generation. In 8th European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering (ESEC/FSE '11), pages 212-222, New York, NY, USA, September 5th - 9th 2011. ACM.
45. M. Harman and B. F. Jones. Search based software engineering. Information and Software Technology, 43(14):833-839, Dec. 2001.
46. M. Harman, A. Mansouri, and Y. Zhang. Search based software engineering: Trends, techniques and applications. ACM Computing Surveys, 2012. To appear.
47. M. Harman and P. McMinn. A theoretical and empirical study of search based testing: Local, global and hybrid search. IEEE Transactions on Software Engineering, 36(2):226-247, 2010.
48. M. Harman, P. McMinn, J. Souza, and S. Yoo. Search based software engineering: Techniques, taxonomy, tutorial. In B. Meyer and M. Nordio, editors, Empirical software engineering and verification: LASER 2009-2010, pages 1-59. Springer, 2012. LNCS 7007.
49. W. Heaven and E. Letier. Simulating and optimising design decisions in quantitative goal models. In 19th IEEE InternationalRequirements Engineering Conference (RE 2011), pages 79-88. IEEE, 2011.
50. H. Hemmati, A. Arcuri, and L. Briand. Achieving scalable model-based testing through test case diversity. ACM Ttransactions on Software Engineering and Methodology (TOSEM). To appear.
51. A. Hindle, E. Barr, Z. Su, P. Devanbu, and M. Gabel. On the naturalness of software. In International Conference on Software Engineering (ICSE 2012), Zurich, Switzerland, 2012.
52. D. Jackson. Self-adaptive focusing of evolutionary effort in hierarchical genetic programming. In A. Tyrrell, editor, 2009 IEEE Congress on Evolutionary Computation, pages 1821-1828, Trondheim, Norway, 18-21 May 2009. IEEE Computational Intelligence Society, IEEE Press.
53. Y. Jia and M. Harman. Milu: A customizable, runtime-optimized higher order mutation testing tool for the full C language. In 3rd Testing Academia and Industry Conference - Practice and Research Techniques (TAIC PART'08), pages 94-98, Windsor, UK, August 2008.
54. J. Krinke, N. Gold, Y. Jia, and D. Binkley. Cloning and copying between GNOME projects. In J. Whitehead and T. Zimmermann, editors, 7th International Working Conference on Mining Software Repositories (MSR 2010), pages 98-101. IEEE, 2010.
55. L. M. Laird and M. C. Brennan. Software measurement and estimation: a practical approach, 2006.
56. K. Lakhotia, M. Harman, and H. Gross. AUSTIN: An open source tool for search based software testing of C programs. Journal of Information and Software Technology. To appear.
57. K. Lakhotia, P. McMinn, and M. Harman. Automated test data generation for coverage: Haven't we solved this problem yet? In 4th Testing Academia and Industry Conference - Practice And Research Techniques (TAIC PART'09), pages 95-104, Windsor, UK, 4th-6th September 2009.
58. K. Lakhotia, N. Tillmann, M. Harman, and J. de Halleux. FloPSy - Search-based floating point constraint solving for symbolic execution. In 22nd IFIP International Conference on Testing Software and Systems (ICTSS 2010), pages 142-157, Natal, Brazil, November 2010. LNCS Volume 6435.
59. W. B. Langdon. Evo indent interactive evolution of GNU indent options. In F. Rothlauf, editor, Genetic and Evolutionary Computation Conference (GECCO 2009), pages 2081-2084, Montreal, Qúebec, Canada, 2009. ACM.
60. W. B. Langdon and M. Harman. Evolving a CUDA kernel from an nVidia template. In IEEE Congress on Evolutionary Computation, pages 1-8. IEEE, 2010.
61. W. B. Langdon, M. Harman, and Y. Jia. Efficient multi objective higher order mutation testing with genetic programming. Journal of Systems and Software, 83(12):2416-2430, 2010.
62. W. B. Langdon and A. P. Harrison. GP on SPMD parallel graphics hardware for mega bioinformatics data mining. Soft Computing, 12(12):1169-1183, Oct. 2008. Special Issue on Distributed Bioinspired Algorithms.
63. C. Le Goues, T. Nguyen, S. Forrest, and W. Weimer. GenProg: A generic method for automatic software repair. IEEE Transactions on Software Engineering, 38(1):54-72, 2012.
64. E. Letier and A. van Lamsweerde. Deriving operational software specifications from system goals. In 10th ACM SIGSOFT symposium on foundations of software engineering (FSE '02), pages 119-128, 2002.
65. P. McMinn. Search-based software test data generation: A survey. Software Testing, Verification and Reliability, 14(2):105-156, June 2004.
66. B. S. Mitchell and S. Mancoridis. On the automatic modularization of software systems using the bunch tool. IEEE Transactions on Software Engineering, 32(3):193-208, 2006.
67. I. H. Moghadam and Mel ́O Cinńeide. Code-Imp: A tool for automated search-based refactoring. In Proceeding of the 4th workshop on Refactoring Tools (WRT '11), pages 41-44, Honolulu, HI, USA, 2011.
68. D. J. Montana. Strongly typed genetic programming. Evolutionary Computation, 3(2):199-230, 1995.
69. A. Ngo-The and G. Ruhe. A systematic approach for solving the wicked problem of software release planning. Soft Computing - A Fusion of Foundations, Methodologies and Applications, 12(1):95-108, August 2008.
70. M. O'Neill and C. Ryan. Grammatical evolution. IEEE Transactions on Evolutionary Computation, 5(4):349-358, Aug. 2001.
71. M. Orlov and M. Sipper. Flight of the FINCH through the java wilderness. IEEE Transactions Evolutionary Computation, 15(2):166-182, 2011.
72. H. A. Partsch. The Specification and Transformation of Programs: A Formal Approach to Software Development. Springer, 1990.
73. S. Paul and A. Prakash. A framework for source code search using program patterns. IEEE Transactions on Software Engineering, 20(6):463-475, June 1994.
74. H. Pohlheim and J. Wegener. Testing the temporal behavior of real-time software modules using extended evolutionary algorithms. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCDO '99), volume 2, page 1795, San Francisco, CA 94104, USA, 13-17 July 1999. Morgan Kaufmann.
75. R. Poli and J. Page. Solving high-order boolean parity problems with smooth uniform crossover, sub-machine code GP and demes. Genetic Programming and Evolvable Machines, 1(1/2):37-56, Apr. 2000.
76. O. Räihä. A survey on search-based software design. Computer Science Review, 4(4):203-249, 2010.
77. C. Ryan. Automatic Re-engineering of Software Using Genetic Programming, volume 2 of Genetic Programming. Kluwer Academic Publishers, 1 Nov. 1999.
78. B. Salamat, T. Jackson, A. Gal, and M. Franz. Orchestra: Intrusion detection using parallel execution and monitoring of program variants in user-space. In 4th EuroSys Conference (EuroSys'09), pages 33-46, Nuremberg, Germany, Apr. 2009. ACM.
79. M. O. Saliu and G. Ruhe. Bi-objective release planning for evolving software systems. In I. Crnkovic and A. Bertolino, editors, Proceedings of the 6th joint meeting of the European Software Engineering Conference and the ACM SIGSOFT International Symposium on Foundations of Software Engineering (ESEC/FSE) 2007, pages 105-114. ACM, Sept. 2007.
80. R. A. Santelices, P. K. Chittimalli, T. Apiwattanapong, A. Orso, and M. J. Harrold. Test-suite augmentation for evolving software. In 23rd Automated Software Engineering (ASE '08), pages 218-227, L'Aquila, Italy, 2008. IEEE.
81. K. Sen, D. Marinov, and G. Agha. CUTE: a concolic unit testing engine for C. In M. Wermelinger and H. Gall, editors, 10th European Software Engineering Conference and 13th ACM International Symposium on Foundations of Software Engineering (ESEC/FSE '05), pages 263-272. ACM, 2005.
82. S. Sidiroglou-Douskos, S. Misailovic, H. Hoffmann, and M. C. Rinard. Managing performance vs. accuracy trade-offs with loop perforation. In T. Gyiḿothy and A. Zeller, editors, 19th ACM Symposium on the Foundations of Software Engineering (FSE-19), pages 124-134, Szeged, Hungary, Sept. 2011. ACM.
83. C. L. Simons, I. C. Parmee, and R. Gwynllyw. Interactive, evolutionary search in upstream object-oriented class design. IEEE Transactions on Software Engineering, 36(6):798-816, 2010.
84. P. Tonella. Evolutionary testing of classes. In Proceedings of the 2004 ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA '04), pages 119-128, Boston, Massachusetts, USA, 11-14 July 2004. ACM.
85. A. van Lamsweerde. Requirements Engineering - From System Goals to UML Models to Software Specifications. Wiley, 2009.
86. J. A. Walker and J. F. Miller. The automatic acquisition, evolution and reuse of modules in cartesian genetic programming. IEEE Transactions on Evolutionary Computation, 12(4):397-417, Aug. 2008.
87. J. Wegener and O. B̈uhler. Evaluation of different fitness functions for the evolutionary testing of an autonomous parking system. In Genetic and Evolutionary Computation Conference (GECCO 2004), pages 1400-1412, Seattle, Washington, USA, June 2004. LNCS 3103.
88. W. Weimer, T. V. Nguyen, C. L. Goues, and S. Forrest. Automatically finding patches using genetic programming. In International Conference on Software Engineering (ICSE 2009), pages 364-374, Vancouver, Canada, 2009.
89. E. J. Weyuker. On testing non-testable programs. The Computer Journal, 25(4):465-470, Nov. 1982.
90. D. R. White. Genetic Programming for Low-Resource Systems. PhD Thesis, Dapartment of Computer Science, University of York, UK, 2010.
91. D. R. White, A. Arcuri, and J. A. Clark. Evolutionary improvement of programs. IEEE Transactions on Evolutionary Computation, 15(4):515-538, 2011.
92. D. R. White, J. Clark, J. Jacob, and S. Poulding. Searching for resource-efficient programs: Low-power pseudorandom number generators. In 2008 Genetic and Evolutionary Computation Conference (GECCO 2008), pages 1775-1782, Atlanta, USA, July 2008. ACM Press.
93. S. Yoo and M. Harman. Regression testing minimisation, selection and prioritisation: A survey. Journal of Software Testing, Verification and Reliability, 22(2):67-120, 2012.
94. S. Yoo and M. Harman. Test data regeneration: Generating new test data from existing test data. Journal of Software Testing, Verification and Reliability, 2012. To appear.
95. S. Yoo, M. Harman, and S. Ur. Highly scalable multi-objective test suite minimisation using graphics cards. In 3rd International Symposium on Search based Software Engineering (SSBSE 2011), pages 219-236, 10th - 12th September 2011. LNCS Volume 6956.
96. S. Yoo, R. Nilsson, and M. Harman. Faster fault finding at Google using multi objective regression test optimisation. In 8th European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering (ESEC/FSE '11), Szeged, Hungary, September 5th - 9th 2011. Industry Track.
97. Y. Zhang, E. Alba, J. J. Durillo, S. Eldh, and M. Harman. Today/future importance analysis. In ACM Genetic and Evolutionary Computation COnference (GECCO 2010), pages 1357-1364, Portland Oregon, USA, 7th-11th July 2010.
98. Y. Zhang, A. Finkelstein, and M. Harman. Search based requirements optimisation: Existing work and challenges. In International Working Conference on Requirements Engineering: Foundation for Software Quality (REFSQ'08), volume 5025, pages 88-94, Montpellier, France, 2008. Springer LNCS.