Performance evaluation of component-based software systems: A survey

Performance Evaluation

Volumn 67, Issue 8, 2010, Pages 634-658

  Koziolek, Heiko ^a  

^a ABB CORPORATE RESEARCH   (Germany)

Author keywords

CBSE; Classification; Measurement; Modelling; Performance; Prediction; Software component; Survey

Indexed keywords

COMPONENT PERFORMANCE; COMPONENT-BASED SOFTWARE ENGINEERING; COMPONENT-BASED SOFTWARE SYSTEMS; INDUSTRIAL USE; MODELLING; PERFORMANCE EVALUATION; PERFORMANCE MODEL; PERFORMANCE PREDICTION; SOFTWARE ARCHITECTS; SOFTWARE COMPONENT; STOCHASTIC PETRI NETS; STOCHASTIC PROCESS ALGEBRAS;

FORECASTING; GRAPH THEORY; INDUSTRIAL APPLICATIONS; MODELS; PETRI NETS; QUEUEING NETWORKS; RANDOM PROCESSES; SOFTWARE ARCHITECTURE; STOCHASTIC MODELS; STOCHASTIC SYSTEMS; SURVEYS;

COMPUTER SOFTWARE REUSABILITY;

EID: 77955305945     PISSN: 01665316     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.peva.2009.07.007     Document Type: Article

References (102)

1. Clemens Szyperski, Daniel Gruntz, and Stephan Murer Component Software: Beyond Object-Oriented Programming 2002 Addison-Wesley
2. George T. Heineman, William T. Councill, Component-Based Software Engineering: Putting The Pieces Together 2001 Addison-Wesley Longman Publishing Co., Inc Boston, MA, USA
3. Edward D. Lazowska, John Zahorjan, Scott Graham, and Kenneth C. Sevcik Quantitative System Performance 1984 Prentice Hall
4. Falko Bause, and Peter S. Kritzinger Stochastic Petri NetsAn Introduction to the Theory 2nd ed. 2002 Vieweg Verlag
5. Holger Hermanns, Ulrich Herzog, and Joost-Pieter Katoen Process algebra for performance evaluation Theoretical Computer Science 274 1-2 2002 43 87
6. Microsoft ACE Team Performance Testing Microsoft.NET Web Applications 2002 Microsoft Press
7. Peter Zadrozny, Philip Aston, and Ted Osborne J2EE Performance Testing 2003 A-Press
8. Microsoft Corporation Improving.NET Application Performance and Scalability (Patterns & Practices) 2004 Microsoft Press
9. Lloyd G. Williams, Connie U. Smith, Making the business case for software performance engineering, in: Proceedings of CMG, 2003 (last retrieved 13.01.08)
10. Murray Woodside, Greg Franks, and Dorina Petriu The future of software performance engineering Future of Software Engineering FOSE'07 2007 IEEE Computer Society Los Alamitos, CA, USA 171 187
11. Simonetta Balsamo, Antinisca DiMarco, Paola Inverardi, and Marta Simeoni Model-based performance prediction in software development: A survey IEEE Trans. Softw. Eng. 30 5 2004 295 310
12. Erik Putrycz, Murray Woodside, and Xiuping Wu Performance techniques for cots systems IEEE Software 22 4 2005 36 44
13. Steffen Becker, Lars Grunske, Raffaela Mirandola, and Sven Overhage Performance prediction of component-based systems: A survey from an engineering perspective Ralf Reussner, Judith Stafford, Clemens Szyperski, Architecting Systems with Trustworthy Components LNCS vol. 3938 2006 Springer 169 192
14. Thomas Stahl, Markus Voelter, and Krzysztof Czarnecki Model-Driven Software Development: Technology, Engineering, Management 2006 John Wiley & Sons
15. Rick Kazman, M. Klein, Paul Clements, ATAM: Method for architecture evaluation, Technical Report CMU/SEI-2000-TR-004, Carnegie Mellon University, Software Engineering Institute, 2000
16. Rick Kazman, Len Bass, Mike Webb, and Gregory Abowd SAAM: A method for analyzing the properties of software architectures Proc. 16th International Conference on Software Engineering ICSE'94 1994 IEEE Computer Society Press Los Alamitos, CA, USA 81 90
17. Antinisca DiMarco, and Paola Inverardi Compositional generation of software architecture performance qn models Proc. 4th Working IEEE/IFIP Conference on Software Architecture WISCA'04 2004 IEEE 37 46
18. Hassan Gomaa, and Daniel A. Menasce Performance engineering of component-based distributed software systems Performance Engineering, State of the Art and Current Trends 2001 Springer-Verlag London, UK 40 55
19. Marcel Harkema, Bart Gijsen, Rob van der Mei, Bart Nieuwenhuis, Performance comparison of middleware threading strategies, in: Proc. of International Symposium on Performance Evaluation of Computer and Communication Systems, SPECTS'04, 2004
20. Samuel Kounev Performance modeling and evaluation of distributed component-based systems using queueing petri nets IEEE Trans. Softw. Eng. 32 7 2006 486 502
21. Malcom Douglas McIlroy Mass-Produced Software Components Software Engineering Concepts and Techniques (NATO Science Committee) 1 1968 88 98
22. Microsoft Corp, The COM homepage. http://www.microsoft.com/com/ (last retrieved 13.01.08)
23. Sun Microsystems Corp., The Enterprise Java Beans homepage. http://java.sun.com/products/ejb/, 2007 (last retrieved 13.01.08)
24. Object Management Group (OMG). Corba component model, v4.0 (formal/2006-04-01). http://www.omg.org/technology/documents/formal/components. htm 2006 (last retrieved 13.01.08)
25. Kung-Kiu Lau, and Zheng Wang Software component models IEEE Transactions on Software Engineering 33 10 2007 709 724
26. Steffen Becker, and Ralf Reussner The impact of software component adaptation on quality of service properties L'objet 12 1 2006 105 125
27. John Cheesman, and John Daniels UML Components: A Simple Process for Specifying Component-based Software Systems 2001 Addison-Wesley
28. Ralf H. Reussner, Iman H. Poernomo, and Heinz W. Schmidt Reasoning on software architectures with contractually specified components A. Cechich, M. Piattini, A. Vallecillo, Component-Based Software Quality: Methods and Techniques LNCS 2693 2003 Springer 287 325
29. Connie U. Smith Performance Solutions: A Practical Guide To Creating Responsive, Scalable Software 2002 Addison-Wesley
30. Thomas Kappler, Heiko Koziolek, Klaus Krogmann, and Ralf Reussner Towards automatic construction of reusable prediction models for component-based performance engineering Proc. Software Engineering 2008 SE'08 LNI vol. 121 2008 GI
31. Object Management Group (OMG). Unified modeling language: Infrastructure version 2.1.1. http://www.omg.org/cgi-bin/doc?formal/07-02-06 February 2007 (last retrieved 13.01.08)
32. Object Management Group (OMG). UML Profile for Schedulability, Performance and Time. http://www.omg.org/cgi-bin/doc?formal/2005-01-02 2005 (last retrieved 13.01.08)
33. Object Management Group (OMG). UML Profile for MARTE, Beta 1. http://www.omg.org/cgi-bin/doc?ptc/2007-08-04 August 2007 (last retrieved 13.01.08)
34. Pekka Kahkipuro UML-based performance modeling framework for component-based distributed systems Performance Engineering, State of the Art and Current Trends 2001 Springer-Verlag London, UK 167 184
35. Antonia Bertolino, and Raffaela Mirandola Cb-spe tool: Putting component-based performance engineering into practice Proc. 7th Int. Symposium on Component-based Software Engineering CBSE'04 LNCS vol. 3054 2004 Springer 233 248
36. Simonetta Balsamo, Moreno Marzolla, and Raffaela Mirandola Efficient performance models in component-based software engineering EUROMICRO '06: Proceedings of the 32nd EUROMICRO Conference on Software Engineering and Advanced Applications 2006 IEEE Computer Society Washington, DC, USA 64 71 (Pubitemid 47159572)
37. Xiuping Wu, and Murray Woodside Performance modeling from software components Proc. 4th International Workshop on Software and Performance, volume 29 WOSP'04 2004 ACM Press New York, NY, USA 290 301
38. Jerome A. Rolia, and Kenneth C. Sevcik The method of layers IEEE Trans. Softw. Eng. 21 8 1995 689 700
39. Jing Xu, Alexandre Oufimtsev, Murray Woodside, and Liam Murphy Performance modeling and prediction of enterprise javabeans with layered queuing network templates SIGSOFT Softw. Eng. Notes 31 2 2006 5
40. Scott A. Hissam, Gabriel A. Moreno, Judith A. Stafford, and Kurt C. Wallnau Packaging predictable assembly Proc. IFIP/ACM Working Conference on Component Deployment CD'02 2002 Springer-Verlag London, UK 108 124
41. Kurt Wallnau, James Ivers, Snapshot of CCL: A Language for Predictable Assembly. Technical report, Software Engineering Institute, Carnegie Mellon University, 2003
42. Scott Hissam, Mark Klein, John Lehoczky, Paulo Merson, Gabriel Moreno, Kurt Wallnau, Performance property theory for predictable assembly from certifiable components (pacc). Technical report, Software Engineering Institute, Carnegie Mellon University, 2004
43. Gabriel Moreno, and Paulo Merson Model-driven performance analysis Proc. 4th Int. Conf. on the Quality of Software Architecture QoSA'08 LNCS vol. 5281 2008 Springer 135 151
44. Scott Hissam, Gabriel Moreno, Daniel Plakosh, Isak Savo, Marcin Stemarczyk, Predicting the behavior of a highly configurable component based real-time system, in: Proc. Euromicro Conf. on Real-Time Systems 2008, ECRTS'08, July 2008, pp. 5768
45. Steffen Gbel, Christoph Pohl, Simone Rüttger, and Steffen Zschaler The comquad component model: Enabling dynamic selection of implementations by weaving non-functional aspects Proceedings of the 3rd International Conference on Aspect-Oriented Software Development AOSD'04 2004 ACM Press New York, NY, USA 74 82
46. Marcus Meyerhfer, Christoph Neumann, TESTEJB-A Measurement Framework for EJBs, in: Proceedings of the 7th International Symposium on Component-Based Software Engineering, CBSE7, 2004
47. Marcus Meyerhfer, Messung und Verwaltung von Software-Komponenten für die Performance-Vorhersage, Ph.D. Thesis, University of Erlangen-Nuernberg, 2007
48. Vincenzo Grassi, Raffaela Mirandola, and Antonino Sabetta From design to analysis models: A kernel language for performance and reliability analysis of component-based systems Proc. 5th International Workshop on Software and Performance WOSP'05 2005 ACM Press New York, NY, USA 25 36
49. Vincenzo Grassi, Raffaela Mirandola, and Antonino Sabetta Filling the gap between design and performance/reliability models of component-based systems: A model-driven approach Journal on Systems and Software 80 4 2007 528 558
50. Egor Bondarev, Johan Muskens, Peter de With, Michel Chaudron, and Johan Lukkien Predicting real-time properties of component assemblies: A scenario-simulation approach Proc. 30th EUROMICRO Conf EUROMICRO'04 2004 IEEE Computer Society Washington, DC, USA 40 47
51. Jean Gelissen, Robocop: Robust open component based software architecture. http://www.hitech-projects.com/euprojects/robocop/deliverables.htm (last retrieved 13.01.08)
52. Egor Bondarev, Michel R.V. Chaudron, and Erwin A. de Kock Exploring performance trade-offs of a JPEG decoder using the deepcompass framework Proc. of the 6th International Workshop on Software and Performance WOSP'07 2007 ACM New York, NY, USA 153 163
53. Steffen Becker, Heiko Koziolek, and Ralf Reussner Model-based performance prediction with the palladio component model Proc. 6th Int. Workshop on Software and Performance WOSP'07 2007 ACM New York, NY, USA 54 65
54. Heiko Koziolek, Parameter dependencies for reusable performance specifications of software components, Ph.D. Thesis, University of Oldenburg, Germany, March 2008
55. Michael Kuperberg, Klaus Krogmann, and Ralf Reussner Performance prediction for black-box components using reengineered parametric behaviour models Proceedings of the 11th International Symposium on Component Based Software Engineering CBSE 2008, Karlsruhe, Germany, 14th-17th October 2008 LNCS vol. 5282 2008 Springer 48 63
56. Steffen Becker, Coupled model transformations for QoS enabled component-based software design, Ph.D. Thesis, University of Oldenburg, Germany, January 2008
57. Jens Happe, Holger Friedrichs, Steffen Becker, and Ralf H. Reussner A pattern-based performance completion for Message-oriented Middleware WOSP '08: Proceedings of the 7th International Workshop on Software and Performance 2008 ACM New York, NY, USA 978-1-59593-873-2 165 176
58. Catalina M. Llado, and Peter G. Harrison Performance evaluation of an enterprise JavaBean server implementation Proc. 2nd Int. Workshop on Software and Performance WOSP'00 2000 ACM New York, NY, USA 180 188
59. Emmanuel Cecchet, Julie Marguerite, and Willy Zwaenepoel Performance and scalability of ejb applications Proc. 17th ACM SIGPLAN Conf. on Object-oriented programming, systems, languages, and applications OOPSLA'02 2002 ACM New York, NY, USA 246 261
60. Giovanni Denaro, Andrea Polini, and Wolfgang Emmerich Early performance testing of distributed software applications Proc. 4th Int. Workshop on Software and Performance WOSP'04 2004 ACM New York, NY, USA 94 103
61. Shiping Chen, Yan Liu, Ian Gorton, and Anna Liu Performance prediction of component-based applications Journal of Systems and Software 74 1 2005 35 43
62. Yan Liu, Alan Fekete, and Ian Gorton Design-level performance prediction of component-based applications IEEE Transactions on Software Engineering 31 11 2005 928 941 Member-Yan Liu and Member-Alan Fekete and Member-Ian Gorton. (Pubitemid 46395334)
63. Martin Reiser, and Stephen S. Lavenberg Mean-value analysis of closed multichain queuing networks Journal of the ACM 27 2 1980 313 322
64. Liming Zhu, Yan Liu, Ngoc Bao Bui, and Ian Gorton Revel8or: Model driven capacity planning tool suite Proc. 29th Int. Conf. on Software Engineering ICSE'07 2007 IEEE Computer Society Washington, DC, USA 797 800
65. Murali Sitaraman, Greg Kuczycki, Joan Krone, William F. Ogden, A.L.N. Reddy, Performance specification of software components, in: Proc. of SSR'01, 2001
66. Donald E. Knuth The Art of Computer Programming 3rd ed. Fundamental Algorithms. vol. 1 1997 Addison-Wesley Professional
67. Dick Hamlet Tools and experiments supporting a testing-based theory of component composition ACM Transactions on Software Engineering Methodology 1049-331X 18 3 2009 1 41
68. Dick Hamlet, David Mason, and Denise Woit Theory of software reliability based on components Proc. 23rd International Conference on Software Engeneering ICSE'01 2001 IEEE Computer Society Los Alamitos, California, May 1219 361 370
69. Dick Hamlet Software component composition: Subdomain-based testing-theory foundation Journal on Software Testing, Verification and Reliability 17 2007 243 269
70. Sherif M. Yacoub Performance analysis of component-based applications Proceedings of the Second International Conference on Software Product Lines SPLC 2 LNCS vol. 2379 2002 Springer-Verlag London, UK 299 315
71. Adrian Mos, and John Murphy A framework for performance monitoring, modelling and prediction of component oriented distributed systems Proceedings of the 3rd International Workshop on Software and Performance WOSP '02 2002 ACM New York, NY, USA 235 236
72. Marcus Meyerhfer, Frank Lauterwald, Towards platform-independent component measurement, in: Proceedings of the 10th Workshop on Component-Oriented Programming, WCOP2005, 2005
73. Ada Diaconescu, and John Murphy Automating the performance management of component-based enterprise systems through the use of redundancy Proc. 20th Int. Conf. on Automated Software Engineering ASE'05 2005 ACM New York, NY, USA 44 53
74. Trevor Parsons, and John Murphy Detecting performance antipatterns in component based enterprise systems Journal of Object Technology 7 3 2008 55 90
75. Murray Woodside, Vidar Vetland, Marc Courtois, and Stefan Bayarov Resource function capture for performance aspects of software components and sub-systems Performance Engineering, State of the Art and Current Trends 2001 Springer-Verlag London, UK 239 256
76. Tom Verdickt, Bart Dhoedt, Frank Gielen, and Piet Demeester Automatic inclusion of middleware performance attributes into architectural uml software models IEEE Transactions on Software Engineering 31 8 2005 695 771
77. Vincenzo Grassi, Raffaela Mirandola, and Antonino Sabetta A model transformation approach for the early performance and reliability analysis of component-based systems Proc. 9th Int. Symposium on Component-Based Software Engineering CBSE'06 LNCS vol. 4063 2006 Springer 270 284
78. Object Management Group (OMG). Metaobject facility (MOF). http://www.omg.org/mof/ 2006 (last retrieved 13.01.08)
79. Murray Woodside, Dorin Petriu, and Khalid Siddiqui Performance-related completions for software specifications Proceedings of the 24th International Conference on Software Engineering ICSE'02 2002 ACM Press New York, NY, USA 22 32
80. Evgeni Eskenazi, Alexandre Fioukov, and Dieter Hammer Performance prediction for component compositions Proc. 7th International Symposium on Component-based Software Engineering CBSE'04 LNCS vol. 3054 2004 Springer
81. Daniel A. Menasce, Virgilio A.F. Almeida, and Lawrence W. Dowdy Performance by Design 2004 Prentice Hall
82. Ralf H. Reussner, Steffen Becker, and Viktoria Firus Component composition with parametric contracts Tagungsband der Net.ObjectDays 2004 2004 155 169
83. Vittorio Cortellessa How far are we from the definition of a common software performance ontology? Proc. 5th International Workshop on Software and Performance WOSP'05 2005 ACM Press New York, NY, USA 195 204
84. Object Management Group (OMG). MOF QVT final adopted specification (ptc/05-11-01). http://www.omg.org/cgi-bin/apps/doc?ptc/05-11-01.pdf 2006 (last retrieved 13.01.08)
85. Raj Jain The Art of Computer Systems Performance Analysis: Techniques for Experimental Design, Measurement, Simulation, and Modeling 1991 Wiley
86. Jing Xu Rule-based automatic software performance diagnosis and improvement Proceedings of the 7th International Workshop on Software and Performance WOSP'08 2008 ACM New York, NY, USA 1 12
87. Anne Martens, Steffen Becker, Heiko Koziolek, and Ralf Reussner An empirical investigation of the effort of creating reusable models for performance prediction Proc. 11th Int. Symposium on Component-based Software Engineerint CBSE08 LNCS vol. 5282 2008 Springer 16 31
88. Reinhard Wilhelm, Jakob Engblom, Andreas Ermedahl, Niklas Holsti, Stephan Thesing, David Whalley, Guillem Bernat, Christian Ferdinand, Reinhold Heckmann, Tulika Mitra, Frank Mueller, Isabelle Puaut, Peter Puschner, Jan Staschulat, and Per Stenstrm The worst-case execution-time problemoverview of methods and survey of tools Trans. on Embedded Computing Sys. 7 3 2008 1 53
89. Egor Bondarev, Peter de With, Michel Chaudron, Johan Musken, Modelling of input-parameter dependency for performance predictions of component-based embedded systems, in: Proc. of the 31th EUROMICRO Conference, EUROMICRO'05, 2005
90. Heiko Koziolek, Steffen Becker, and Jens Happe Predicting the performance of component-based software architectures with different usage profiles Proc. 3rd International Conference on the Quality of Software Architectures QoSA'07 LNCS vol. 4880 2007 Springer 145 163
91. ej technologies. JProfiler http://www.ej-technologies.com, 2008 (last retrieved 07.01.09)
92. Rich Friedrich, and Jerome Rolia Next generation data centers: Trends and implications Proceedings of the 6th International Workshop on Software and Performance WOSP'07 2007 ACM New York, NY, USA 1 2
93. Vincenzo Grassi, Raffaela Mirandola, Antonino Sabetta, A model-driven approach to performability analysis of dynamically reconfigurable component-based systems, in: Proc. 6th Workshop on Software and Performance, WOSP'07, February 2007
94. Vittorio Cortellessa, and Laurento Frittella A framework for automated generation of architectural feedback from software performance analysis Proc. 4th European Performance Engineering Workshop EPEW'07 LNCS vol. 4748 2007 Springer 171 185
95. Tom Verdickt, Bart Dhoedt, Filip De Turck, and Piet Demeester Hybrid performance modeling approach for network intensive distributed software Proc. 6th International Workshop on Software and Performance WOSP'07 2007 ACM Sigsoft Notes 189 200
96. Dick Hamlet, Dave Mason, and Denise Woit Component-based software development: Case studies Properties of Software Systems Synthesized from Components Series on Component-Based Software Development vol. 1 2004 World Scientific Publishing Company 129 159
97. Walter Binder, Jarle Hulaas, Philippe Moret, and Alex Villazon Platform-independent profiling in a virtual execution environment Software Practice and Experience 39 1 2009 47 79
98. Tao Zheng, Murray Woodside, and Marin Litoiu Performance model estimation and tracking using optimal filters IEEE Transactions on Software Engineering 34 3 2008 391 406
99. Claes Wohlin, Per Runeson, Martin Hst, Magnus C. Ohlsson, Bjorn Regnell, and Anders Wessln Experimentation in Software Engineering: An Introduction 2000 Kluwer Academic Publishers Norwell, MA, USA
100. Andreas Rausch, Ralf Reussner, Raffaela Mirandola, Frantisek Plasil, The Common Component Modeling Example: Comparing Software Component Models LNCS vol. 5153 2008 Springer
101. Jayshankar Sankarasetty, Kevin Mobley, Libby Foster, Tad Hammer, and Terri Calderone Software performance in the real world: Personal lessons from the performance trauma team Proceedings of the 6th international workshop on Software and performance WOSP'07 2007 ACM New York, NY, USA 201 208
102. Swapna S. Gokhale Architecture-based software reliability analysis: Overview and limitations IEEE Transactions on Dependable and Secure Computing 4 1 2007 32 40