Introduction to software performance engineering: Origins and outstanding problems

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 4486 LNCS, Issue , 2007, Pages 395-428

  Smith, Connie U ^a  

^a Performance Engineering Services   (United States)

Author keywords

Performance antipattterns; Performance models; Performance patterns; Performance prediction; Queueing network models; Software performance engineering; SPE; SPE process

Indexed keywords

BIBLIOGRAPHIC RETRIEVAL SYSTEMS; COMPUTER SIMULATION; DATA ACQUISITION; DATA REDUCTION; PROBLEM SOLVING;

CASE STUDY; OUTSTANDING PROBLEMS; SOFTWARE PERFORMANCE ENGINEERING (SPE);

SOFTWARE ENGINEERING;

EID: 34548101755     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/978-3-540-72522-0_10     Document Type: Conference Paper

References (141)

1. C.T. Alexander. Performance Engineering: Various Techniques and Tools. in Proceedings Computer Measurement Group Conference. 1986. Las Vegas, NV.
2. W. Alexander and R. Brice. Performance Modeling in the Design Process. in Proceedings National Computer Conference. 1982. Houston, TX.
3. S. Alter, A Model for Automating File and Program Design in Business Application Systems. Communications of the ACM, 1979. 22(6): p. 345-353.
4. G.E. Anderson, The Coordinated Use of Five Performance Evaluation Methodologies. Communications of the ACM, 1984. 27(2): p. 119-125.
5. R.L. Bagrodia and C. Shen, MIDAS: Integrated Design and Simulation of Distributed Systems. IEEE Transactions on Software Engineering, 1991. 17(10): p. 1042-58.
6. S. Balsamo and M. Marzolla. Performance Evaluation of UML Software Architectures with Multiclass Queueing Network Models. in WOSP 2005. 2005. Palma de Mallorca: ACM.
7. F. Baskett, et al., Open, Closed, and Mixed Networks of Queues with Different Classes of Customers. Journal of the ACM, 1975. 22(2): p. 248-260.
8. H. Beilner, J. Mater, and N. Weissenburg. Towards a Performance Modeling Environment: News on HIT. in Proceedings 4th International Conference on Modeling Techniques and Tools for Computer Performance Evaluation. 1988: Plenum Publishing.
9. H. Beilner, J. Mäter, and C. Wysocki, The Hierarchical Evaluation Tool HIT, in Performance Tools & Model Interchange Formats, F. Bause and H. Beilner, Editors. 1995, Universität Dortmund, Fachbereich Informatik: D-44221 Dortmund, Germany. p. 6-9.
10. B. Beizer, Micro-Analysis of Computer System Performance. 1978, New York, NY: Van Nostrand Reinhold.
11. B. Beizer, Software Performance, in Handbook of Software Engineering, C.R. Vicksa and C.V. Ramamoorthy, Editors. 1984, Van Nostrand Reinhold: New York, NY. p. 413-436.
12. T.E. Bell, ed. Special Issue on Software Performance Engineering. Computer Measurement Group Transactions. 1988.
13. T.E. Bell, D.X. Bixler, and M.E. Dyer, An Extendible Approach to Computer-aided Software Requirements Engineering. IEEE Transactions on Software Engineering, 1977. 3(1): p. 49-59.
14. T.E. Bell and A.M. Falk. Performance Engineering: Some Lessons From the Trenches. in Proceedings Computer Measurement Group Conference. 1987. Orlando, FL.
15. J.L. Bentley, Writing Efficient Programs. 1982, Englewood Cliffs, NJ: Prentice-Hall.
16. S. Bernardi, S. Donatelli, and J. Merseguer. From UML Sequence Diagrams and Statecharts to Analysable Petri Net Models, in Proc. Third Int. Workshop on Software and Performance (WOSP02). 2002. Rome: ACM Press.
17. M. Bernardo, P. Ciancarini, and L. Donatelli. AEMPA: A Process Algebraic Description Language for the Performance Analysis of Software Architectures., in Proc. Second Int. Workshop on Software and Performance (WOSP00). 2000. Ottawa: ACM Press.
18. BMC, BMC Software, in 2101 City West Blvd., (713) 918-8800, www.bmc.com: Houston, TX 77042.
19. T.L. Booth, Performance Optimization of Software Systems Processing Information Sequences Modeled by Probabilistic Languages. IEEE Transactions on Software Engineering, 1979. 5(1): p. 31-44.
20. T.L. Booth. Use of Computation Structure Models to Measure Computation Performance, in Proceedings Conference on Simulation, Measurement, and Modeling of Computer Systems. 1979. Boulder, CO.
21. T.L. Booth, R.O. Hart, and B. Qin. High Performance Software Design. in Proceedings Hawaii International Conference on System Sciences. 1986. Honolulu, HI.
22. T.L. Booth and C.A. Wiecek, Performance Abstract Data Types as a Tool in Software Perfromance Analysis and Design. IEEE Transactions on Software Engineering, 1980. 6(2): p. 138-151.
23. G. Brataas, Performance Engineering Method for Workflow Systems: An Integrated View of Human and Computerised Work Processes, in Physics, Informatics, and Mathematics. 1996, Norwegian University of Science and Technology: Trondheim, Norway, p. 237.
24. R.J. Buhr, et al., Software CAD: A Revolutionary Approach. IEEE Transactions on Software Engineering, 1989. 15(3): p. 234-249.
25. D. Bulka and D. Mayhew, Efficient C++: Performance Programming Techniques. 2000: Addison Wesley Longman.
26. J.P. Buzen, Queueing Network Models of Multiprograming. 1971, Harvard University.
27. C.K. Chang, et al., Modeling a Real-Time Multitasking System in a Timed PQ Net. IEEE Transactions on Software Engineering, 1989. 6(2): p. 46-51.
28. CMG, Computer Measurement Group, in PO Box 1124, (800) 436-7264, www.cmg.org: Turnersville, NJ 08012.
29. P. Crain and C. Hanson. Web Application Tuning. in CMG. 1999. Reno.
30. A. D'Ambrogio. A Model Transformation Framework for the Automated Building of Performance Models from UML Models, in Proc. 2005 Workshop on Software and Performance. 2005. Palma de Mallorca: ACM Press.
31. J. Dilly, et al. Measurement Tools and Modeling Techniques for Evaluating Web Server Performance. in Proc. 9th Int. Conf. on Modelling Techniques and Tools. 1997. St. Malo, France: Springer-Verlag.
32. H. El-Sayed, D. Cameron, and C.M. Woodside. Automated Performance Modeling from Scenarios and SDL Designs of Telecom Systems. in Proc. Int. Symposium on Software Engineering for Parallel and Distributed Systems (PDSE98). 1998. Kyoto, Japan.
33. G. Estrin, et al., SARA (System ARchitects' Apprentice): Modeling, Analysis, and Simulation Support for Design of Concurrent Systems. IEEE Transactions on Software Engineering, 1986. SE-12(2): p. 293-311.
34. D. Ferrari, Workload Characterization and Selection in Computer Performance Measurement. IEEE Computer, 1972. 5(4): p. 18-24.
35. D. Ferrari, Computer Systems Performance Evaluation. 1978, Englewood Cliffs, NJ: Prentice-Hall.
36. D. Ferrari, G. Serazzi, and A. Zeigner, Measurement and Tuning of Computer Systems. 1983, Englewood Cliffs, NJ: Prentice-Hall.
37. G. Fox. Take Practical Performance Engineering Steps Early, in Proceedings Computer Measurement Group Conference. 1987. Orlando, FL.
38. G. Fox. Performance Engineering as a Part of the Development Lifecycle for Large-Scale Software Systems. in Proceedings 11th International Conference on Software Engineering. 1989. Pittsburgh, PA.
39. D. Garcia, et al. Performance Model Interchange Format: Semantic Validation. in 2006 International Conference on Software Engineering Advances. 2006. Papeete Tahiti: INIRIA.
40. H. Gomaa and D.A. Menasce, eds. Session: From Specifications to Performance Models. Second Int. Workshop on Software and Performance (WOSPOO). 2000, ACM Press: Ottawa. 47-88.
41. R.M. Graham, G.J. Clancy, and D.B. DeVaney, A Software Design and Evalation System. Communications of the ACM, 1973. 16(2): p. 110-116.
42. G. Gu and D. Petriu. XSLT Transformation from UML Models to LQN Performance Models. in Proc. Workshop on Software and Performance. 2002. Rome: ACM.
43. G. Gu and D. Petriu. From UML to LQN by XML Algebra-based Model Transformations. in WOSP 2005. 2005. Palma de Mallorca: ACM.
44. N. Gunther, The Practical Performance Analyst: Performance-By-Design Techniques for Distributed Systems. 1998: www.perfdynamics.com.
45. N. Gunther. E-Ticket Capacity Planning: Riding the E-Commerce Growth Curve. in Proc. Computer Measurement Group. 2000. Orlando.
46. C Hanson, P. Crain, and S. Wigginton. User and Computer Performance Optimization. in CMG. 1999. Reno.
47. G. Haring, On State-dependent Workload Characterization by Software Resources. ACM SIGMETRICS Performance Evaluation Review, 1982. 11(4): p. 51-57.
48. G. Hills, J.A. Rolia, and G. Serazzi. Performance Engineering of Distributed Software Process Architectures. in Modelling Techniques and Tools for Computer Performance Evaluation. 1995. Heidelberg, Germany: Springer.
49. R.P. Hopkins, et al. Two Approaches to Integrating UML and Performance Models, in Proc. Third Int. Workshop on Software and Performance (WOSP02). 2002. Rome: ACM Press.
50. N.R. Howes. Toward a Real-Time Ada Design Methodology, in Proceedings Tri-Ada 90. 1990. Baltimore, MD.
51. C. Hrischuk, J.A. Rolia, and C.M. Woodside. Automatic Generation of a Software Performance Model Using an Object-Oriented Prototype, in Proceedings of the Third International Workshop on Modeling, Analysis, and Simulation of Computer and Telecommunications Systems. 1995. Durham, NC.
52. C.E. Hrischuk, et al., Trace-Based Load Characterization for Generating Performance Software Models. TSE, 1999. 25(1): p. 122-135.
53. P. Hughes. Towards a Common Process Model for Systems Development and Performance Engineering. in Proc. Second Int. Workshop on Software and Performance (WOSP00). 2000. Ottawa, Canada: ACM Press.
54. HyPerformix, Inc., in 4301 West Bank Dr, Bldg A, (512)328-5544, www.hyperformix.com: Austin, TX 78746.
55. R. Jain, Art of Computer Systems Performance Analysis. 1991, New York, NY: John Wiley.
56. P.J. Jalics, Improving Performance The Easy Way. Datamation, 1977. 23(4): p. 135-148.
57. E. Kant, Efficiency in Program Synthesis. 1981, Ann Arbor, MI: UMI Research Press.
58. P. King and R. Pooley. Derivation of Petri Net Performance Models from UML Specifications of Communications Software, in Modelling Tools and Techniques. 2000. Schaumburg, IL: Springer.
59. D.E. Knuth, An Empirical Study of FORTRAN Programs. Software Practice & Experience, 1971. 1(2): p. 105-133.
60. D.E. Knuth, The Art of Computer Programming, Vol.1: Fundamental Algorithms, Third Edition. 1997, Reading, MA: Addison-Wesley.
61. D.E. Knuth, The Art of Computer Programming, Vol.3: Sorting and Searching, Second Edition. 1998, Reading, MA: Addison-Wesley.
62. H. Kopetz. Design Principles of Fault Tolerant Real-Time Systems. in Proceedings Hawaii International Conference on System Sciences. 1986. Honolulu, HI.
63. L&S, Computer Technology, Inc., Performance Engineering Services Division, in # 110, PO Box 9802, (505) 988-3811, www.spe-ed.com: Austin, TX 78766.
64. B.W. Lampson, Hints for Computer System Design. IEEE Software, 1984. 2(1): p. 11-28.
65. C. Larman and R. Guthrie, Java 2 Performance and Idiom Guide. 2000, Upper Saddle River, NJ: Prentice Hall PTR.
66. E.D. Lazowska, et al., Quantitative System Performance: Computer System Analysis Using Queuing Network Models. 1984, Englewod Cliffs, NJ: Prentice-Hall, Inc.
67. E. LeMer. MEDOC: A Methodology for Designing and Evaluating Large-Scale Real-Time Systems, in Proceedings National Computer Conference, 1982. 1982. Houston, TX.
68. S. Levi and A.K. Agrawala, Real-Time System Design. 1990, New York, NY: McGraw-Hill.
69. K. Lor and D.M. Berry, Automatic Synthesis of SARA Design Models from System Requirements. IEEE Transactions on Software Engineering, 1991. 17(12): p. 1229-1240.
70. C.R. Martin, An Integrated Software Performance Engineering Environment. 1988, Duke University.
71. D.A. Menascé and V.A.F. Almeida, Scaling for E-Business: Technologies, Models, Performance, and Capacity Planning. 2000: Prentice Hall.
72. D.A. Menascé, V.A.F. Almeida, and L.W. Dowdy, Capacity Planning and Performance Modeling. 1994, Englewood Cliffs, NJ: PTR Prentice Hall. 412.
73. D.A. Menascé and H. Gomaa. On a Language Based Method for Software Performance Engineering of Client/Server Systems. in Workshop on Software and Performance. 1998. Santa Fe, NM: ACM.
74. K.M. Nichols, Performance Tools. IEEE Software, 1990. 7(3): p. 21-30.
75. K.M. Nichols and P. Oman, Special Issue in High Performance. IEEE Software, 1991. 8(5).
76. OMG, UML Profile for Schedulability, Performance and Time, formal/03-09-01. 2003, OMG Full Specification.
77. A. Opdahl. A CASE Tool for Performance Engineering During Software Design. in Proceedings Fifth Nordic Workshop on Programming Environmental Research. 1992. Tampere, Finland.
78. A. Opdahl and A. Sølvberg. Conceptual Integration of Information System and Performance Modeling. in Proceedings Working Conference on Information System Concepts: Improving the Understanding. 1992.
79. M. Paterok, R. Heller, and H. deMeer. Performance Evaluation of an SDL Run Time System - A Case Study, in Proceedings 5th International Conference on Modeling Techniques and Tools for Computer Performance Evaluation. 1991. Torino, Italy.
80. D. Petriu and C.M. Woodside. Software Performance Models from System Scenarios in Use Case Maps. in Proceedings 12th Int. Conf. Modeling Tools and Techniques. 2002. London.
81. R. Pooley. The Integrated Modeling Support Environment. in Proceedings 5th International Conference on Modeling Techniques and Tools for Computer Performance Evaluation. 1991. Torino, Italy.
82. QEST, Quantitative Evaluation of Systems. 2004++, www.qest.org.
83. B. Qin, A Model to Predict the Average Response Time of User Programs. Performance Evaluation, 1989. 10: p. 93-101.
84. QSP, Quantitative System Performance, in 7576 34th Ave N., A.M.aAP. QSP, Editor: Seattle, WA 98117-4723.
85. W.E. Riddle, et al., Behavior Modeling During Software Design. IEEE Transactions on Software Engineering, 1978. 4.
86. J.A. Rolia, Predicting the Performance of Software Systems. 1992, University of Toronto.
87. J.A. Rolia and K.C. Sevcik, The Method of Layers. IEEE Trans. on Software Engineering, 1995. 21(8): p. 689-700.
88. J.A. Rolia and V. Vetland. Correlating Resource Demand Information with ARM Data for Application Services. in Int. Workshop on Software and Performance. 1998. Santa Fe, NM: ACM.
89. J. Rosselló, et al. A Web Service for Solving Queueing Network Models Using PMIF. in Proc. Workshop on Software and Performance. 2005. Palma de Mallorca: ACM Press.
90. R.A. Sahner and K.S. Trivedi, Performance and Reliability Analysis Using Directed Acyclic Graphs. IEEE Transactions on Software Engineering, 1987. 13(10): p. 1105-1114.
91. J.W. Sanguinetti. A Formal Technique for Analyzing the Performance of Complex Systems. in Proceedings Performance Evaluation Users Group 14. 1978. Boston, MA.
92. A. Schmietendorf, A. Scholz, and C Rautenstrauch. Evaluating the Performance Engineering Process, in Proc. Workshop on Software and Performance (WOSP2000). 2000. Ottawa, Canada.
93. T. Schneider, Performance Management of SAP Solutions, in Performance Engineering: State of the Art and Current Trends, Dumke, et al., Editors. 2001, Springer Verlag LNCS 2047.
94. K.C. Sevcik, C.U. Smith, and L.G. Williams, Performance Prediction Techniques Applied to Electronic Commerce Systems, in Electronic Commerce Technology Trends: Challenges and Opportunities, W. Kou and Y. Yesha, Editors. 2000, IBM Press.
95. F. Sheikh and C.M. Woodside, Layered Analytic Performance Modelling of a Distributed Database System. ICDCS, 1997.
96. H. Sholl and S. Kim. An Approach to Performance Modeling as an Aid in Structuring Real-time, Distributed System Software. in Proceedings Hawaii International Conference on System Sciences. 1986. Honolulu, HI.
97. H.A. Sholl and T.L. Booth, Software Performance Modeling Using Computation Structures. IEEE Transactions on Software Engineering, 1975. 1(4).
98. C.U. Smith, The Prediction and Evaluation of the Performance of Software from Extended Design Specifications. 1980, University of Texas.
99. C.U. Smith. Software Performance Engineering, in Proceedings Computer Measurement Group Conference XII. 1981.
100. C.U. Smith. A Methodology for Predicting the Memory Management Overhead of New Software Systems. in Proceedings Hawaii International Conference on System Sciences. 1982. Honolulu, HI.
101. C.U. Smith, Special Issue on Software Performance Engineering, in Computer Measurement Group Transactions. 1985.
102. C.U. Smith, Independent General Principles for Constructing Responsive Software Systems. ACM Transactions on Computer Systems, 1986. 4(1): p. 1-31.
103. C.U. Smith, Applying Synthesis Principles to Create Responsive Software Systems. IEEE Transactions on Software Engineering, 1988. 14(10): p. 1394-1408.
104. C.U. Smith, Who Uses SPE? Computer Measurement Group Transactions, 1988: p. 69-75.
105. C.U. Smith, Performance Engineering of Software Systems. 1990, Reading, MA: Addison-Wesley.
106. C.U. Smith and J.C. Browne. Performance Specifications and Analysis of Software Designs, in Proceedings ACM Sigmetrics Conference on Simulation Measurement and Modeling of Computer Systems. 1979. Boulder, CO.
107. C.U. Smith and J.C. Browne, Aspects of Software Design Analysis: Concurrency and Blocking. Proceedings ACM Sigmetrics Conference on Simulation Measurement and Modeling of Computer Systems, 1980.
108. C.U. Smith and J.C. Browne. Performance Engineering of Software Systems: A Case Study. in Proceedings National Computer Conference. 1982. Houston, TX.
109. C.U. Smith, et al. From UML models to software performance results: An SPE process based on XML interchange formats. in Proc. 5th Int. Workshop on Software and Performance. 2005. Palma, Illes Baleare, Spain: ACM Press.
110. C.U. Smith and C.M. Lladó. Performance Model Interchange Format (PMIF 2.0): XML Definition and Implementation, in Proc. 1st Int. Conf. on Quantitative Evaluation of Systems (QEST). 2004. Enschede, NL: IEEE Computer Society.
111. C.U. Smith and D.D. Loendorf. Performance Analysis of Software for an MIMD Computer, in Proceedings ACM Sigmetrics Conference on Measurement and Modeling of Computer Systems. 1982. Seattle, WA.
112. C.U. Smith and L.G. Williams, Software Performance Engineering: A Case Study with Design Comparisons. IEEE Transactions on Software Engineering, 1993. 19(7): p. 720-741.
113. C.U. Smith and L.G. Williams, A Performance Model Interchange Format, in Performance Tools and Model Interchange Formats, F. Bause and H. Beilner, Editors. 1995, Universität Dortmund, Informatik IV: D-44221 Dortmund, Germany, p. 67-85.
114. C.U. Smith and L.G. Williams, Performance Engineering of Object-Oriented Systems with SPEED, in Lecture Notes in Computer Science 1245: Computer Performance Evaluation, M.R.e. al., Editor. 1997, Springer: Berlin, Germany, p. 135-154.
115. C.U. Smith and L.G. Williams, Performance Engineering Evaluation of CORBA-based Distributed Systems with SPEED, in Lecture Notes in Computer Science, R. Puigjaner, Editor. 1998, Springer: Berlin, Germany.
116. C.U. Smith and L.G. Williams. Performance Engineering Models of CORBA-based Distributed Object Systems. in Proc. Computer Measurement Group. 1998. Anaheim.
117. C.U. Smith and L.G. Williams. Performance Models of Distributed System Architectures., in Proc. Computer Measurement Group. 1998. Anaheim.
118. C.U. Smith and L.G. Williams, A Performance Model Interchange Format. Journal of Systems and Software, 1999. 49(1).
119. C.U. Smith and L.G. Williams. Software Performance Antipatterns. in Workshop on Software and Performance. 2000. Ottawa, Canada: ACM Press.
120. C.U. Smith and L.G. Williams. SPE Models for Web Applications. in Proc. Computer Measurement Group. 2000. Orlando.
121. C.U. Smith and L.G. Williams. New Software Performance Antipatterns: More Ways to Shoot Yourself in the Foot. in Proc. CMG. 2002. Reno, NV.
122. C.U. Smith and L.G. Williams, Performance Solutions: A Practical Guide to Creating Responsive, Scalable Software. 2002: Addison-Wesley.
123. C.U. Smith and L.G. Williams. Best Practices for Software Performance Engineering. in Proc. CMG. 2003. Dallas, TX.
124. C.U. Smith and L.G. Williams. More New Software Performance Antipatterns: Even More Ways to Shoot Yourself in the Foot, in Proc. CMG. 2003. Dallas, TX.
125. C.U. Smith and B. Wong. SPE Evaluation of a Client/Server Application, in Proc. Computer Measurement Group. 1994. Orlando, FL.
126. C.U. Smith and C.M. Woodside, Performance Validation at Early Stages of Software Development, in System Performance Evaluation: Methodologies and Applications, E. Gelenbe, Editor. 1999, CRC Press.
127. SPE·ED, http://www.spe-ed.com.
128. M. Steppler. Performance Analysis of Communication Systems Formally Specified in SDL. in Proc. 1st Int. Workshop on Software and Performance (WOSP98). 1998. Santa Fe, NM.
129. K.S. Trivedi, Probability and Statistics With Reliability, Queueing, and Computer Science Applications. 1982, Englewood Cliffs, NJ: Prentice-Hall.
130. V. Vetland, Measurement-Based Composite Computational Work Modelling of Software. 1993, University of Trondheim.
131. V. Vetland, P. Hughes, and A. Solvberg. A Composite Modelling Approach to Software Performance Measurement. in Proc. ACM Sigmetrics. 1993. Santa Clara, CA.
132. L.G. Williams and C.U. Smith. Information Requirements for Software Performance Engineering, in Proceedings 1995 International Conference on Modeling Techniques and Tools for Computer Performance Evaluation. 1995. Heidelberg, Germany: Springer.
133. SM: A Method for the Performance Assessment of Software Architectures, in Proc. 3rd Int. Workshop on Software and Performance. 2002. Rome, IT: ACM Press.
134. L.G. Williams and C.U. Smith. Making the Business Case for Software Performance Engineering, in Proc. CMG. 2003. Dallas, TX.
135. C.M. Woodside, Throughput Calculation for Basic Stochastic Rendezvous Networks. 1986, Carleton University, Ottawa, Canada.
136. C.M. Woodside, Throughput Calculation for Basic Stochastic Rendezvous Networks. Performance Evaluation, 1989. 9.
137. C.M. Woodside, et al., The CAEDE Performance Analysis Tool. Ada Letters, 1991. XI(3).
138. C.M. Woodside, et al. A Wideband Approach to Integrating Performance Prediction into a Software Design Environment. in Proc. First Int. Workshop on Software and Performance (WOSP98). 1998. Santa Fe, NM.
139. C.M. Woodside, et al., The Stochastic Rendezvous Network Model for Performance of Synchronous Client-Server-like Distributed Software. IEEE Trans. Computers, 1995. 44(1): p. 20-34.
140. WOSP, Workshop on Software and Performance. 1998-2007, ACM Press.
141. X. Wu and C.M. Woodside. Performance Modeling from Software Components. in Proc. Workshop on Software and Performance. 2004. Redwood Shores, CA: ACM 488043.