Profiling deployed software: Assessing strategies and testing opportunities

IEEE Transactions on Software Engineering

Volumn 31, Issue 4, 2005, Pages 312-327

  Elbaum, Sebastian ^a   Diep, Madeline ^a  

^a UNIVERSITY OF NEBRASKA LINCOLN   (United States)

Author keywords

Empirical studies; Instrumentation; Profiling; Software deployment; Testing

Indexed keywords

COMPUTER SIMULATION; DATA TRANSFER; PROGRAM DIAGNOSTICS; QUALITY CONTROL; RELIABILITY; TESTING;

EMPIRICAL STUDIES; PROFILING; SOFTWARE DEPLOYMENT;

SOFTWARE ENGINEERING;

EID: 22944444504     PISSN: 00985589     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSE.2005.50     Document Type: Article

References (35)

1. M. Arnold and B. Ryder, "A Framework for Reducing the Cost of Instrumented Code," Proc. Conf. Programming Language Design and Implementation, pp. 168-179, 2001.
2. T. Ball and J. Larus, "Optimally Profiling and Tracing Programs," ACM Trans. Programming Languages and Systems, vol. 16, no. 4, pp. 1319-1360, 1994.
3. T. Ball and J. Laurus, "Optimally Profiling and Tracing Programs," Proc. Ann. Symp. Principles of Programming, pp. 59-70, Aug. 1992.
4. B. Calder, P. Feller, and A. Eustace, "Value Profiling," Proc. Int'l Symp. Microarchitecture, pp. 259-269, Dec. 1997.
5. W. Dickinson, D. Leon, and A. Podgurski, "Finding Failures by Cluster Analysis of Execution Profiles," Proc. Int'l Conf. Software Eng., pp. 339-348, May 2001.
6. S. Elbaum and M. Hardojo, "An Empirical Study of Profiling Strategies for Released Software and Their Impact on Testing Activities," Proc. Int'l Symp. Software Testing and Analysis, pp. 65-75, July 2004.
7. S. Elbaum, S. Kanduri, and A. Andrews, "Anomalies as Precursors of Field Failures," Proc. Int'l Symp. Software Reliability Eng., pp. 108-118, 2003.
8. S. Elbaum, A.G. Malishevsky, and G. Rothermel, "Test Case Prioritization: A Family of Empirical Studies," IE Trans. Software Eng., vol. 28, no. 2, pp. 159-182, Feb. 2002.
9. M. Ernst, J. Cockrell, W. Griswold, and D. Notkin, "Dynamically Discovering Likely Program Invariants to Support Program Evolution," IEEE Trans. Software Eng., vol. 27, no. 2, pp. 99-123, Feb. 2001.
10. A. Glenn, T. Ball, and J. Larus, "Exploiting Hardware Performance Counters with Flow and Context Sensitive Profiling," ACM SIGPLAN Notices, vol. 32, no. 5, pp. 85-96, 1997.
11. S. Graham and M. McKusick, "Gprof: A Call Graph Execution Profiler," ACM SIGPLAN, vol. 17, no. 6, pp. 120-126, June 1982.
12. K. Gross, S. McMaster, A. Porter, A. Urmanov, and L. Votta, "Proactive System Maintenance Using Software Telemetry," Proc. Workshop Remote Analysis and Monitoring Software Systems, pp. 24-26, 2003.
13. MIT Program Analysis Group, The Daikon Invariant Detector User Manual. http://pag.csail.mit.edu/daikon/download/doc/daikon.html, 2005.
14. M. Harrold, R. Lipton, and A. Orso, "Gamma: Continuous Evolution of Software After Deployment," cc.gatech.edu/aristotle/Research /Projects/gamma.html, 2003.
15. D. Hilbert and D. Redmiles, "An Approach to Large-Scale Collection of Application Usage Data over the Internet," Proc. Int'l Conf. Software Eng., pp. 136-145, 1998.
16. D. Hilbert and D. Redmiles, "Separating the Wheat from the Chaff in Internet-Mediated User Feedback," 1998.
17. InCert, "Rapid Failure Recovery to Eliminate Application Downtime," www.incert.com, June 2001.
18. J. Bowring, A. Orso, and M. Harrold, "Monitoring Deployed Software Using Software Tomography," Proc. Workshop Program Analysis for Software Tools and Eng., pp. 2-9, 2002.
19. J. Bowring, J. Rehg, and M.J. Harrold, "Active Learning for Automatic Classification of Software Behavior," Proc. Int'l Symp. Software Testing and Analysis, pp. 195-205, 2004.
20. D. Leon, A. Podgurski, and L. White, "Multivariate Visualization in Observation-Based Testing," Proc. Int'l Conf. Software Eng., pp. 116-125, May 2000.
21. D. Libes, Exploring Expect: A Tcl-Based Toolkit for Automating Interactive Programs. Sebastopol, Calif.: O'Reilly and Associates, Inc., Nov. 1996.
22. B. Liblit, A. Aiken, Z. Zheng, and M. Jordan, "Bug Isolation via Remote Program Sampling," Proc. Conf. Programming Language Design and Implementation, pp. 141-154, June 2003.
23. A. Memon, A. Porter, C. Yilmaz, A. Nagarajan, D.C. Schmidt, and B. Natarajan, "Skoll: Distributed Continuous Quality Assurance," Proc. Int'l Conf. Software Eng., pp. 449-458, May 2004.
24. J. Musa, Software Reliability Engineering. McGraw-Hill, 1999.
25. Netscape, "Netscape Quality Feedback System,"home.netscape. com/communicator/navigator/v4.5/qfs1.html, 2000.
26. Nielsen, "Nielsen Net Ratings: Nearly 40 Million Internet Users Connect Via Broadband," www.nielsen-netratings.com, 2003.
27. University of Washington, Pine Information Center, http://www.washington. edu/pine/, 2005.
28. A. Orso, T. Apiwattanapong, and M.J. Harrold, "Leveraging Field Data for Impact Analysis and Regression Testing," Foundations of Software Eng., pp. 128-137, Sept. 2003.
29. A. Orso, D. Liang, M. Harrold, and R. Lipton, "Gamma System: Continuous Evolution of Software after Deployment," Proc. Int'l Symp. Software Testing and Analysis, pp. 65-69, 2002.
30. C. Pavlopoulou and M. Young, "Residual Test Coverage Monitoring," Proc. Int'l Conf. Software Eng., pp. 277-284, May 1999.
31. S. Reiss and M. Renieris, "Encoding Program Executions," Proc. Int'l Conf. Software Eng., pp. 221-230, May 2001.
32. D. Richardson, L. Clarke, L. Osterweil, and M. Young, "Perpetual Testing Project," http://www.ics.uci.edu/~djr/edcs/PerpTest. html, 1997.
33. A. van der Hoek, R. Hall, D. Heimbigner, and A. Wolf, "Software Release Management," Proc. European Software Eng. Conf., M. Jazayeri and H. Schauer, eds., pp. 159-175, 1997.
34. C. Yilmaz, M.B. Cohen, and A. Porter, "Covering Arrays for Efficient Fault Characterization in Complex Configuration Spaces," Proc. Int'l Symp. Software Testing and Analysis, pp. 45-54, July 2004.
35. C. Yilmaz, A. Porter, and A. Schmidt, "Distributed Continuous Quality Assurance: The Skoll Project," Proc. Workshop Remote Analysis and Monitoring Software Systems, pp. 16-19, 2003.