Skoll: A process and infrastructure for distributed continuous quality assurance

IEEE Transactions on Software Engineering

Volumn 33, Issue 8, 2007, Pages 510-525

  Porter, Adam ^a   Yilmaz, Cemal ^b   Memon, Atif M ^a   Schmidt, Douglas C ^c   Natarajan, Bala ^c,d  

^a University of Maryland ^*  (United States)

^b IBM T J WATSON RESEARCH CENTER   (United States)

^c VANDERBILT UNIVERSITY   (United States)

^d Symantech India ^*  (India)

Author keywords

Configurable components; Distributed continuous quality assurance; Distributed testing; Testing

Indexed keywords

CONFIGURABLE COMPONENTS; END USER CUSTOMIZATION;

QUALITY ASSURANCE; QUALITY CONTROL; SOFTWARE DESIGN; SOFTWARE PACKAGES; STATISTICAL PROCESS CONTROL;

SOFTWARE ENGINEERING;

EID: 34547137910     PISSN: 00985589     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSE.2007.70719     Document Type: Article

References (37)

1. R. Sangwan, M. Bass, N. Mullick, D.J. Paulish, and J. Kazmeier, Global Software Development Handbook, Auerbach Series on Applied Software Eng., Sept. 2006.
2. M. Haran, A. Karr, A. Orso, A. Porter, and A. Sanil, "Applying Classification Techniques to Remotely-Collected Program Execution Data," Proc. Int'l Symp. Foundations of Software Eng., Sept. 2005.
3. M. Haran, A.F. Karr, M. Last, A. Orso, A. Porter, A. Sanil, and S. Fouche, "Techniques for Classifying Executions of Deployed Software to Support Software Engineering Tasks," IEEE Trans. Software Eng., vol. 33, no. 5, pp. 287-304, May 2007.
4. A.M. Memon, A. Porter, C. Yilmaz, A. Nagarajan, D.C. Schmidt, and B. Natarajan, "Skoll: Distributed Continuous Quality Assurance," Proc. 26th Int'l Conf. Software Eng., May 2004.
5. R.W. Selby and A.A. Porter, "Learning from Examples: Generation and Evaluation of Decision Trees for Software Resource Analysis," IEEE Trans. Software Eng., vol. 14, no. 12, pp. 1743-1757, Dec. 1988.
6. "Mozilla," The Mozilla Organization, www.mozilla.org/, 1998.
7. A. Zeller and R. Hildebrandt, "Simplifying and Isolating Failure-Inducing Input," IEEE Trans. Software Eng., vol. 28, no. 2, pp. 183-200, Feb. 2002.
8. X11-GUITest-0.20, search.cpan.org/ctrondlp/X11-GUITest-0.20, 2005.
9. D. Schmidt and S. Huston, C++ Network Programming: Resolving Complexity with ACE and Patterns. Addison-Wesley, Feb. 2001.
10. D.C. Schmidt, D.L. Levine, and S. Mungee, "The Design and Performance of Real-Time Object Request Brokers," Computer Comm., vol. 21, no. 4, pp. 294-324, Apr. 1998
11. GNU GCC, http://gcc.gnu.org, 2004.
12. Comprehensive PERL Archive Network (CPAN), http://www.cpan.org, 2005.
13. Tinderbox, The Mozilla Organization, http://www.mozilla.org, 2004.
14. "Doc Group Virtual Scoreboard," www.dre.vanderbilt.edu/ scoreboard/, 2005.
15. Bugs, The Mozilla Organization, http://www.mozilla.org/bugs/, 2004.
16. Netscape Quality Feedback System, http://www.netscape.com, 2004.
17. Microsoft XP Error Reporting, http://support.microsoft.com/, 2004.
18. A. Orso, D. Liang, M.J. Harrold, and R. Lipton, "Gamma System: Continuous Evolution of Software After Deployment," Proc. ACM SIGSOFT Int'l Symp. Software Testing and Analysis (ISSTA 2002), pp. 65-69, July 2002.
19. A. Podgurski, D. Leon, P. Francis, W. Masri, M.M. Sun, and B. Wang, "Automated Support for Classifying Software Failure Reports," Proc. 25th Int'l Conf. Software Eng., pp. 465-474, May 2003.
20. J.F. Bowring, J.M. Rehg, and M.J. Harrold, "Active Learning for Automatic Classification of Software Behavior," Proc. Int'l Symp. Software Testing and Analysis, pp. 195-205, July 2004.
21. Y. Brun and M.D. Ernst, "Finding Latent Code Errors via Machine Learning over Program Executions," Proc. 26th Int'l Conf. Software Eng., pp. 480-490, May 2004.
22. B. Liblit, M. Naik, A.X. Zheng, A. Aiken, and M.I. Jordan, "Scalable Statistical Bug Isolation," Proc. Conf. Programming Language Design and Implementation, June 2005.
23. S. Elbaum and M. Diep, "Profiling Deployed Software: Assessing Strategies and Testing Opportunities," IEEE Trans. Software Eng., vol. 31, no. 4, pp. 312-327, 2005.
24. A. Michail and T. Xie, "Helping Users Avoid Bugs in GUI Applications," Proc. 27th Int'l Conf. Software Eng. (ICSE '05), pp. 107-116, 2005.
25. C. Yilmaz, A. Krishna, A.M. Memon, A. Porter, D. Schmidt, A. Gokhale, and B. Natarajan, "Main Effects Screening: A Distributed Continuous Quality Assurance Process for Monitoring Performance Degradation in Evolving Software Systems," Proc. 27th Int'l Conf. Software Eng., 2005.
26. J. van Gurp, J. Bosch, and M. Svahnberg, "On the Notion of Variability in Software Product Lines," Proc. Second Working IEEE/IFIP Conf. Software Architecture (WICSA), pp. 45-54, 2001.
27. D.M. Weiss and C.T. R. Lai, Software Product-Line Engineering: A Family-Based Software Development Process. Addison-Wesley, 1999.
28. S. Buhne, G. Halmans, and K. Pohl, "Modeling Dependencies between Variation Points in Use Case Diagrams," Proc. Ninth Int'l Workshop Requirements Eng. - Foundations for Software Quality, pp. 59-70, June 2003.
29. R. Brownlie, J. Prowse, and M.S. Phadke, "Robust Testing of AT&T PMX/StarMAIL Using OATS," AT&T Technical J., vol. 71, no. 3, pp. 41-7, 1992
30. K. Burr and W. Young, "Combinatorial Test Techniques: Table-Based Automation, Test Generation and Code Coverage," Proc. Int'l Conf. Software Testing Analysis & Rev., 1998.
31. D.M. Cohen, S.R. Dalal, M.L. Fredman, and G.C. Patton, "The AETG System: An Approach to Testing Based on Combinatorial Design," IEEE Trans. Software Eng., vol. 23, no. 7, pp. 437-444, July 1997.
32. S.R. Dalal, A. Jain, N. Karunanithi, J.M. Leaton, C.M. Lott, G.C. Patton, and B.M. Horowitz, "Model-Based Testing in Practice," Proc. Int'l Conf. Software Eng, pp. 285-294, 1999.
33. I.S. Dunietz, W.K. Ehrlich, B.D. Szablak, C.L. Mallows, and A. Iannino, "Applying Design of Experiments to Software Testing," Proc. Int'l Conf. Software Eng., pp. 205-215, 1997.
34. R. Mandl, "Orthogonal Latin Squares: An Application of Experiment Design to Compiler Testing," Comm. ACM, vol. 28, no. 10, pp. 1054-1058, 1985.
35. D. Kuhn and M. Reilly, "An Investigation of the Applicability of Design of Experiments to Software Testing," Proc. 27th Ann. NASA Goddard/IEEE Software Eng. Workshop, pp. 91-95, 2002.
36. C. Yilmaz, M.B. Cohen, and A. Porter, "Covering Arrays for Efficient Fault Characterization in Complex Configuration Spaces, Proc. Int'l Symp. Software Testing and Analysis, pp. 45-54, 2004.
37. A.M. Memon and Q. Xie, "Studying the Fault-Detection Effectiveness of GUI Test Cases for Rapidly Evolving Software," IEEE Trans. Software Eng., vol. 31, no. 10, pp. 884-896, Oct. 2005.