Interactive fault localization techniques in a spreadsheet environment

IEEE Transactions on Software Engineering

Volumn 32, Issue 4, 2006, Pages 213-239

  Ruthruff, Joseph R ^a   Burnett, Margaret ^b   Rothermel, Gregg ^a  

^a UNIVERSITY OF NEBRASKA LINCOLN   (United States)

^b Oregon State University   (United States)

Author keywords

Debugging; End user programming; End user software engineering; Fault localization; Spreadsheets

Indexed keywords

BY-DEMONSTRATION MACRO BUILDER; ELECTRONIC MAIL FILTERING EDITOR; END-USER PROGRAMMER; INTERACTIVE FAULT LOCALIZATION; MULTIMEDIA SIMULATION BUILDER; SOFTWARE AUTHORING DEVICE;

ELECTRONIC MAIL; INTERACTIVE COMPUTER SYSTEMS; MULTIMEDIA SYSTEMS; PERSONNEL; PROGRAM DEBUGGING; SPREADSHEETS; USER INTERFACES;

COMPUTER AIDED SOFTWARE ENGINEERING;

EID: 33646514834     PISSN: 00985589     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSE.2006.37     Document Type: Article

References (68)

1. R. Abraham and M. Erwig, "Header and Unit inference for Spreadsheets through Spatial Analyses," Proc. IEEE Int'l Symp. Visual Languages and Human-Centric Computing, pp. 165-172, Sept. 2004.
2. H. Agrawal and J.R. Horgan, "Dynamic Program Slicing," Proc. ACM SIGPLAN 1990 Conf. Programming Language Design and Implementation, pp. 246-256, June 1990.
3. H. Agrawal, J.R. Horgan, S. London, and W.E. Wong, "Fault Localization Using Execution Slices and Dataflow Tests," Proc. Sixth IEEE Int'l Symp. Software Reliability Eng., pp. 143-151, Oct. 1995.
4. Y. Ahmad, T. Antoniu, S. Goldwater, and S. Krishnamurthi, "A Type System for Statically Detecting Spreadsheet Errors,Proc. 18th IEEE Int'l Conf. Automated Software Eng., pp 174-183, Oct. 2003.
5. C.M. Allwood, "Error Detection Processes in Statistical Problem Solving," Cognitive Science, vol. 8 no. 4, pp. 413-437, Oct.-Dec. 1984.
6. T. Antoniu, P.A. Steckler, S. Krishnamurthi, E. Neuwirth, and M. Felleisen, "Validating the Unit Correctness of Spreadsheet Programs," Proc. 26th Int'l Conf. Software Eng., pp. 439-448, May 2004.
7. Y. Ayalew and R. Mittermeir, "Spreadsheet Debugging," Proc. European Spreadsheet Risks Interest Group, July 2003.
8. M. Betts and A.S. Horowitz, "Oops! Audits Find Errors in 49 Out of 54 Spreadsheets," Computerworld, p. 47, May 2004.
9. B. Boehm and V.R. Basili, "Software Defect Reduction Top 10 List," Computer, vol. 34, no. 1, pp. 135-137, Jan. 2001.
10. D. Brown, M. Burnett, G. Rothermel, H. Fujita, and F. Negoro, "Generalizing WYSIWYT Visual Testing to Screen Transition Languages," Proc. IEEE Symp. Human-Centric Computing, Languages, and Environments, pp. 203-210, Oct. 2003.
11. P. Bunus and P. Fritzson, "Semi-Automatic Fault Localization and Behavior Verification for Physical System Simulation Models," Proc. 18th IEEE Int'l Conf. Automated Software Eng., pp. 251-258 Oct. 2003.
12. M. Burnett, J. Atwood, R. Djang, H. Gottfried, J. Reichwein, and S. Yang, "Forms/3: A First-Order Visual Language to Explore the Boundaries of the Spreadsheet Paradigm," J. Functional Programming, vol. 11, no. 2, pp. 155-206, Mar. 2001.
13. M. Burnett, C. Cook, O. Pendse, G. Rothermel, J. Summet, and C. Wallace, "End-User Software Engineering with Assertions in the Spreadsheet Paradigm," Proc. 25th Int'l Conf. Software Eng., pp. 93-103, May 2003.
14. M. Burnett, C. Cook, and G. Rothermel, "End-User Software Engineering," Comm. ACM, vol. 47, no. 9, pp. 53-58, Sept. 2004.
15. M. Burnett, A. Sheretov, and G. Rothermel, "Scaling Up a 'What You See is What You Test' Methodology to Spreadsheet Grids," Proc. IEEE Symp Visual Languages, pp. 30-37, Sept. 1999.
16. T.Y. Chen and Y.Y. Cheung, "On Program Dicing," Software Maintenance: Research and Practice, vol. 9, no. 1, pp. 33-46, Jan.-Feb. 1997.
17. M. Clermont, "Analyzing Large Spreadsheet Programs," Proc. 10th Working Conf. Reverse Eng., pp. 306-315, Nov. 2003.
18. M. Clermont and R. Mittermeir, "Auditing Large Spreadsheet Programs" Proc. Int'l Conf. Information Systems Implementation and Modelling, pp. 87-97, Apr. 2003.
19. H. Cleve and A. Zeller, "Locating Causes of Program Failures," Proc. 27th Int'l Conf. Software Eng., pp. 342-351, May 2005.
20. C. Cook, M. Burnett, and D. Boom, "A Bug's Eye View of Immediate Visual Feedback in Direct-Manipulation Programming Systems," Proc. Empirical Studies of Programmers: Seventh Workshop, pp. 20-41, Oct. 1997.
21. P.T. Cox, F.R. Giles, and T. Pietrzykowski, "Prograph: A Step towards Liberating Programming from Textual Conditioning," Proc. IEEE Workshop Visual Languages, pp. 150-156, Oct. 1989.
22. R.A. DeMillo, H. Pan, and E.H. Spafford, "Critical Slicing for Software Fault Localization," Proc. Int'l Symp. Software Testing and Analysis, pp. 121-134, Jan. 1996.
23. E. Duesterwald, R. Gupta, and M.L. Soffa, "Rigorous Data Flow Testing through Output Influences," Proc. Second Irvine Software Symp., pp. 131-145, Mar. 1992.
24. M. Erwig and M. Burnett, "Adding Apples and Oranges," Proc. Fourth Int'l Symp. Practical Aspects of Declarative Languages, pp. 173-191, Jan. 2002.
25. M. Fisher II, D. Jin, G. Rothermel, and M. Burnett, "Test Reuse in the Spreadsheet Paradigm," Proc. IEEE Int'l Symp. Software Reliability Eng., pp. 257-268, Nov. 2002.
26. M. Fisher II and G. Rothermel, "The EUSES Spreadsheet Corpus: A Shared Resource for Supporting Experimentation with Spreadsheet Dependability Mechanisms," Proc. First Workshop End-User Software Eng, pp. 47-51, May 2005.
27. M. Fisher II, G. Rothermel, T. Creelan, and M. Burnett, "Scaling a Dataflow Testing Methodology to the Multiparadigm World of Commercial Spreadsheets," Technical Report TR-UNL-CSE-2005-0003, Univ. of Nebraska-Lincoln, Sept. 2005.
28. M. Francel and S. Rugaber, "Fault Localization Using Execution Traces," Proc. ACM 30th, Ann. Southeast Regional Conf., pp. 69-76, Apr. 1992.
29. D.S. Hilzenrath, "Finding Errors a Plus, Fannie Says; Mortgage Giant Tries to Soften Effect of $1 Billion in Mistakes," The Washington Post, 31 Oct. 2003.
30. T. Igarashi, J.D. Mackinlay, B.W. Chang, and P.T. Zellweger, "Fluid Visualization of Spreadsheet Structures," Proc. IEEE Symp. Visual Languages, pp. 118-125, Sept. 1998.
31. J.A. Jones, M.J. Harrold, and J. Stasko, "Visualization of Test Information to Assist Fault Localization," Proc. 24th Int'l Conf. Software Eng., pp. 467-477, May 2002.
32. M. Karam and T. Smedley, "A Testing Methodology for a Dataflow Based Visual Programming Language," Proc. IEEE Symp. Human-Centric Computing, Languages, and Environments, pp. 280-287, Sept. 2001.
33. A.J. Ko and B.A. Myers, "Development and Evaluation of a Model of Programming Errors," Proc. IEEE Symp. Human-Centric Computing, Languages, and Environments, pp. 7-14, Oct. 2003.
34. A.J. Ko and B.A. Myers, "Designing the Whyline: A Debugging Interface for Asking Questions about Program Failures," Proc. ACM Conf. Human Factors in Computing Systems, pp. 151-158. Apr. 2004.
35. B. Korel and J. Laski, "Dynamic Slicing of Computer Programs," J. Systems and Software, vol. 13, no. 3, pp. 187-195, Nov. 1990.
36. V. Krishna, C. Cook, D. Keller, J. Cantrell, C. Wallace, M. Burnett, and G. Rothermel, "Incorporating Incremental Validation and Impact Analysis into Spreadsheet Maintenance: An Impirical Study," Proc. IEEE Int'l Cont. Software Maintenance, pp. 72-81, Nov. 2001.
37. J. Laski and B. Korel, "A Data Flow Oriented Program Testing Strategy," IEEE Trans. Software Eng., vol. 9, no. 3, pp. 347-354, May 1993.
38. J.R. Lyle and M. Weiser, "Automatic Program Bug Location by Program Slicing," Proc. Second Int'l Conf. Computers and Applications, pp. 877-883 June 1987.
39. R.C. Miller and B.A. Myers, "Outlier Finding: Focusing User Attention on Possible Errors," Proc. ACM Symp. User Interface Software and Technology, pp. 81-90, Nov. 2001.
40. S.C. Ntafos, "On Required Element Testing," IEEE Trans. Software Eng., vol. 10, no. 6, pp. 795-803, Nov. 1984.
41. H. Pan and E. Spafford, "Toward Automatic Localization of Software Faults", Proc. 10th Pacific Northwest Software Quality Conf., Oct. 1992.
42. R. Panko, "Finding Spreadsheet Errors: Most Spreadsheet Errors Have Design Flaws that May Lead to Long-Term Miscalculation," Information Week, p. 100, May 1995.
43. R. Panko, "What We Know about Spreadsheet Errors," J. End User Computing, pp. 15-21, Spring 1998.
44. A. Phalgune, C. Kissinger, M. Burnett, C. Cook, L. Beckwith, and J.R. Ruthruff, "Garbage In, Garbage Out? An Empirical Look at Oracle Mistakes by End-User Programmers," Proc. 2005 IEEE Symp. Visual Languages and Human-Centric Computing, pp. 45-52, Sept. 2005.
45. B.C. Pierce and D.N. Turner, "Local Type Inference," ACM Trans. Programming Languages and Systems. vol. 22, no. 1. pp. 1-44, Jan. 2000.
46. B. Pytlik, M. Renieris, S. Krishnamurthi, and S.P Reiss "Automated Fault Localization Using Potential Invariants," Proc. Fifth Int'l Workshop Automated and Algorithmic Debugging, pp. 273-276, Sept. 2003.
47. S. Rapps and E.J. Weyuker, "Selected Software Test Data Using Data Flow Information," IEEE Trans. Software Eng., vol. 11, no. 4, pp. 367-375, Apr. 1985.
48. O. Raz, P. Koopman and M. Shaw "Semantic Anomaly Detection on Online Data Sources," Proc. 24th Int'l Conf. Software Eng., pp. 302-312, May 2002.
49. M. Renieris and S.P. Reiss, "Fault Localization with Nearest Neighbor Queries," Proc. 18th IEEE Int'l Conf. Automated Software Eng., pp. 30-39, Oct. 2003.
50. G. Robertson, "Officials Red-Faced by $24m Gaffe: Error in Contract Bid Hits Bottom Line of Trans Alta Corp.," Ottawa Citizen, 5 June 2003.
51. G. Rothermel, M. Burnett. L. Li, C. Dupuis, and A. Sheretov, "A Methodology for Testing Spreadsheets," ACM Trans. Software Eng. and Methodology, vol. 10, no. 1, pp. 110-147, Jan. 2001.
52. K.J. Rothermel, C.R. Cook, M.M. Burnett, J. Schonfeld, T.R.G. Green, and G. Rothermel, "WYSIWYT Testing in the Spreadsheet Paradigm: An Empirical Evaluation," Proc. 22nd Int'l Conf. Software Eng., pp. 230-239, June 2000.
53. J. Ruthruff, E. Creswick, M. Burnett, C. Cook, S. Prabhakararao, M. Fisher II, and M. Main, "End-User Software Visualizations for Fault Localization, " Proc. ACM Symp. Sotfware Visualization, pp. 123-132, June 2003.
54. J.R. Ruthruff and M. Burnett, "Six Challenges in Supporting, End-User Debugging," Proc. First Workshop End-User Software Eng., pp. 57-61, May 2005.
55. J.R. Ruthruff, M. Burnett, and G. Rothermel, "An Empirical Study of Fault Localization for End-User Programmers," Proc. 27th Int'l Conf. Software Eng., pp. 352-361, May 2005.
56. J.R. Ruthruff, A. Phalgune, L. Beckwith, M. Burnett, and C. Cook, "Rewarding 'Good' Behavior: End-User Debugging and Rewards," Proc. IEEE Symp. Visual Languages and Human-Centric Computing, pp. 115-122, Sept. 2004.
57. J.R. Ruthruff, S. Prabhakararao, J. Reichwein, C. Cook, E. Creswick, and M. Burnett, "Interactive, Visual Fault Localization Support for End-User Programmers," J. Visual Languages and Computing, vol. 16, nos. 1-2. pp. 3-40, Feb.-Apr. 2005.
58. J. Sajaniemi, "Modeling Spreadsheet Audit: A Rigorous Approach to Automatic Visualization," J. Visual Languages and Computing, vol. 11, no. 1, pp. 49-82, Feb. 2000.
59. C. Scaffidi, M. Shaw, and B. Myers, "Estimating the Numbers of End Users and End User Programmers," Proc. 2005 IEEE Symp. Visual Languages and Human-Centric Computing, pp. 207-217, Sept. 2005.
60. S. Siegel and N.J. Castellan Jr., Non-Parametric Statistics for the Behavioral Sciences. Boston: McGraw-Hill, 1998.
61. D.H. Stamatis, Failure Mode Effect Analysis: FMEA from Theory to Execution, second ed. ASQ Quality Press, June 2003.
62. F. Tip, "A Survey of Program Slicing Techniques," J. Programming Languages, vol. 3, no. 3, pp. 121-189, 1995.
63. J.M. Voas, "Software Testability Measurement for Assertion Placement and Fault Localization," Proc. Int'l Workshop Automated and Algorithmic Debugging, pp. 133-144, 1995.
64. E. Wagner and H. Leiberman, "An End-User Tool for e-Commerce Debugging," Proc. Int'l Conf. Intelligent Interfaces, pp. 331, Jan. 2003.
65. E.J. Wagner and H. Lieberman, "Supporting User Hypotheses in Problem Diagnosis on the Web and Elsewhere," Proc. Int'l Conf. Intelligent User Interfaces, pp. 30-37, Jan. 2004.
66. M. Weiser, "Program Slicing," IEEE Trans. Software Eng., vol. 10, no. 4, pp. 352-357, July 1984.
67. A. Wilson, M. Burnett, L. Beckwith, O. Granatir, L. Casburn, C. Cook, M. Durham, and G. Rothermel, "Harnessing Curiosity to Increase Correctness in End-User Programming," Proc. ACM Conf. Human Factors in Computing Systems, pp. 305-312, Apr. 2003.
68. C. Wohlin, P. Runeson, M. Host, B. Regnell, and A. Wesslen, Experimentation in Software Engineering. Boston: Kluwer Academic, 2000.