Automatically finding the control variables for complex system behavior

Automated Software Engineering

Volumn 17, Issue 4, 2010, Pages 439-468

  Gay, Gregory ^a   Menzies, Tim ^a   Davies, Misty ^b   Gundy Burlet, Karen ^b  

^a WEST VIRGINIA UNIVERSITY   (United States)

^b NASA AMES RESEARCH CENTER   (United States)

Author keywords

Contrast set learning; Monte carlo filtering; Optimization; Search based software engineering; Simulation

Indexed keywords

AMES RESEARCH CENTERS; AUTOMATED METHODS; CLASS DISTRIBUTIONS; COMPLEX SYSTEMS; CONTRAST-SET LEARNING; CONTROL VARIABLE; CRITICAL FAILURES; CRITICAL SYSTEMS; DESIGN FAULTS; DOMAIN EXPERTS; FAULT POINT; MONTE CARLO FILTERING; NATIONAL AERONAUTICS AND SPACE ADMINISTRATIONS; NUMERICAL OPTIMIZATION TECHNIQUES; OPTIMIZATION METHOD; OPTIMIZATION TECHNIQUES; RUNNING SIMULATIONS; SEARCH-BASED SOFTWARE ENGINEERING; SIMULATION;

COMPUTER SOFTWARE; LARGE SCALE SYSTEMS; MONTE CARLO METHODS; NASA; OPTIMIZATION; SYSTEMS ENGINEERING; TIME DOMAIN ANALYSIS;

ENGINEERING EDUCATION;

EID: 77956914785     PISSN: 09288910     EISSN: 15737535     Source Type: Journal    
DOI: 10.1007/s10515-010-0072-x     Document Type: Article

References (58)

1. Acevedo, A., Arnold, J., Othon, W., Berndt, J.: ANTARES: Spacecraft simulation for multiple user communities and facilities. In: AIAA Modeling and Simulation Technologies Conference and Exhibit, pp. 2007-6888 (2007)
2. Agrawal, R., Imeilinski, T., Swami, A.: Mining association rules between sets of items in large databases. In: Proceedings of the 1993 ACM SIGMOD Conference, Washington, DC, USA (1993). Available from http://citeseer.nj.nec. com/agrawal93mining.html
3. Antoniol, G., Gueheneuc, Y.: Feature identification: a novel approach and a case study. In: ICSM 2005, pp. 357-366 (2005)
4. Antoniol, G., Canfora, G., Casazza, G., De Lucia, A., Merlo, E.: Recovering traceability links between code and documentation. IEEE Trans. Softw. Eng. 28(10), 970-983 (2002)
5. Austin, P., Grootendorst, P., Anderson, G.: A comparison of the ability of different propensity score models to balance measured variables between treated and untreated subjects: a Monte Carlo study. Stat. Med. 26, 734-753 (2007)
6. Basili, V., McGarry, F., Pajerski, R., Zelkowitz, M.: Lessons learned from 25 years of process improvement: the rise and fall of the NASA software engineering laboratory. In: Proceedings of the 24th International Conference on Software Engineering (ICSE) 2002, Orlando, Florida (2002). Available from http://www.cs.umd.edu/projects/SoftEng/ESEG/papers/83.88pdf
7. Bay, S. B., Pazzani, M. J.: Detecting change in categorical data: mining contrast sets. In: Proceedings of the Fifth International Conference on Knowledge Discovery and Data Mining (1999). Available from http://www.ics.uci. edu/pazzani/Publications/stuccopdf
8. Bishop, C.: Pattern Recognition and Machine Learning. Springer, New York (2007)
9. Boehm, B., Papaccio, P.: Understanding and controlling software costs. IEEE Trans. Softw. Eng. 14(10), 1462-1477 (1988)
10. Boetticher, G.: An assessment of metric contribution in the construction of a neural network-based effort estimator. In: Second International Workshop on Soft Computing Applied to Software Engineering, Enschade, NL (2001). Available from: http://nas.cl.uh.edu/boetticher/publications.html
11. Breiman, L., Friedman, J. H., Olshen, R. A., Stone, C. J.: Classification and regression trees. Technical report, Wadsworth International, Monterey, CA (1984)
12. Cai, C. H., Fu, A. W. C., Cheng, C. H., Kwong, W. W.: Mining association rules with weighted items. In: Proceedings of International Database Engineering and Applications Symposium (IDEAS 98) (August 1998). Available from http://www.cse.cuhk.edu.hk/kdd/assoc-rule/paperpdf
13. Cleland-Huang, J., Settimi, R., Zou, X., Solc, P.: The detection and classification of non-functional requirements with application to early aspects. In: RE 2006, pp. 36-45 (2006)
14. Cornford, S. L., Feather, M. S., Hicks, K. A.: DDP a tool for life-cycle risk management. In: IEEE Aerospace Conference, Big Sky, Montana, pp. 441-451 (March 2001)
15. Dechter, R.: Constraint Processing. Morgan Kaufmann, San Mateo (2003)
16. Eruhimov, V., Martyanov, V., Tuv, E.: Knowledge discovery in databases: PKDD 2007. In: Constructing High Dimensional Feature Space for Time Series Classification, pp. 414-421. Springer, Berlin (2007)
17. Fayyad, U., Irani, I.: Multi-interval discretization of continuous-valued attributes for classification learning. In: Proceedings of the Thirteenth International Joint Conference on Artificial Intelligence, pp. 1022-1027 (1993)
18. Feather, M., Cornford, S., Hicks, K., Kiper, J., Menzies, T.: Application of a broad-spectrum quantitative requirements model to early-lifecycle decision making. In: IEEE Software (2008). Available from http://menzies.us/pdf/08ddppdf
19. Fischer, B., Schumann, J.: Autobayes: a system for generating data analysis programs from statistical models. J. Funct. Program. 13, 483-508 (2003)
20. Gay, G., Menzies, T., Jalali, O., Mundy, G., Gilkerson, B., Feather, M., Kiper, J.: Finding robust solutions in requirements models. Autom. Softw. Eng. 17(1), 87-116 (2010)
21. Gigerenzer, G., Goldstein, D. G.: Reasoning the fast and frugal way: models of bounded rationality. Psy-chol. Rev. 650-669 (1996)
22. Gill, P. E., Murray, W., Wright, M. H.: Practical Optimization. Academic Press, San Diego (1981)
23. Goldberg, D. E.: Genetic Algorithms in Search, Optimization, and Machine Learning. Addison-Wesley, Reading (1989)
24. Gu, J., Purdom, P., Franco, J., Wah, B.: Algorithms for the satisfiability (sat) problem: a survey. In: DI-MACS Series in Discrete Mathematics and Theoretical Computer Science, pp. 19-152. American Mathematical Society, Providence (1997)
25. Gundy-Burlet, K., Schumann, J., Barrett, T., Menzies, T.: Parametric analysis of ANTARES re-entry guidance algorithms using advanced test generation and data analysis. In: 9th International Symposium on Artificial Intelligence, Robotics and Automation in Space (2007)
26. Gundy-Burlet, K., Schumann, J., Barrett, T., Menzies, T.: Parametric analysis of a hover test vehicle using advanced test generation and data analysis. In: AIAA Aerospace (2009)
27. Holte, R. C.: Very simple classification rules perform well on most commonly used datasets. Mach. Learn. 11, 63 (1993)
28. Holzmann, G. J.: The model checker SPIN. IEEE Trans. Softw. Eng. 23(5), 279-295 (1997)
29. Hu, Y.: Treatment learning: implementation and application. Master's thesis, Department of Electrical Engineering, University of British Columbia (2003)
30. Jing, H., George, R., Tuv, E.: Contributors to a signal from an artificial contrast. In: Informatics in Control, Automation and Robotics II, pp. 71-78. Springer, Berlin (2007)
31. Kirkpatrick, S., Gelatt, C. D., Vecchi, M. P.: Optimization by simulated annealing. Science 4598, 671-680 (1983)
32. Kohavi, R., John, G.: Wrappers for feature subset selection. Artif. Intell. 97 (1-2), 273-324 (1997)
33. Mann, H. B., Whitney, D. R.: On a test of whether one of two random variables is stochastically larger than the other. Ann. Math. Stat. 18(1), 50-60 (1947). Available on-line at http://projecteuclid.org/DPubS?service= UI&version=1.0&verb=Display&handle=euclid.aoms/1177730491
34. Marcus, A., Maletic, J.: Recovering documentation-to-source code traceability links using latent semantic indexing. In: Proceedings of the Twenty-Fifth International Conference on Software Engineering (2003)
35. Menzies, T., Hu, Y.: Data mining for very busy people. In: IEEE Computer (November 2003). Available from http://menzies.us/pdf/03tar2pdf
36. Menzies, T., Sinsel, E.: Practical large scale what-if queries: case studies with software risk assessment. In: Proceedings ASE 2000 (2000). Available from http://menzies.us/pdf/00asepdf
37. Menzies, T., Dekhtyar, A., Distefano, J., Greenwald, J.: Problems with precision. IEEE Trans. Softw. Eng. (September 2007). Available from http://menzies.us/pdf/07precisionpdf
38. Menzies, T., Greenwald, J., Frank, A.: Data mining static code attributes to learn defect predictors. IEEE Trans. Soft. Eng. (January 2007). Available from http://menzies.us/pdf/06learnPredictpdf
39. Metropolis, N., Rosenbluth, A. W., Rosenbluth, M. N., Teller, A. H., Teller, E.: Equation of state calculations by fast computing machines. J. Chem. Phys 21, 1087-1092 (1953)
40. Oakley, J., O'Hagan, A.: Probabilistic sensitivity analysis of complex models: a Bayesian approach. J. R. Stat. Soc. B 66(3), 751-769 (2004)
41. Orrego, A. S.: Sawtooth: Learning from huge amounts of data. Master's thesis, Computer Science, West Virginia University (2004)
42. Quinlan, R.: C4.5: Programs for Machine Learning. Morgan Kaufman, San Mateo (1992). ISBN: 1558602380
43. Rose, K., Smith, E., Gardner, R., Brenkert, A., Bartell, S.: Parameter sensitivities, Monte Carlo filtering, and model forecasting under uncertainty. J. Forecast. 10, 117-133 (1991)
44. Saltelli, A., Chan, K., Scott, E. M.: Sensitivity Analysis. Wiley, New York (2000)
45. Saltelli, A., Ratto, M., Andres, T., Campolongo, F., Cariboni, J., Gatelli, D., Saisana, M., Tarantola, S.: Global Sensitivity Analysis: The Primer. Wiley, New York (2008)
46. Schumann, J., Gundy-Burlet, K., Pasareanu, C., Menzies, T., Barrett, T.: Tool support for parametric analysis of large software systems. In: Proc. Automated Software Engineering, 23rd IEEE/ACM International Conference (2008)
47. Schumann, J., Gundy-Burlet, K., Pasareanu, C., Menzies, T., Barrett, A.: Software V&V support by parametric analysis of large software simulation systems. In: 2009 IEEE Aerospace Conference (2009)
48. Sendall, S., Kozacaynski, W.: Model transformation: the heart and soul of model-driven software development. IEEE Softw. 20(5), 42-45 (2003)
49. Sims, C.: Matlab optimization software. QM&RBC Codes, Quantitative Macroeconomics & Real Business Cycles (March 1999)
50. Spear, R., Grieb, T., Shang, N.: Parameter uncertainty and interaction in complex environmental models. Water Resour. Res. 30(11), 3159-3169 (1994)
51. Taylor, B. J., Darrah, M. A.: Rule extraction as a formal method for the verification and validation of neural networks. In: IJCNN '05: Proceedings. 2005 IEEE International Joint Conference on Neural Networks, vol. 5, pp. 2915-2920 (2005)
52. Torkkola, K., Tuv, E.: Ensembles of regularized least squares classifiers for high-dimensional problems. In: Feature Extraction, pp. 297-313. Springer, Berlin (2006)
53. Towell, G., Shavlik, J.: Extracting refined rules from knowledge-based neural networks. Mach. Learn. 13, 71-101 (1993)
54. Turhan, B., Menzies, T., Bener, A. B., Di Stefano, J.: On the relative value of cross-company and within-company data for defect prediction. In: Empirical Software Engineering (2009). Available from http://menzies.us/pdf/ 08ccwcpdf
55. Tuv, E., Borisov, A., Torkkola, K.: Best subset feature selection for massive mixed-type problems. In: Intelligent Data Engineering and Automated Learning-IDEAL 2006, pp. 1048-1056. Springer, Berlin (2006)
56. Uribe, T., Stickel, M.: Ordered binary decision diagrams and the Davis-Putnam procedure. In: Proc. of the 1st International Conference on Constraints in Computational Logics, pp. 34-49. Springer, Berlin (1994)
57. Witten, I. H., Frank, E.: Data Mining: Practical Machine Learning Tools and Techniques with Java Implementations. Morgan Kaufmann, San Mateo (1999)
58. Witten, I. H., Frank, E.: Data Mining: Practical Machine Learning Tools and Techniques, 2nd edn. Morgan Kaufmann, San Mateo (2005)