Development of advanced verification and validation procedures and tools for the certification of learning systems in aerospace applications

Collection of Technical Papers - InfoTech at Aerospace: Advancing Contemporary Aerospace Technologies and Their Integration

Volumn 1, Issue , 2005, Pages 104-119

  Jacklin, Stephen A ^a,c,d   Schumann, Johann M ^a   Gupta, Pramod P ^a   Richard, Michael ^b,d,e   Guenther, Kurt ^b,d,f   Soares, Fola ^b,d,g  

^a NASA AMES RESEARCH CENTER   (United States)

^b NASA DRYDEN FLIGHT RESEARCH CENTER   (United States)

^c M/S 269 2 ^*  (United States)

^d AIAA   (United States)

^e M/S 269 3 ^*  (United States)

^f AS and M   (United States)

^g M/S 4830D ^*  (United States)

Author keywords

[No Author keywords available]

Indexed keywords

AUTOMATIC FLIGHT CONTROL; SOFTWARE FAILURES; VEHICLE RECOVERY;

ADAPTIVE CONTROL SYSTEMS; AEROSPACE APPLICATIONS; CODES (STANDARDS); COMPUTER AIDED SOFTWARE ENGINEERING; COMPUTER SIMULATION; LEARNING ALGORITHMS;

LEARNING SYSTEMS;

EID: 33748706018     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (89)

1. Kaneshige, J., Bull, J., and Totah, J., "Generic Neural Flight Control and Autopilot System," AIAA-2000-4281.
2. Rysdyk, R. T., and Calise, A.J., "Fault tolerant flight control via Adaptive Neural Network Augmentation," AIAA 98-4483, August 1998.
3. Nguyen, N.T., Bright, M.M., and Culley, D.E., "Feedback Adaptive Flow Control of Air Injection in Compressors", AIAA Guidance, Navigation, and Control Conference, San Francisco, CA, Aug. 15-18, 2005.
4. Hall, R., Barrington, R., Kirchwey, K. and Alaniz, A., "Shuttle Stability and Control during the Orbiter Repair Manuever, AIAA Guidance, Navigation an dControl Conference and Exhibit, 2005, AIAA 2005-5852.
5. KrishnaKumar, K., Kaneshige, J., Waterman, R., Pires, C., and Ippoloito, C., "A Plug and Play GNC Architecture Using FPGA Components," Proceedings of Infotech@aerospace Conference, Arlington, VA, Sept. 26-29, 2005.
6. Degani, A., Taming HAL, Designing Interfaces Beyond 2001, Published by Palgrave-Macmillan, 2004
7. Deep Space 1 program NASA's Jet Propulsion Laboratory, http://www.jpl.nasa.gov/news/fact_sheets/ds1.pdf
8. Franklin G, Powell J, Emami-Naeini A. Feedback Control of Dynamic Systems. Addison-Wesley Publishing Company: MA, 1994
9. Tomayko, J.E., edited by Gelzer, Christian, The Story of Self-Repairing Flight Control Systems, NASA Dryden Flight Research Center, 2003
10. Ljung, L., System Identification: Theory for the user. Prentice Hall, Englewood Cliffs, NJ, 1987.
11. Widrow, B. and Stearns, S.D., Adaptive Signal Processing. Prentice-Hall, Englewood Cliffs, NJ, 1985.
12. Jacklin, S. A., "Comparison of Five System Identification Algorithms for Rotorcraft Higher Harmonic Control," NASA TP 1998-207687, May 1998.
13. Kim, B. S. and Cause, A. J., "Nonlinear Flight Control using Neural Networks," AIAA J. Guidance, Control, and Dynamics, Vol. 20, No. 1, 1997
14. Ng, S., Cheung, C., Leung, S., and Luk, A., "Fast Convergence for Back-propagation Network with Magnified Gradient Function," Proc. of IEEE Inter. Joint Conf. on Neural Networks (IJCNN'03), Portland, USA, 20-24 June 2003, vol.3, pp. 1903-1908.
15. Malki, H., Canelon, J., Shieh, L., and Jacklin, S. "Neural Network-Based Frequency-Domain Vibration Modeling for Black Hawk Helicopter" AIAA-2004-1977 45th AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics and Materials Conference, Palm Springs, California, Apr. 19-22, 2004
16. Jacklin, S.A., Lowry, M.R., Schumann, J.M., Gupta, P.P., Bosworth, J.T., Zavala, E., Kelly, J.W., Hayhurst, K.J., Belcastro, C.M., and Belcastro, C.M., "Verification, Validation, and Certification Challenges for Adaptive Flight-Critical Control System Software," AIAA Guidance Navigation and Control Conference and Exhibit, 2004. Automated Reasoning, Lect. Notes Comp.
17. Software Considerations in Airborne Systems and Equipment Certification, Document No RTCA (Requirements and Technical Concepts for Aviation) /DO-178B, December 1, 1992.
18. Santhanam, V. "Can Adaptive Flight Control Software be Certified to DO-178B Leve A?", NASA and FAA Software and CEH Conference, Norfolk, VA, July 26-28, 2005.
19. Fromion, V., and Scorletti, G., "A Theoretical Framework for Gain Scheduling," International Journal of Robust and Nonlinear Control, 13(6):951-982, 2003.
20. Nelson, S. D., and Schumann, J. "What Makes a Code Review Trustworthy?", Proceedings of 37th Hawaii International Conference on System Sciences (HICSS-37 2004), 5-8 January 2004
21. http://www.coverity.com
22. http://www.polyspace.com
23. http://www.parasoft.com
24. Holzmann, G.J., "Static Source Code Checking for User-Defined Properties," Integrated Design and Process Technology, IDPT-2002.
25. Brat, G., and Venet, A., "Precise and Scalable Static Program Analysis of NASA Flight Software," Proceedings of the 2005 IEEE Aerospace Conference, Big Sky, MT, March 5-12, 2005.
26. Jacobs, R.A., "Increased rates of convergence through learning rate adaptation," Neural Networks, vol. 1, pp. 295-307, 1988.
27. Weir, M.K., "A method for self-determination of adaptive learning rates in back propagation," Neural Networks, vol. 4, pp. 371-379, 1991.
28. th July 1993.
29. van Ooyen, A. and Nienhuis, B., "Improving the convergence of the back-propagation algorithm," Neural Networks, vol. 5, pp. 465-471, 1992.
30. Ahmad, M. and Salam, F.M.A., "Supervised Learning Using the Cauchy Energy Function," International Conference on Fuzzy Logic and Neural Networks, 1992.
31. Fahlman, S.E., "An empirical study of learning speed in backpropagation networks," Technical report, CMU-CS-88-162, 1988.
32. Riedmiller, M., and Braun, H., "A direct adaptive method for faster back-propagation learning: The RPROP Algorithm," Proc. of Inter. Conf. on Neural Networks, vol 1, pp. 586-591, 1993.
33. Treadgold, N.K. and Gedeon, T.D., "Simulated Annealing and Weight Decay in Adaptive Learning: The SARPROP Algorithm," IEEE Trans. Neural Networks, vol. 9, no. 4, July 1998, pp.662-668.
34. Rumelhart, D.E., Hinto, G.E., and Williams, R.J, "Learning representations by back-propagation errors", NATURE, vol.323, 9, pp.533-536, Oct., 1986.
35. Hagan, M.T. and Menhaj, M., "Training feedforward networks with the Marquardt algorithm", IEEE Trans. Neural Networks, vol.5, pp.989-993, Nov., 1994.
36. Battiti, R. "First and second-order methods for learning: Between steepest descent and Newton's method", Neural Computation, vol.4, no.2, pp. 141-166, 1992
37. Ninomiya, H., Tomita, C., and Asai, H., "An Efficient Algorithm with Second-Order Convergence for Myltilayer Neural Networks", Proc.IEEE&INNS/IJCNN'03, July, 2003.
38. Ninomiya, H. and Kinoshita, N., "A New learning Algorithm without Explicit Error Back-Propagation", Proceedings of IEEE&INNS/IJCNN'99, July, 1999.
39. Ninomiya, H. and Sasaki, A., "3-Layer Recurrent Neural Networks and their Supervised Learning Algorithm", Proc.IEEE&INNS/IJCNN'01, July, 2001.
40. Coetzee, P.M., "Homotopy Approaches for the Analysis and Solution of Neural Network and Other Nonlinear Systems of Equations", Doctoral Thesis, Carnegie Mellon University, May, 1995.
41. Coetzee, F.M., and Stonick, V.L., "On a Natural Homotopy Between Linear and Nonlinear Single-Layer Networks", IEEE Trans. On Neural Networks, vol. 7, no. 2, pp. 307-317, March 1996.
42. Tomita, C., Yashida, M., Ninomiya, H., and Asai, H., "Note on learning algorithm based on Newton homotopy method for feedforward neural networks", Proc. 2004 RISP International Workshop on Nonlinear Circuit and Signal Processing, pp. 128-132, March, 2004.
43. Yamamura, K., Sekiguchi, T., and Inoue, Y., "A Fixed-Point Homotopy Method for Solving Modifyed Nodal Equations", IEEE Trans. Circuit & Systems -I, vol.46, no.6 pp.654-665, June, 1999.
44. Ushida, A., Yamagami, Y., and Nishio, Y., "An Efficient Algorithm for Finding Multiple DC Solutions Based on the SPICE-Oriented Newton Homotopy Method", IEEE Trans. CAD, vol.21, no.3 pp.337-347, March, 2002.
45. Abid, S., Fnaiech, F., and Najim, M., "A Fast Feed-Forward Training algorithm using A Modified Form of the standard Back Propagation Algorithm" IEEE Trans. Neural Network Vol.12, No 2, March 2001.
46. Abid, S., Fnaiech, F., and Najim, M., "Evaluation of the feedforward neural network covariance matrix error", IEEE International Conference on Acoustic Speech and Signal Processing (ICASSP' 2000) June 5-9, Istanbul, Turkey.
47. Fnaiech, N., Sayadi, M., Fnaiech, F., Jervis, B.W. and Cheriet, M., "A Pruned Multilayer Neural Network For Brain Diseases Classification" , Proc. of NNESMED/CIMED 03, July 2003 Sheffield, UK.
48. Hagon, M.T. and Menhaj, M.B., "Training feedforward networks with the Marquardt algorithm" IEEE Transactions on Neural Networks, Vol. 5, pp. 989, November 1994.
49. Scalero, R. S. and Tepedelenglioglu, N., "A Fast New Algorithm for Training Feedforward Neural Networks" IEEE Transactions on signal processing. Vol 40. No, 1, January 1992.
50. Abid, S., Fnaiech, F., Jervis, B.W., and Cheriet, M., "Fast Training of Multilayer Perceptions with a Mixed Norm Algorithm", IJCNN 2005 Conference Proceedings, Montreal, Canada.
51. Rao, Y.N., Kim, S.P., Sanchez, J.C., Erdogmus, D., Principe, J.C., Carmena, J.M., Lebedev, M.A., and Nicolelis, M.A., "Learning Mappings in Brain Machine Interfaces with Echo State Networks", accepted to ICASSP, 2005.
52. Wang, Y., Kim, S., and Principe, J.C., "Comparison of TDNN Training Algorithms in Brain Machine Interfaces", IJCNN 2005 Conference Proceedings, Montreal, Canada.
53. Yang, X., Song, Q., and Liu, S., "Pre-selection of Working Set for SVM Decomposition Algorithm", IJCNN 2005 Conference Proceedings, Montreal, Canada.
54. Piche, S., "Steepest Descent Algorithms for Neural Network Controllers and Filters," IEEE Trans. Neural Networks, vol. 5, no. 2, pp. 198-221, March 1994.
55. Frasconi, P., Gori, M., and Soda, G., "Local Feedback Multilayered Networks," Neural Computation, vol. 4, pp. 120-130, 1992.
56. Tsoi, A.C. and Back, A.D., "Locally Recurrent Globally Feedforward Networks: A Critical Review of Architectures," IEEE Trans. Neural Networks, vol. 5, no. 2, pp. 229-239, March 1994.
57. Mastorocostas, P.A., "A Constrained Optimization Algorithm for Training Locally Recurrent Globally Feedforward Neural Networks", IJCNN 2005 Conference Proceedings, Montreal, Canada.
58. Oyan, Y., Ou, Y., Hwang, S., Chen, C., and Chang, D.T., "Data Classification with a Relaxed Model of Variable Kernel Density Estimation", IJCNN 2005 Conference Proceedings, Montreal, Canada.
59. Atherton, D.P., Nonlinear Control Engineering, Describing Function Analysis and Design, Van Nostrand Reinhold, London 1975.
60. Blaquiere, A., Nonlinear Control Systems, Academic Press, New York, 1966.
61. Gibson, J.E., Nonlinear Automatic Control, McGraw-Hill, New York, 1963.
62. Inagaki S., Kushiro I., and Yamamoto M, "Analysis on Vehicle Stability in Critical Cornering Using Phase-Plane Method," JSAE Review, Vol 16, 1995.
63. Gelb, A. and van der Velde, W.E., Multiple input Describing Functions and Nonlinear Systems Design, McGraw-Hill, New York, 1968
64. Penchuck A., Hattis P., Kubiak E., A frequency domain stability analysis for a phase plane control system, Journal of Guidnace, Control and Dynamics, vol. 8, 1985 pp. 50-55.
65. Yerramalla, S., Cukic, B., and Fuller, E., "Lyapunov Analysis of Neural Network Stability in an Adaptive Flight Control System,", Sixth Symposium on Self-Stabilbizing Systems, 2003.
66. Yerramalla, S., Cukic, B., and Fuller, E., "Lyapunov Stability Analysis of Quantization Error for DCS Neural Networks," International Joint Conference on Neural Networks, Oregon, July 2003.
67. MATLAB, software simulation tool from The MathWorks Inc. http://www.mathworks.com/products.
68. Belcastro, Celeste, and Belcastro, Christine, "On the Validation of Safety Critical Aircraft Systems, Part II: Analytical & Simulation Methods", Proceedings of AIAA Guidance Navigation and Control Conference, Austin TX, August 2003.
69. Duke, E.L., Brumbaugh, R.W., and Disbrow, D., "A Rapid Prototyping Facility for Flight Research in Advanced Systems Concepts," IEEE Computer, May 1989.
70. Hayhurst, K.J., Veerhusan, D.S., Chilenski, J.J., and Rierson, L.K., "A Practical Tutorial on Modified Condition/Decision Coverage," NASA/TM-2001-210876, May 2001.
71. Soares, F., Loparo, K.A., Burken, J., Jacklin, S.A., and Gupta, P.P., "Verification and Validation of Real-time Adaptive Neural Networks using ANCT Tools and Methodologies," Proceedings of Infotech@aerospace Conference, Arlington, VA, Sept. 26-29, 2005.
72. Bird, R.: RASCLE Version 2.0: Design Specification, Programmer's Guide, and User's Guide. Baron Associates, Inc., February, 2002.
73. Belcastro, Christine, and Belcastro, Celeste, "On the Validation of Safety Critical Aircraft Systems, Part I: Analytical & Simulation Methods", Proceedings of AIAA Guidance Navigation and Control Conference, Austin TX, August 2003.
74. Holzmann, G. J., The Spin Model Checker Primer and Reference Manual, Addison-Wesley, Boston, MA, 2004.
75. McMillan, K., Symbolic Model Checking. Kluwer Academic Publishers, Boston, MA, 2003.
76. Visser, W., Havelund, K., Brat, G., Park, S., and Lerda, F., "Model Checking Programs", Kluwer Academic Publisher, 2002.
77. Havelund, K., "Using Runtime Analysis to Guide Model Checking of Java Programs," SPIN Model Checking and Software Verification, Vol. 1885 of Lecture Notes in Computer Science, pp. 245-264, Springer, 2000.
78. Giannakopoulou, D., Pasareanu, C., and Cobleigh, J., "Assume- Guarantee Verification of Source Code with Design-Level Assumptions", Proceedings of the 26th International Conference on Software Engineering (ICSE'2004), Edinburgh, Scotland, May 2004.
79. Artho, C., Havelund, K., and Biere, A., "High-Level Data Races", Proceedings 1st International Workshop on Verification and Validation of Enterprise Information Systems (VVEIS'03), Angers, France, 2003.
80. Miller, S., Anderson, E., Wagner, L., Whalen, M., and Heimdahl, M., "Formal Verification of Flight Critical Software,", AIAA Guidance, Navigation, and Control Conference, San Francisco, CA, Aug 15-18, 2005. Paper AIAA-2005-6431.
81. Scherer, S., Lerda, F., Clarke, E., "Model Checking of Robotic Control Systems," Proceedings of ISAIRAS 2005 Conference, Munich, Germany, Sept. 5-8, 2005
82. Denney, E., Fischer, B., Schumann, J., Richardson, J., "Automatic Certification of Kalman Filters for Reliable Code Generation", IEEE Paper No. 1207: 0-7803-8870-4/05.
83. Dormoy, F.X., "SCADE* The Cost and Time Effective solution for Safety Critical Software Development", Esterel Technologies © 2001 - www.esterel-technologies.com
84. Santhanam, V., "Crafting an FAA - Qualifiable Compiler," Software Tools Forum, Embry-Riddle Aeronautical University, College of Engineering, Daytona Beach, FL, May 18-19, 2004.
85. Gupta, P. and Schumann, J., "A Tool for Verification and Validation of Neural Network Based Adaptive Controllers for High Assurance Systems", In Proceedings High Assurance Software Engineering (HASE). IEEE, 2004.
86. Marjorie, D., Taylor, B., and Skias, S., "Rule Extraction From Dynamic Cell Structure Neural Network Used in a Safety Critical Application," Proceeding of Florida Artificial Intelligence Research Society Conference, Miami FL, May 17-19, 2004.
87. Andrews, R. and Geva, S., "RULEX & CEBP Networks As the Basis for a Rule Refinement System," Hybrid Problems, Hybrid Solutions, John Hallam (Ed), IDS Press, 1995, pp1-12.
88. Andrews, R., J. Diederich, and A. B. Tickle. 1995. A survey and critique of techniques for extracting rules from trained artificial neural networks. Knowledge-Based Systems, 8(6):373-389.
89. Heimdahl, M.P.E, "Tool Intensive Software Development: New Challenges for Verification, Validation, and Certification," Proceedings of FAA and Embry Riddle Aeronautical University Software Tools Forum,May, 2004.