Validation and verification of intelligent systems - what are they and how are they different?

Journal of Experimental and Theoretical Artificial Intelligence

Volumn 12, Issue 4, 2000, Pages 407-420

  Gonzalez, Avelino J ^a,b   Barr, Valerie ^a,b  

^a University of Central Florida   (United States)

^b HOFSTRA UNIVERSITY   (United States)

Author keywords

Evaluation; Expert systems; Intelligent systems; Validation; Verification

Indexed keywords

EID: 0034555289     PISSN: 0952813X     EISSN: 13623079     Source Type: Journal    
DOI: 10.1080/095281300454793     Document Type: Article

References (42)

1. Adrion, W., Branstad, M., and Cherniavski, J., 1982, Validation, verification and testing of computer software. IEEE Transactions on Systems, Man and Cybernetics, 21 (2): 293-301.
2. Barr, V., 1995, TRUBAC: A tool for testing expert systems with rule-base coverage measures, In Proceedings, 13th Annual Pacific Northwest Software Quality Conference, Portland, OR.
3. Barr, V., 1996, Applications of rule-base coverage measures to expert system evaluation, Rutgers University Technical Report DCS-TR-340.
4. Barr, V., 1997, Application of control flow and data flow analysis to rule-based systems. Annals of Software Engineering, 4: 171-189.
5. Barr, V., 1999a, Applications of rule-base coverage measures to expert system evaluation. Journal of Knowledge Based Systems, 12: 27-35.
6. Barr, V., 1999b, Applying reliability engineering to expert systems. In Proceedings, FLAIRS-99, Orlando, FL, pp. 494-498.
7. Chang, C. L., Combs, J. B., and Stachowitz, R. A., 1990, A report of the Expert system Validation Associate (EVA). Expert Systems with Applications, 1: 217-230.
8. Cragun, B. J., and Steudel, H. J., 1987, A decision-table-based processor for checking completeness and consistency in rule-based expert systems. International Journal of Man-Machine Studies, 26: 633-648.
9. Dahl, M., and Williamson, K., 1993, Experiences of using verification tools for maintenance of rule-based systems. In Working Notes, AAAI-93 Workshop on Validation and Verification of Knowledge-Based Systems, Washington, DC, pp. 114-119.
10. Davis, R., 1985, Interactive transfer of expertise. In B. G. Buchanan and E. H. Shortliffe (eds) Rule-Based Expert Systems (Reading, MA: Addison Wesley), pp. 171-205.
11. US Department of Defense, 1994, US Department of Defense Directive DoDD 5000.59.
12. Glenford, J., and Myers, S., 1976, Software Reliability-Principles and Practices (New York: John Wiley and Sons).
13. Ginsberg, A., 1987, A new approach to checking knowledge bases for inconsistency and redundancy. In Proceedings, Third Annual Expert Systems in Government Conference, Washington, DC, pp. 102-111.
14. Ginsberg, A., 1988, Knowledge-base reduction: a new approach to checking knowledge bases for inconsistency and redundancy. In Proceedings, Seventh Annual National Conference on Artificial Intelligence, pp. 585-589.
15. Ginsberg, A., Weiss, S., and Politakis, P., 1985, SEEK2: A generalized approach to automatic knowledge base refinement. In Proceedings, IJCAI-85, pp. 367-374.
16. Green, C. J. R., and Keyes, M. M., 1987, Verification and validation of expert systems. In Proceedings, Western Conference on Expert Systems, Anaheim, CA, pp. 38-43.
17. Grogono, P. D., Preece, A. D., Shingal, R., and Suen, C. Y., 1993, A review of expert systems evaluation techniques In Working Notes, AAAI-93 Workshop on Validation and Verification of Knowledge-Based Systems, Washington, DC, pp. 120-125.
18. Grossner, C., Preece, A. D., Chander, P. G., Radhakrishnan, T., and Suen, C. Y., 1993, Exploring the structure of rule based systems. In Proceedings, AAAI-93, Washington, DC, pp. 704-709.
19. Hutter, D., Langenstein, B., Rock, G., Siekmann, J. H., Stephan, W., and Vogt, R., 2000, Formal software development in Verification Support Environment (VSE). Journal of Experimental and Theoretical AI, this issue.
20. IEEE, 1990, Std. 610.12-1990, Glossary of Software Engineering Terminology.
21. Jafar, M. J., 1989, A tool for interactive verification and validation of rule-based expert systems, Doctoral Dissertation, University of Arizona.
22. Jensen, R., and Tonies, C. (eds), 1979, Software Engineering (Englewood Cliffs, NJ: Prentice Hall).
23. Knauf, R., 1998, Personal communication.
24. Knauf, R., 1999, Validating rule-based systems-a complete methodology. Habilitation Thesis, Technical University of Ilmenau, Germany.
25. Knauf, R., Gonzalez, A. J., and Jantke, K. P., 1999, Validating rule-based systems: a complete methodology, In Proceedings, IEEE Conference on Systems, Man and Cybernetics, Tokyo, pp. V744-V749.
26. McGraw, K. L., and Harbison-Briggs, K., 1989, Knowledge Acquisition-Principles and Guidelines (Englewood Cliffs, NJ: Prentice Hall).
27. Nguyen, T. A., Perkins, W. A., Laffey, T. J., and Pecora, D., 1985, Checking an expert systems knowledge base for consistency and completeness In Proceedings, IJCAI-85, Menlo Park, CA, pp. 374-378.
28. Nguyen, T. A., Perkins, W. A., Laffey, T. J., and Pecora, D., 1987, Knowledge base verification. AI Magazine, 8 (2): 69-75.
29. O'Keefe, R. M., and O'Leary, D. E., 1993, Expert system verification and validation: a survey and tutorial. Artificial Intelligence Review, 7: 3-42.
30. O'Keefe, R. M., Balci, O., and Smith, E. P., 1987, Validating expert systems performance. IEEE Expert, Winter: 81.
31. Polat, F., and Guvenir, H. A., 1991, A unification-based approach for knowledge base verification, Expert systems. The International Journal of Knowledge Engineering, 8 (4): 251-259.
32. Politakis, P., and Weiss, S., 1984, Using empirical analysis to refine expert system knowledge bases. Artificial Intelligence, 22.
33. Preece, A. D., 1989, Verification of rule-based expert systems in wide domains. In Research and development in Expert Systems VI, Proceedings of Expert Systems '89, London, pp. 66-77.
34. Preece, A. D., and Shinghal, R., 1992, Analysis of verification methods for expert systems. In Working Notes, AAAI-92 Workshop on Validation and Verification of Knowledge-Based Systems, San Jose, CA.
35. Preece, A. D., Shinghal, R., and Batarekh, A., 1992, Verifying expert systems: a logical framework and a practical tool. Expert Systems with Applications, 5: 421-436.
36. Preece, A. D., Grossner, C., Chander, P. G., and Radhakrishnan, T., 1993, Structural validation of expert systems using a formal model. In Working Notes, AAAI-93 Workshop on Validation and Verification of Knowledge-Based Systems, Washington, DC, pp. 19-26.
37. Smith, S., 1991, Validation and verification of role-based expert systems. Doctoral Dissertation, Florida State University.
38. Suwa, M., Scott, S. C., and Shortliffe, E. H., 1982, An approach to verifying completeness and consistency in rule-based expert systems. AI Magazine, 3 (4): 16-21.
39. Valiente, G., 1993, Verification of knowledge base redundancy and subsumption using graph transformation. International Journal of Expert Systems, 6 (3): 41-355.
40. van Melle, W., Shortliffe, E. H., and Buchanan, B. G., 1985, EMYCIN: A knowledge engineer's tool for constructing rule-based expert systems. In B. G. Buchanan and E. H. Shortliffe (eds) Rule-Based Expert Systems (Reading, MA: Addison Wesley), pp. 302-313.
41. Zhang, D., and Nguyen, D., 1989, A technique for knowledge base verification. In Proceedings, IEEE International Workshop on Tools for Artificial Intelligence, pp. 399-406.
42. Zlatareva, N., 1992, A framework for knowledge-based system validation, verification and refinement: the VVR system. In Proceedings, FLAIRS-92, Ft. Lauderdale, FL, pp. 10-14.