Verification and validation of knowledge-based systems

IEEE Transactions on Knowledge and Data Engineering

Volumn 11, Issue 1, 1999, Pages 202-212

  Tsai, Wei Tek ^a,b   Vishnuvajjala, Rama ^c   Zhang, Du ^a,d  

^a IEEE

^b UNIVERSITY OF MINNESOTA   (United States)

^c NOKIA RESEARCH CENTER   (United States)

^d CALIFORNIA STATE UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

SYSTEMS VALIDATION AND VERIFICATION;

EXPERT SYSTEMS; FAILURE ANALYSIS; RELIABILITY;

KNOWLEDGE ENGINEERING;

EID: 0032677016     PISSN: 10414347     EISSN: None     Source Type: Journal    
DOI: 10.1109/69.755629     Document Type: Article

References (49)

1. Validation, Verification, and Test of Knowledge-Based Systems, M. Ayel and J.P. Laurent, eds., John Wiley and Sons, Chichester, U.K., 1991.
2. M. Ayel and J.P. Laurent, "SACCO-SYCOJET: Two Different Ways of Verifying Knowledge-Based Systems," Validation, Verification, and Test of Knowledge-Based Systems, M. Ayel and J.P. Laurent, eds., John Wiley and Sons, Chichester, U.K., pp. 63-76, 1991.
3. F.B. Bastani and I.R. Chen, "Assessment of Reliability of Artificial Intelligence Programs," Proc. IEEE Conf. Tools for Artificial Intelligence, pp. 753-759, 1990.
4. V.E. Barker, D.E. O'Connor, J. Bachant, and E. Soloway, "Expert Systems for Configuration at Digital: XCON and Beyond," Comm. ACM, vol. 32, no. 3, pp. 298-318, 1989.
5. B. Beizer, Software Testing Techniques, Van Nostrand Reinhold, New York, 1990.
6. N. Botten, "Complex Knowledge Base Verification Using Matrices," Lecture Notes in Artificial Intelligence, F. Belli and F.J. Radermacher, eds., pp. 225-235, Springer-Verlag, Berlin, 1992.
7. B.C. Buchanan, "Artificial Intelligence As An Experimental Science," J.E. Fetzer, ed., Aspects of Artificial Intelligence, Kluwer, pp. 209-250, 1988.
8. C.L. Chang, J.B. Combs, and R.A. Stachowitz, "A Report on the Expert Systems Validation Associate (EVA)," Expert Systems with Applications, vol. 1, pp. 217-230, 1990.
9. E. Charles and O. Dubois, "MELODIA: Logical Methods for Checking Knowledge Bases," Validation, Verification, and Test of Knowledge-Based Systems, M. Ayel and J.P. Laurent, eds., John Wiley and Sons, Chichester, U.K., pp. 95-105, 1991.
10. B.J. Cragun and H.J. Steudel, "A Decision-Table-Based Processor for Checking Completeness and Consistency in Rule-Based Expert Systems," Int'l J. Man-Machine Studies, vol. 26, pp. 633-648, 1987.
11. S. Craw, "Judging Knowledge Base Quality," Validation, Verification, and Test of Knowledge-Based Systems, M. Ayel and J.P. Laurent, ed., pp. 207-219, John Wiley and Sons, Chichester, U.K., 1991.
12. Expert Systems With Applications, special issue on verification and validation of knowledge-based systems, C. Culbert, ed., vol. 1, no. 3, 1990.
13. L. De Raedt, G. Sablon, and M. Bruynooghe, "Using Interactive Concept Learning for Knowledge-Base Validation and Verification," Validation, Verification, and Test of Knowledge-Based Systems, M. Ayel and J.P. Laurent, ed., pp. 177-190, John Wiley and Sons, Chichester, U.K., 1991.
14. A. Ginsberg, S. Weiss, and P. Politakis, "SEEK2: A Generalized Approach to Automatic Knowledge Base Refinement," Proc. Int'l Joint Conf. Artificial Intelligence, pp. 367-374, 1985.
15. A. Ginsberg, "Knowledge-Base Reduction: A New Approach to Checking Knowledge Bases for Inconsistency and Redundancy," Proc. Seventh Nat'l Conf. Artificial Intelligence, pp. 585-589, 1988.
16. A. Ginsberg and K. Williamson, "Inconsistency and Redundancy Checking for Quasi-First-Order-Logic Knowledge Bases," Int'l J. Expert Systems, vol. 6, no. 3, pp. 321-340, 1993.
17. Validating and Verifying Knowledge-Based Systems, U.G. Gupta, ed., IEEE CS Press, Los Alamitos, Calif., 1991.
18. D. Hamilton, K. Kelley, and C. Culbert, "State-of-the-Practice in Knowledge-Based System Verification and Validation," Expert Systems With Applications, vol. 3, pp. 403-410, 1991.
19. F. Hayes-Roth and N. Jacobstein, "The State of Knowledge-Based Systems," Comm. ACM, vol. 37, no. 3, pp. 27-39, 1994.
20. H. Huang, W.T. Tsai, and S. Subramanian, "Generalized Program Slicing for Software Maintenance," Proc. Software Eng. and Knowledge Eng., pp. 261-268, 1996.
21. IEEE Glossary of Software Eng. Terminology, IEEE Standard 610.12-1990, IEEE, 1990.
22. S. Kirani, I.A. Zualkernan, and W.T. Tsai, "Evaluation of Expert System Testing Methods," Comm. ACM, vol. 37, no. 11, pp. 71-81, 1994.
23. P. Lafon, "A Descriptive Model of Predicates for Verifying Production Systems," Validation, Verification, and Test of Knowledge-Based Systems, M. Ayel and J.P. Laurent, eds., John Wiley and Sons, Chichester, U.K., pp. 149-162, 1991.
24. T.J. Lydiard, "Overview of Current Practice and Research Initiatives for the Verification and Validation of KBS," Knowledge Eng. Rev., vol. 7, no. 2, pp. 101-113, 1992.
25. H. Marathe, T.K. Ma, and C.C. Liu, "An Algorithm for Identification of Relations Among Rules," Proc. IEEE Int'l Workshop Tools for Artificial Intelligence, pp. 360-366, 1989.
26. D. Marr, Vision, W.H. Freeman, New York, 1982.
27. P. Meseguer and A. Verdaguer, "Verification of Multi-Level Rule-Based Expert Systems: Theory and Practice," Int'l J. Expert Systems, vol. 6, no. 2, pp. 163-192, 1993.
28. L.A. Miller, "Recommended Guidelines for V&V of Various Kinds of Systems at Various Lifecycle Phases," Proc. AAAI Workshop Validation and Verification of Knowledge-Based Systems, pp. 1-9, 1994.
29. J. Musa, A. Iannino, and K. Okumoto, Software Reliability: Measurement, Prediction, Application, McGraw-Hill, New-York, 1987.
30. D.L. Nazareth, "Issues in the Verification of Knowledge in Rule-Based Systems," Int'l J. Man-Machine Studies, vol. 30, pp. 255-271, 1989.
31. T.A. Nguyen, W.A. Perkins, T.J. Laffey, and D. Pecora, "Knowledge Base Verification," Artificial Intelligence, vol. 8, pp. 69-75, 1987.
32. R.M. O'Keefe and D.E. O'Leary, "Expert System Verification and Validation: A Survey and Tutorial," Artificial Intelligence Rev., vol. 7, pp. 3-42, 1993.
33. D.E. O'Leary, "Design, Development, and Validation of Expert Systems: A Survey of Developers," Validation, Verification, and Test of Knowledge-Based Systems, M. Ayel and J.P. Laurent, eds., John Wiley and Sons, Chichester, U.K., pp. 3-19, 1991.
34. Int'l J. Intelligent Systems, special issue on verification and validation of intelligent systems: five years of AAAI workshops," D.E. O'Leary, ed., vol. 9, nos. 8-9, 1994.
35. "Validation and Verification of Knowledge-Based Systems," E. Plaza, ed., IEEE Expert, vol. 8, pp. 45-81, 1993.
36. A. Preece, R. Shinghal, and A. Batarekh, "Verifying Expert Systems: A Logical Framework and A Practical Tool," Expert Systems with Applications, vol. 5, nos. 2-3, pp. 421-436, 1992.
37. Int'l J. Expert Systems, A. Preece and C. Suen, eds., special issues on verification and validation of knowledge-based systems, vol. 6, nos. 2-3, 1993.
38. R.S. Pressman, Software Eng.: A Practitioner's Approach, McGraw-Hill, New York, 1987.
39. M.C. Rousset, "On the Consistency of Knowledge Bases: The COVADIS System," Proc. Eighth European Conf. Artificial Intelligence, pp. 79-84, 1988.
40. J. Rushby, "Quality Measures and Assurance for Artificial Intelligence Software," NASA Contractor Report 4187, Oct. 1988.
41. J. Rushby, " Formal Verification of Artificial Intelligence Software," NASA Contractor Report 181827, Feb. 1989.
42. S. Subramanian, W.T. Tsai, and S. Kirani, "Hierarchical Data Flow Analysis for OO Programs," J. Object-Oriented Programming, vol. 7, no. 2, pp. 36-46, 1994.
43. W.T. Tsai and I.A. Zualkernan, "Towards a Unified Framework for Testing Expert Systems," Proc. Int'l Conf. Software Eng. and Knowledge Eng., 1990.
44. W.T. Tsai, I.A. Zualkernan, and S. Kirani, "Pragmatic Testing Methods for Expert Systems," Int'l J. Artificial Intelligence Tools, vol.2, no. 2, pp. 181-217, 1993.
45. D.C. Wilkins and B.C. Buchanan, "On Debugging Rule Sets When Reasoning Under Uncertainty," Proc. Fourth Nat'l Conf. Artificial Intelligence, pp. 448-454, 1986.
46. D. Zhang and D. Nguyen, "A Tool for Knowledge Base Verification," N. Bourbakis, ed., Knowledge Eng. Shells - Systems and Techniques, Advanced Series on Artificial Intelligence, vol. 2, World Scientific, pp. 455-486, 1993.
47. D. Zhang and D. Nguyen, " PREPARE: A Tool for Knowledge Base Verification," IEEE Trans. Knowledge and Data Eng., vol. 6, no. 6, pp. 983-989, 1994.
48. N. Zlatareva and A. Preece, "State of the Art in Automated Validation of Knowledge-Based Systems," Expert Systems with Applications, vol. 7, no. 2, pp. 151-167, 1994.
49. I.A. Zualkernan, W.T. Tsai, P.E. Johnson, and J.H. Moller, "Utility of Knowledge-Level Specifications," Proc. Fourth Ann. Artificial Intelligence and Advanced Computer Technology Conf., pp. 79-85, 1988.