Empirical evaluation of decision support systems: Needs, definitions, potential methods, and an example pertaining to waterfowl management

Environmental Modelling and Software

Volumn 22, Issue 2, 2007, Pages 269-277

  Sojda, Richard S ^a  

^a MONTANA STATE UNIVERSITY   (United States)

Author keywords

Decision support system; Empirical evaluation; Model; Trumpeter swan; Validation; Verification

Indexed keywords

DATA REDUCTION; ECOLOGY; ENERGY MANAGEMENT; MATHEMATICAL MODELS;

EMPIRICAL EVALUATION; TRUMPETER SWAN; VALIDATION; VERIFICATION;

DECISION SUPPORT SYSTEMS;

DECISION SUPPORT SYSTEM; EXPERT SYSTEM; MODEL TEST; MODEL VALIDATION; NATURE CONSERVATION; WATERFOWL;

ANATIDAE; CYGNUS BUCCINATOR;

EID: 33749261816     PISSN: 13648152     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.envsoft.2005.07.023     Document Type: Article

References (51)

1. Adelman L. Experiments, quasi-experiments, and case studies: a review of empirical methods for evaluating decision support systems. IEEE Transactions on Systems, Man, and Cybernetics 21 2 (1991) 293-301
2. Adelman L. Evaluating Decision Support and Expert Systems (1992), John Wiley and Sons, New York, NY
3. Andriole S.J. Handbook of Decision Support Systems (1989), TAB Professional and Reference Books, Blue Ridge Summit, Pennsylvania
4. Adrion W.R., Branstad M.A., and Cherniavsky J.C. Validation, verification, and testing of computer software. ACM Computing Surveys 14 2 (1982) 159-192
5. Bahill T.A. Verifying and Validating Personal Computer-based Expert Systems (1991), Prentice-Hall, Inc., Englewood Cliffs, New Jersey
6. Boehm B.W. Software Engineering Economics (1981), Prentice-Hall, Inc., Englewood Cliffs, New Jersey
7. Boland R.J., Mahewshwari A.K., Te'eni D., Schwartz D.G., and Tenkasi R.V. Sharing perspectives in distributed decision making. Proceedings of the Conference on Computer-supported Cooperative Work (1992), Association for Computing Machinery, New York, NY
8. Brehmer B. Distributed decision making: some notes on the literature. In: Rasmussen J., Brehmer B., and Leplat J. (Eds). Distributed Decision Making: Cognitive Models for Cooperative Work (1991), John Wiley and Sons, Chichester, England
9. Carter G.M., Murray M.P., Walker R.G., and Walker W.E. Building Organizational Decision Support Systems (1992), Economic Press Inc., San Diego, California
10. Chau M., Zeng D., Chen H., Huang M., and Hendriawan D. Design and evaluation of a multiagent collaborative Web mining system. Decision Support Systems 35 (2003) 167-183
11. Cohen P.R., and Howe A.E. Toward AI research methodology: three case studies in evaluation. IEEE Transactions on Systems, Man, and Cybernetics 19 3 (1989) 634-646
12. Dshalalow J.H. An anthology of classical queuing models. In: Dshalalow J.H. (Ed). Advances in Queuing: Theory, Methods, and Open Problems (1995), CRC Press, Boca Raton, Florida
13. D'Erchia F., Korschgen C., Nyquist M., Root R., Sojda R., and Stine P. A framework for ecological decision support systems: building the right systems and building the systems right. Information and Technology Report USGS/BRD/ITR-2001-0002 (2001), U.S. Geological Survey, Biological Resources Division, Washington, DC
14. Eason K. Information Technology and Organizational Change (1988), Taylor and Francis Publishing, London, United Kingdom
15. Fishmann G.S., and Kiviat P.J. The statistics of discrete-event simulation. Simulation 10 (1968) 185-195
16. Graham I., and Jones P.L. Expert Systems: Knowledge, Uncertainty, and Decision (1988), Chapman and Hall, New York, NY
17. Graham, J.R., 2001. Real-time scheduling in distributed multi agent systems. PhD dissertation, University of Delaware, Newark, Delaware.
18. Graham J.R., and Decker K.S. Towards a distributed, environment-centered agent framework. In: Jennings Nicholas R., and Lesperance Y. (Eds). Proceedings of the Sixth International Workshop on Agent, Theories, Architectures, and Languages (ATAL-99) (2000), Springer-Verlag, Berlin, Germany
19. Gupta U. Validating and Verifying Knowledge-based Systems (1991), IEEE Computer Society Press, Washington, DC
20. Haberlandt U., Krysanova V., and Bardossy A. Assessment of nitrogen leaching from arable land in large river basins - Part II: regionalisation using fuzzy rule based modelling. Ecological Modelling 150 (2002) 277-294
21. Hillier F.S., and Lieberman G.J. Introduction to Operations Research (1995), McGraw-Hill, Inc., New York, NY
22. Johnson D.H. Validating and evaluating models. In: Shenk T.M., and Franklin A.B. (Eds). Modelling in Natural Resource Management: Development, Interpretation, and Application (2001), Island Press, Washington, DC
23. Kanungo S., Sharma S., and Jain P.K. Evaluation of a decision support system for credit management decisions. Decision Support Systems 30 (2001) 419-436
24. Mathevet R., Bousquet F., Le Page C., and Antona M. Agent-based simulations of interactions between duck population, farming decisions and leasing of hunting rights in the Camargue (Southern France). Ecological Modelling 165 (2003) 107-126
25. Mielke P.W., and Berry K.J. Permutation Methods: A Distance Function Approach (2001), Springer-Verlag, New York, NY
26. Mihram G.A. Some practical aspects of the verification and validation of simulation models. Operational Research Quarterly 23 1 (1972) 17-29
27. Mosqueira-Rey E., and Moret-Bonillo V. Validation of intelligent systems: a critical study and a tool. Expert Systems with Applications 18 (2000) 1-16
28. Murrell S., and Plant R.T. A survey of tools for the validation and verification of knowledge-based systems: 1985-1995. Decision Support Systems 21 (1997) 307-323
29. O'Keefe R.M., Balci O., and Smith E.P. Validating expert system performance. IEEE Expert 2 4 (1987) 81-90
30. O'Leary D.O. Verification of multiple agent knowledge-based systems. International Journal of Intelligent Systems 16 (2001) 361-376
31. Plant R., and Gamble R. Methodologies for the development of knowledge-based systems, 1982-2002. The Knowledge Engineering Review 81 1 (2003) 47-81
32. Pretzsch H., Biber P., and Dursky J. The single tree-based stand simulator SILVA: construction, application and evaluation. Forest Ecology and Management 162 (2002) 3-21
33. Priya S., and Shibasaki R. National spatial crop yield simulation using GIS-based crop production model. Ecological Modelling 135 (2001) 113-129
34. Rao A.S., and Georgeff M.P. Modeling rational agents within a BDI-architecture. In: Allen J., Fikes R., and Sandewall E. (Eds). Principles of Knowledge Representation and Reasoning: Proceedings of the Second International Conference (1991), Morgan Kaufmann Publishers, San Mateo, California 1-18
35. Rao A.S., and Georgeff M.P. BDI agents: from theory to practice. Proceedings of the First International Conference on Multiagent Systems (1995), AAAI Press, Menlo Park, California 312-319
36. Reed, T., 2000. 2000 Fall trumpeter swan survey. Unpublished Report, U.S. Fish and Wildlife Service, Lakeview, Montana.
37. Rice J.A., and Cochran P.A. Independent evaluation of a bioenergetics model for largemouth bass. Ecology 65 3 (1984) 732-739
38. Rios-Insua D., Gallego E., Mateos A., and Rios-Insua S. MOIRA: a decision support system for decision making on aquatic ecosystems contaminated by radioactive fallout. Annals of Operations Research 95 (2000) 341-364
39. Rushby J. Validation and testing of knowledge-based systems: how bad can it get?. In: Gupta U. (Ed). Validating and Verifying Knowledge-based Systems (1988), IEEE Computer Society Press, Los Alamitos, California
40. Rusu V. Combining formal verification and conformance testing for validating reactive systems. Software Testing, Verification and Reliability 13 (2003) 157-180
41. Rykiel Jr. E.J. Testing ecological models: the meaning of validation. Ecological Modelling 90 (1996) 229-244
42. Santos Jr. E. Verification and validation of Bayesian knowledge-bases. Data and Knowledge Engineering 37 (2001) 307-329
43. Scheller R.M., and Mladenoff D.J. A forest growth and biomass module for a landscape simulation model, LANDIS: design validation, and application. Ecological Modelling 180 (2004) 211-229
44. Sojda, R.S., 2002. Artificial intelligence based decision support for trumpeter swan management. PhD dissertation, Colorado State University, Fort Collins, Colorado.
45. Sojda R.S., and Howe A.E. Applying cooperative distributed problem solving methods to trumpeter swan management. In: Cortes U., and Sanchez-Marre M. (Eds). Environmental Decision Support Systems and Artificial Intelligence. American Association for Artificial Intelligence Technical Report WS-99-07 (1999), AAAI Press, Menlo Park, California
46. Sprague Jr. R.H., and Carlson E.D. Building Effective Decision Support Systems (1982), Prentice-Hall, Englewood Cliffs New Jersey
47. Stuth J.W., and Smith M.S. Decision support for grazing lands: an overview. In: Stuth J.W., and Lyons B.G. (Eds). Decision Support Systems for the Management of Grazing Lands. Man and the Biosphere Series Volume 11: Papers From the International Conference on Decision Support Systems for Resource Management (1993), The Parthenon Publishing Group, Pearl River, New York
48. Virvou M., and Kabassi K. Evaluating an intelligent graphical user interface by comparison with human experts. Knowledge-based Systems 17 (2004) 31-37
49. Wallace D.R., and Fujii R.U. Software verification and validation: an overview. IEEE Software 6 3 (1989) 10-17
50. Wang J., and LeDoux C.B. Estimating and validating ground-based timber harvesting production through computer simulation. Forest Science 49 1 (2003) 64-76
51. Weiss G. Prologue: multiagent systems and distributed artificial intelligence. In: Weiss G. (Ed). Multiagent Systems: A Modern Approach to Distributed Artificial Intelligence (1999), MIT Press, Cambridge, Massachusetts