Resolving structural uncertainty in natural resources management using POMDP approaches

Ecological Modelling

Volumn 222, Issue 5, 2011, Pages 1092-1102

  Williams, Byron K ^a  

^a U S GEOLOGICAL SURVEY   (United States)

Author keywords

Markov decision process; Natural resources; Partial observability; Structural uncertainty; Value iteration

Indexed keywords

CONCEPTUAL FRAMEWORKS; CONSERVATION BIOLOGY; CONSERVATION STRATEGIES; DECISION MAKERS; MANAGEMENT EFFECTIVENESS; MARKOV DECISION PROCESSES; OPTIMAL STRATEGIES; OPTIMIZATION PROBLEMS; PARTIAL OBSERVABILITY; PROCESS CLASS; RAPID GROWTH; REDUCED DIMENSIONALITY; RESOURCE MANAGEMENT; RESOURCE SYSTEM; SOLUTION ALGORITHMS; STRUCTURAL UNCERTAINTY; SYSTEM STATE; UNCERTAIN PROCESS; VALUE ITERATION;

CONSERVATION; DECISION MAKING; ECOLOGY; HISTORIC PRESERVATION; ITERATIVE METHODS; LEARNING ALGORITHMS; MARKOV PROCESSES; OBSERVABILITY; STRUCTURAL OPTIMIZATION; UNCERTAINTY ANALYSIS;

NATURAL RESOURCES MANAGEMENT;

ALGORITHM; CONCEPTUAL FRAMEWORK; DECISION MAKING; ECOLOGICAL MODELING; MARKOV CHAIN; NATURAL RESOURCE; OPTIMIZATION; RESOURCE MANAGEMENT; UNCERTAINTY ANALYSIS;

EID: 79551515102     PISSN: 03043800     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ecolmodel.2010.12.015     Document Type: Article

References (47)

1. Bertsekas D.P. Dynamic Programming and Optimal Control 1995, vol. 1. Athena Scientific, Belmont, MA.
2. Cassandra, A.R., 1994. Optimal policies for partially observable Markov decision processes. Technical Report CS-94-14. Brown University, Providence, RI.
3. Cassandra, A.R., 1998. Exact and approximate algorithms for partially observable Markov decision processes. Ph.D. thesis. Brown University, Providence, RI.
4. Cassandra, A., Littman, M.L., Zhang, N.L., 1997. Incremental pruning: a simple, fast, exact method for partially observable Markov decision processes. Proceedings of the thirteenth Conference on Uncertainty in Artificial Intelligence. Providence, RI.
5. Chadès I., McDonald-Madden E., McCarthy M.A., Wintle B., Linkie M., Possingham H.P. When to stop managing or surveying cryptic threatened species. Proceedings of the National Academy of Sciences 2008, 105:13936-13940.
6. Haight R.G., Polasky S. Optimal control of an invasive species with imperfect information about the level of infestation. Resource and Energy Economics 2010, 32:519-533.
7. Hauskrecht M. Value-function approximations for partially observable Markov decision processes. Journal of Artificial Intelligence Research 2000, 13:33-94.
8. Johnson J.A., Kendall W.L., Dubovsky J.A. Conditions and limitations on learning in the adaptive management of mallard harvests. Wildlife Society Bulletin 2002, 30:176-185.
9. Kaelbling L.P., Littman M.L., Cassandra A.R. Planning and acting in partially observable stochastic domains. Artificial Intelligence 1998, 101:99-134.
10. Lane D. A partially observable model of decision making by fishermen. Operations Research 1989, 37:240-254.
11. Lee P.M. Bayesian Statistics: An Introduction 1989, Edward Arnold, London, UK.
12. Littman, M.L., 1996. Algorithms for sequential decision making. Ph.D. thesis. Brown University, Providence, RI.
13. Lovejoy W.S. A survey of algorithmic methods for partially observable Markov decision processes. Annals of Operations Research 1991, 28:47-65.
14. Lovejoy W.S. Computationally feasible bounds on partially observed Markov decision processes. Operations Research 1991, 39:162-175.
15. Lubow B.C. SDP: generalized software for solving stochastic dynamic optimization problems. Wildlife Society Bulletin 1995, 23:738-742.
16. MacKenzie D.I. Getting the biggest bang for our conservation buck. Trends in Ecology and Evolution 2009, 24:175-177.
17. McDonald-Madden E., Probert W.J.M., Hauser C.E., Runge M.C., Possingham H.P., Jones M.E., Moore J.L., Rout T.M., Vesk P.A., Wintle B.A. Active adaptive conservation of threatened species in the face of uncertainty. Ecological Applications 2010, 20:1476-1489.
18. Monahan G.E. A survey of partially observable Markov decision processes: theory, models, and algorithms. Management Science 1982, 28:1-16.
19. Moore C.T., Conroy M.J. Optimal regeneration planning for old-growth forests: addressing scientific uncertainty in endangered species recovery through adaptive management. Forest Science 2006, 52:155-172.
20. Nichols J.D., Johnson F.A., Williams B.K. Managing North American waterfowl in the face of uncertainty. Annual Review of Ecology and Systematics 1995, 26:177-199.
21. Nichols J.D., Williams B.K. Monitoring for conservation. Trends in Ecology and Evolution 2006, 21:668-673.
22. Nichols, J.D., Koneff, M.D., Heglund, P.J., Knutson, M.G., Seamans, M.E., Lyons, J.E., Morton, J.M., Jones, M.T., Boomer, G.S., Williams, B.K., 2010. Climate change, uncertainty and natural resource management. Journal of Wildlife Management, in press.
23. Pineau J., Gordon G., Thrun S. Point-based value iteration: an anytime algorithm for POMDPs. Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence 2003.
24. Poupart, P., 2005. Exploiting structure to efficiently solve large scale partially observable Markov decision processes. Ph.D. thesis. University of Toronto, Canada.
25. Probert, W.J.M., McDondald-Madden, E., Hau7ser, C.E., Baxter, P.W.J., Possingham, H.P., 2009. Knowledge gain or system benefit in environmental decision making? 18th World IMAC/MODSIM Congress. Cairns, Australia.
26. Puterman M.L. Markov Decision Processes: Discrete Stochastic Dynamic Programming 1994, John Wiley and Sons, New York, NY.
27. Regan H.M., Colyvan M., Burgman M.A. A taxonomy and treatment of uncertainty for ecology and conservation biology. Ecological Applications 2002, 12:618-628.
28. Rout T.M., Hauser C.E., Possingham H.P. Optimal adaptive management for the translocation of a threatened species. Ecological Applications 2009, 19:515-526.
29. Shea K., Possingham H.P., Murdoch W.W., Roush R. Active adaptive management in insect pest and weed control: intervention with a plan for learning. Ecological Applications 2002, 12:927-936.
30. Smallwood R.D., Sondik E.J. The optimal control of partially observable Markov processes over a finite horizon. Operations Research 1973, 21:1071-1088.
31. Sondik, E.J., 1971. The optimal control of Markov decision processes. Ph.D. thesis. Stanford University, Palo Alto, CA.
32. Sondik E.J. The optimal control of partially observable Markov processes over the infinite horizon. Operations Research 1978, 26:282-304.
33. Tomberlin D. Endangered seabird habitat management as a partially observable Markov decision process. Marine Resources Economics 2010, 25:93-104.
34. Tomberlin D. Managing multi-stock fisheries under both process and observation uncertainty. Proceedings of the 25th Mini-EURO Conference on Uncertainty and Risk in Planning and Decision Making 2010, Institute for Systems Engineering and Computers, University of Coimbra, Portugal.
35. Walters C.J. Adaptive Management of Renewable Resources 1986, Blackburn Press, Caldwell, NJ.
36. White, C.C.III, 1991. Partially observed Markov decision processes: a survey. Annals of Operations Research.
37. Williams B.K. Optimal management strategies in variable environments: stochastic optimal control methods. Journal of Environmental Management 1985, 21:95-115.
38. Williams B.K. Uncertainty, learning, and the optimal management of wildlife. Environmental and Ecological Statistics 2001, 8:269-288.
39. Williams B.K. Markov decision processes in natural resources management: observability and uncertainty. Ecological Modeling 2009, 220:830-840.
40. Williams, B.K. Adaptive management of natural resources-framework and issues. Journal of Environmental Management, in press.
41. Williams B.K., Nichols J.D. Systems identification and the adaptive management of waterfowl in the United States. Wildlife Biology 2001, 7:129-142.
42. Williams B.K., Nichols J.D., Conroy M.J. Analysis and Management of Animal Populations 2002, Academic Press, San Diego, CA, 817 pp.
43. Williams, B.K., Shapiro, C.D., Brown, E.D.. Adaptive management: The U.S. Department of the Interior Applications Guide. Adaptive Management Working Group. U.S. Department of the Interior, Washington, DC, in press.
44. Williams, B.K., Szaro, R.C., Shapiro, C.D., 2007. Adaptive management: The U.S. Department of the Interior Technical Guide. Adaptive Management Working Group. U.S. Department of the Interior, Washington, DC.
45. Zhang, N.L., Liu, W., 1996. Planning in stochastic domains: Problem characteristics and approximation. Technical Report HKUST-CS96-31. Department of Computer Science, Hong Kong University of Science and Technology, China.
46. Zhang N.L., Zhang W. Speeding up the convergence of value-iteration in partially observable Markov decision processes. Journal of Artificial Intelligence Research 2001, 14:29-51.
47. Zhou R., Hansen E.A. An improved grid-based approximation algorithm for POMDPs. Proceedings of the Seventeenth International Joint Conference on Artificial Intelligence 2001, 707-714.