Markov decision processes in natural resources management: Observability and uncertainty

Ecological Modelling

Volumn 220, Issue 6, 2009, Pages 830-840

  Williams, Byron K ^a  

^a U S GEOLOGICAL SURVEY   (United States)

Author keywords

Markov decision process; Natural resources; Observability; Structural uncertainty

Indexed keywords

CONSERVATION; LEARNING ALGORITHMS; NATURAL RESOURCES MANAGEMENT; OBSERVABILITY;

COMPUTING ALGORITHMS; DECISION PROCESS; MARKOV DECISION PROCESS; PARTIAL OBSERVABILITIES; STRUCTURAL UNCERTAINTY;

DECISION MAKING;

ALGORITHM; DECISION MAKING; MARKOV CHAIN; NATURAL RESOURCE; RESOURCE MANAGEMENT;

EID: 60649092155     PISSN: 03043800     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ecolmodel.2008.12.023     Document Type: Article

References (41)

1. Bertsekas D.P. Dynamic Programming and Optimal Control vol. 1 (1995), Athena Scientific, Belmont, MA
2. Chang S.J. A generalized Faustmann model for the determination of optimal harvest age. Canadian Journal of Forest Research 28 (1998) 652-659
3. Cheng, H.-T., 1988. Algorithms for Partially Observable Markov Decision Processes. PhD Dissertation, University of British Columbia, British Columbia, Canada.
4. Christensen V., and Walters C.J. Trade-offs in ecosystem-scale optimization of fisheries management policies. Bulletin of Marine Science 74 (2004) 549-562
5. Crepin A.-S. Multiple species boreal forests: what Faustmann missed. Environmental and Resource Economics 26 (2003) 625-646
6. Intrilligator M.D. Mathematical Optimization and Economic Theory (1971), Prentice-Hall, Englewood Cliffs, NJ
7. Lane D.E. A partially observable model of decision making by fishermen. Operations Research 37 (1989) 240-254
8. Lee P.M. Bayesian Statistics: An Introduction (1989), Edward Arnold, London, UK
9. Littman, M.L., Cassandra, A.R., Kaelbling, L.P., 1996. Efficient Dynamic-Programming Updates in Partially Observable Markov Decision Processes. Technical Report CS-95-19, Brown University, Providence, RI.
10. Loehle C. Control theory and the management of ecosystems. Journal of Applied Ecology 43 (2006) 957-966
11. Lovejoy W.S. A survey of algorithmic methods for partially observable Markov decision processes. Annals of Operations Research 28 (1991) 47-65
12. Lubow B.C. SDP: generalized software for solving stochastic dynamic optimization problems. Wildlife Society Bulletin 23 (1995) 737-742
13. Lubow, B.C., 1997. Adaptive Stochastic Dynamic Programming (ASDP): Supplement to SFP User's Guide. Version 2.0. Colorado Cooperative Fish and Wildlife Research Unit, Colorado State University, Fort Collins, CO.
14. Monahan G.E. A survey of partially observable Markov decision processes: theory, models, and algorithms. Management Science 28 (1982) 1-16
15. Newman D.H. Forestry's golden rule and the development of the optimal forest rotation literature. Journal of Forest Economics 8 (2002) 5-27
16. Nichols J.D., Johnson F.A., and Williams B.K. Managing North American waterfowl in the face of uncertainty. Annual Review of Ecology and Systematics 26 (1995) 177-199
17. Nichols J.D., Runge M.C., Johnson F.A., and Williams B.K. Adaptive harvest management of North American waterfowl populations: a brief history and future prospects. Journal of Ornithology 148 (2007) S343-S349
18. Nichols J.D., and Williams B.K. Monitoring for conservation. Trends in Ecology and Evolution 21 (2006) 668-673
19. Powell W.B. Approximate Dynamic Programming: Solving the Curses of Dimensionality (2007), John Wiley and Sons, Hoboken, NJ
20. Puterman M.L. Markov Decision Processes: Discrete Stochastic Dynamic Programming (1994), John Wiley and Sons, New York, NY
21. Richards S.A., Possingham H.P., and Tizard J. Optimal fire management for maintaining community diversity. Ecological Applications 9 (1999) 880-892
22. Rose S.K., and Chapman D. Timber harvest adjacency economies, hunting, species protection, and old growth value: seeking the dynamic optimum. Ecological Economics 44 (2003) 325-344
23. Shea K., and Possingham H.P. Optimal release strategies for biological control agents: an application of stochastic dynamic programming to population management. Journal of Applied Ecology 37 (2000) 77-86
24. Shea K., Possingham H.P., Murdoch W.W., and Roush R. Active adaptive management in insect pest and weed control: intervention with a plan for learning. Ecology 12 (2002) 927-936
25. Smallwood R.D., and Sondik E.J. The optimal control of partially observable Markov processes over a finite horizon. Operations Research 21 (1973) 1071-1088
26. Smith, T.J., Marshall, L.A., 2008. Bayesian methods in hydrologic modeling: a study of recent advancements in Markov chain Monte Carlo techniques. Water Resources Research 44, W00B05, doi:10.1029/2007WR006705.
27. Sohngen B., and Mendelsohn R. An optimal control model of forest carbon sequestration. American Journal of Agricultural Economics 85 (2003) 448-457
28. Sondik E.J. The optimal control of partially observable Markov processes over the infinite horizon. Operations Research 26 (1978) 282-304
29. Stigter J.D., and van Langevelde F. Optimal harvesting in a two-species model under critical depensation: the case of optimal harvesting in semi-arid grazing systems. Ecological Modelling 191 (2004) 275-290
30. Walters C.J. Adaptive Management of Renewable Resources (1986), Blackburn Press, Caldwell, NJ
31. Walters, C., 1997. Challenges in adaptive management of riparian and coastal ecosystems. Conservation Ecology [online, URL: http://www.consecol.org/vol1/iss2/art1&.].
32. Westphal M.I., Pickett M., Getz W.M., and Possingham H.P. The use of stochastic dynamic programming in optimal landscape reconstruction for metapopulations. Ecological Applications 13 (2003) 543-555
33. White III, C.C., 1991. A survey of solution techniques for the partially observed Markov decision process. Annals of Operations Research 32(1) (online).
34. Williams B.K. Optimal management strategies in variable environments: stochastic optimal control methods. Journal of Environmental Management 21 (1985) 95-115
35. Williams B.K. Review of dynamic optimization methods in renewable natural resources management. Natural Resource Modeling 3 (1989) 137-216
36. Williams B.K. Approaches to the management of waterfowl under uncertainty. Wildlife Society Bulletin 25 (1997) 714-720
37. Williams B.K. Uncertainty, learning, and the optimal management of wildlife. Environmental and Ecological Statistics 8 (2001) 269-288
38. Williams B.K., and Nichols J.D. Systems identification and the adaptive management of waterfowl in the United States. Wildlife Biology 7 (2001) 129-142
39. Williams B.K., Nichols J.D., and Conroy M.J. Analysis and Management of Animal Populations (2002), Academic Press, San Diego, CA
40. Williams, B.K., Szaro, R.C., Shapiro, C.D., 2007. Adaptive Management: U.S. Department of the Interior Technical Guide. Adaptive Management Working Group, U.S. Department of the Interior, Washington, DC.
41. Zhang, L.N., Liu, W., 1996. Planning in Stochastic Domains: Problem Characteristics and Approximations. Technical Report HKUST-CS96-31, Department of Computer Science, Hong Kong University of Science and Technology.