An intelligent agent for optimal river-reservoir system management

Water Resources Research

Volumn 48, Issue 9, 2012, Pages

  Rieker, Jeffrey D ^a   Labadie, John W ^b  

^a U S BUREAU OF RECLAMATION   (United States)

^b Department of Mechanical Engineering   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ADAPTIVE BEHAVIOR; CALIFORNIA; DROUGHT PERIODS; DYNAMIC DISPLAYS; FISH SPECIES; HYDROLOGIC SYSTEMS; LEARNING METHODS; LEARNING PROCESS; MARKOV DECISION PROCESSES; OPERATIONAL POLICY; OPTIMAL MANAGEMENT; PROBABILISTIC MODELS; QUALITY ENHANCEMENT; STOCHASTIC OPTIMIZATIONS; STREAMFLOW AUGMENTATION; SYSTEM MANAGEMENT; WATER TEMPERATURES;

ARTIFICIAL INTELLIGENCE; GRAPHICAL USER INTERFACES; INTELLIGENT AGENTS; MARKOV PROCESSES; OPTIMIZATION; REINFORCEMENT LEARNING; RIVERS; WATER MANAGEMENT; WATER QUALITY;

RESERVOIR MANAGEMENT;

ARTIFICIAL INTELLIGENCE; CALIBRATION; DECISION MAKING; DROUGHT; ENDANGERED SPECIES; MARKOV CHAIN; OPTIMIZATION; POLICY ANALYSIS; RESERVOIR; RIVER SYSTEM; SOFTWARE; STOCHASTICITY; STREAMFLOW; SURVIVAL; VISUALIZATION; WATER MANAGEMENT; WATER QUALITY; WATER STORAGE; WATER TEMPERATURE;

TRUCKEE RIVER; UNITED STATES;

EID: 84867226614     PISSN: 00431397     EISSN: None     Source Type: Journal    
DOI: 10.1029/2012WR011958     Document Type: Article

References (48)

1. Abolpour, B. (2007), Water allocation improvement in river basin using adaptive neural fuzzy reinforcement learning approach, Appl. Soft Com-put., 7(1), 265-285. (Pubitemid 44739432)
2. Bertsekas, D., and J. Tsitsiklis (1996), Neuro-Dynamic Programming, Athena Sci., Belmont, Mass.
3. Bhattacharya, B., A. H. Lobbrecht, and D. P. Solamatine (2003), Neural networks and reinforcement learning in control of water systems, J. Water Resour. Plann. Manage., 129(6), 458-465. (Pubitemid 37450405)
4. Bouchart, F. (1998), A reinforcement learning model for the operation of conjunctive use schemes, in Hydraulic Engineering Software VII, edited by W. Blaine, pp. 319-329, Comput. Mech., Southampton, U. K.
5. Breiman, L. (2001), Random forests, Mach. Learn., 45(1), 5-32. (Pubitemid 32933532)
6. Brown, W. M., J. O. Nowlin, L. H. Smith, and M. R. Flint (1986), River-quality assessment of the Truckee and Carson River system, California and Nevada: Hydrologic characteristics, U.S. Geol. Surv. Open File Rep., 84-576, Denver, Colo.
7. Busoniu, L., R. Babuska, B. De Schutter, and D. Ernst (2010), Reinforcement Learning and Dynamic Programming Using Function Approxima-tors, CRC Press, Boca Raton, Fla.
8. Castelletti, A. (2002), Reinforcement learning in the operational management of a water system, in Modeling and Control in Environmental Issues, edited by A. Sano, S. Nishioka, and H., Tamura, pp. 303-308, Pergamon, Oxford, U. K.
9. Castelletti, A., S. Galelli, M. Restelli, and R. Soncini-Sessa (2010), Tree-based reinforcement learning for optimal water reservoir operation, Water Resour. Res., 46, W09507, doi:10.1029/2009WR008898.
10. Chang, L. C., and F. J. Chang (2001), Intelligent control for modelling of real-time reservoir operation, Hydrol. Processes, 15(9), 1621-1634.
11. Chang, Y. T., L. C. Chang, and F. J. Chang (2005), Intelligent control for modeling of real time reservoir operation: Part II ANN with operating curves, Hydrol. Processes, 19(7), 1431-1444.
12. City of Reno, City of Sparks, County of Washoe, U.S. Department of the Interior, U.S. Department of Justice, U.S. Environmental Protection Agency, Nevada Division of Environmental Protection, and Pyramid Lake Paiute Tribe (1996), Truckee River water quality settlement agreement.
13. Coors, S. (2006), Simulating operations in the Truckee-Carson RiverWare modeling system, J. Nev. Water Resour. Assoc., 3(1), 23-32.
14. Covay, K. (1996), Environmental and hydrologic settings of the Las Vegas Valley area and the Carson and Truckee River basins, Nevada and California, U.S. Geol. Surv. Water Resour. Invest. Rep., 96-4087.
15. Davies, S. (1997), Multidimensional triangulation and interpolation for reinforcement learning, in Advances in Neural Information Processing Systems, edited by M. Mozer, M. Jordan, and T. Petsche, 9, 1005-1011, MIT Press, Cambridge, Mass.
16. Dreyfus, S., and A. Law (1977), The Art and Theory of Dynamic Programming, Academic, New York.
17. Geist, M., and O. Pietquin (2011), Parametric value function approximation: A unified view, paper presented at IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning, Paris, 11-15 April.
18. Haykin, S. (2008), Neural Networks and Learning Machines, Prentice-Hall, Upper Saddle River, N. J.
19. Kilroy, K. (1997), Water-quality assessment of the Las Vegas Valley area and the Carson and Truckee River Basins, Nevada and California: Nutrients, pesticides, and suspended sediment, October 1969-April (1990), U.S. Geol. Surv. Water Resour. Invest. Rep., 97-4106.
20. Labadie, J. (2004), Optimal operation of multireservoir systems: State-of-the-art review, J. Water Resour. Plann. Manage., 130(2), 93-111. (Pubitemid 38459486)
21. Lee, J. H., and J. W. Labadie (2007), Stochastic optimization of multireservoir systems via reinforcement learning, Water Resour. Res., 43, W11408, doi:10.1029/2006WR005627.
22. Mariano-Romero, C. (2007), Multi-objective optimization of water-using systems, Eur. J. Oper. Res., 181(3), 1691-1707. (Pubitemid 46522497)
23. Neumann, D. (2001), An operations model for temperature management of the Truckee River above Reno, Nevada, MS thesis, Dep. of Civ. Eng., Univ. of Colo., Boulder.
24. Neumann, D., B. Rajagopalan, and E. Zagona (2003), Regression model for daily maximum stream temperature, J. Environ. Eng., 129(7), 667-674. (Pubitemid 36901394)
25. Nevada Division of Water Planning (1997), Truckee River chronology, report, Carson City, Nev.
26. Pohll, G., et al. (2001). Evaluation of groundwater and solute transport in the Fernley-Wadsworth area, Publ. 41173, Desert Res. Inst., Reno, Nev.
27. Powell, W. (2011), Approximate Dynamic Programming: Solving the Curses of Dimensionality, John Wiley, Hoboken, N. J.
28. Rani, D., and M. Moreira (2010), Simulation-optimization modeling: A survey and potential application in reservoir systems operation, Water Resour. Manage., 24(6), 1107-1138.
29. Rieker, J. D. (2011), Optimal reservoir operations for riverine water quality improvement: A reinforcement learning strategy, Ph.D. dissertation, Dep. of Civ. Environ. Eng., Colo. State Univ., Fort Collins.
30. Rieker, J. D., S. Coors, M. Mann, and T. Scott (2005). Modeling in support of water operations in the Truckee River basin, paper presented at Watershed 2005, Environ. and Water Resour. Inst., Am. Soc. of Civ. Eng., Williamsburg, Va.
31. Russell, S. J., and P. Norvig (2010), Artificial Intelligence: A Modern Approach, Prentice-Hall, Upper Saddle River, N. J.
32. Shawe-Taylor, J., and N. Cristianini (2004), Kernel Methods for Pattern Analysis, Cambridge Univ. Press, Cambridge, U. K.
33. Sherstov, A., and P. Stone (2005). Function approximation via tile coding: Automating parameter choice, in Proceedings of the 6th International Symposium on Abstraction, Reformulation and Approximation (SARA-05), Lect. Notes Comput. Sci., vol. 3607, pp. 194-205, Springer, Berlin.
34. Smola, A. J., and B. Schölkopf (2004), A tutorial on support vector regression, Stat. Comput., 14(3), 199-222. (Pubitemid 39063488)
35. Sutton, R. S., and A. G. Barto (1998), Reinforcement Learning: An Introduction, MIT Press, Cambridge, Mass.
36. Szepesvári, C. (2010), Algorithms for Reinforcement Learning, Morgan and Claypool, San Rafael, Calif.
37. Tesauro, G. (1995), Temporal difference learning and TD-Gammon, Commun. ACM, 38(3), 58-68.
38. Tizhoosh, H. R., and M. Ventresca (2008), Oppositional Concepts in Computational Intelligence, Springer, Berlin.
39. U.S. Army Corps of Engineers (USACE) (1995), Lower Truckee River, Nevada, reconnaissance report, Washington, D. C.
40. U.S. Department of the Interior (USDOI) (2002), Final environmental impact statement: Truckee River water quality settlement agreement, Fed. Water Rights Acquis. Program, Phoenix, Ariz.
41. U.S. Department of the Interior (USDOI) (2008), Final environmental impact statement, environmental impact report: Truckee River operating agreement, Washington, D. C.
42. Warwick, J. J., D. Cockrum, and A. McKay (1999), Modeling the impact of subsurface nutrient flux on water quality in the Lower Truckee River, Nevada, J. Am. Water Resour Assoc., 35(4), 837-851. (Pubitemid 29388499)
43. Watkins, C. J. (1989), Learning from delayed rewards, PhD thesis, Cambridge Univ., Cambridge, U. K.
44. Whiteson, S., and P. Stone (2006), Evolutionary function approximation for reinforcement learning, J. Mach. Learn. Res., 7, 877-917. (Pubitemid 43736560)
45. Wilson, G. (1996), Reinforcement learning: A new technique for the realtime optimal control of hydraulic networks, in Hydroinformatics '96, vol. 2, pp. 893-900, Taylor and Francis, London, U. K.
46. Wurbs, R. A. (1993), Reservoir system simulation and optimization models, J. Water Resour. Plann. Manage., 119(4), 455-472. (Pubitemid 24394999)
47. Yeh, W.-G. (1985), Reservoir management and operations models: A state-of-the-art review, Water Resour. Res., 21(12), 1797-1818.
48. Zagona, E. A. (2001), Riverware: A generalized tool for complex reservoir system modeling, J. Am. Water Resour. Assoc., 37(4), 913-929. (Pubitemid 32948843)