Stochastic optimization of multireservoir systems via reinforcement learning

Water Resources Research

Volumn 43, Issue 11, 2007, Pages

  Lee, Jin Hee ^a   Labadie, John W ^b  

^a Inha University   (South Korea)

^b Engineering Research Center and Hydraulics Laboratory   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

DYNAMIC PROGRAMMING; MARKOV PROCESSES; MONTE CARLO METHODS; OPTIMAL SYSTEMS; PROBLEM SOLVING; REINFORCEMENT LEARNING; STOCHASTIC CONTROL SYSTEMS;

HIGH-DIMENSIONAL STATE SPACE; MARKOV DECISION PROCESSES; MULTIRESERVOIR SYSTEMS; Q-LEARNING METHOD; STOCHASTIC OPTIMIZATION;

RESERVOIRS (WATER);

DYNAMIC PROGRAMMING; MARKOV PROCESSES; MONTE CARLO METHODS; OPTIMAL SYSTEMS; PROBLEM SOLVING; REINFORCEMENT LEARNING; RESERVOIRS (WATER); STOCHASTIC CONTROL SYSTEMS;

ARTIFICIAL INTELLIGENCE; COGNITION; LEARNING; MONTE CARLO ANALYSIS; OPTIMIZATION; RESEARCH WORK; SIMULATION; STOCHASTICITY;

ASIA; EURASIA; FAR EAST; KOREA; KUM RIVER; SOUTH KOREA;

GEUM;

EID: 37549024571     PISSN: 00431397     EISSN: None     Source Type: Journal    
DOI: 10.1029/2006WR005627     Document Type: Article

References (36)

1. Bellman, R. (1957), Dynamic Programming, Princeton Univ. Press, Princeton, N. J.
2. Bellman, R., and S. Dreyfus (1962), Applied Dynamic Programming, Princeton Univ. Press, Princeton, N. J.
3. Bertsekas, D., and J. Tsitsiklis (1996), Neuro-dynamic Programming, Athena Sci., Belmont, Mass.
4. Birge, J. (1985), Decomposition and partitioning methods for multi-stage stochastic linear programs, Oper. Res., 33, 989-1007.
5. Braga, B., W. Yeh, L. Becker, and M. Barros (1991), Stochastic optimization of multiple reservoir system operation, J. Water Resour. Plann. Manage., 117(4), 471-481.
6. Chandramouli, V., and H. Raman (2001), Multireservoir modeling with dynamic programming and neural networks, J. Water Resour. Plann. Manage., 127(2), 89-98.
7. El-Awar, F., J. Labadie, and T. Ouarda (1998), Stochastic differential dynamic programming for multi-reservoir system control, J. Stochastic Hydrol Hydraul., 12, 247-266.
8. Faber, B., and J. Stedinger (2001), Reservoir optimization using sampling SDP with ensemble streamflow prediction (ESP) forecasts, J. Hydrol., 249(1-4), 113-133.
9. Foufoula-Georgiou, E. (1991), Convex interpolation for gradient dynamic-programming, Water Resour. Res., 27(1), 31-36.
10. Grygier, J., and J. Stedinger (1990), SPIGOT: A synthetic streamflow generation software package, technical description, version 2.5, Sch. of Civil and Environ. Eng., Cornell Univ., Ithaca, N. Y.
11. Hastie, T., R. Tibshirani, and J. Friedman (2001), The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Springer, New York.
12. Howard, R. (1960), Dynamic Programming and Markov Processes, MIT Press, Cambridge, Mass.
13. Jacobson, H., and Q. Mayne (1970), Differential Dynamic Programming, Elsevier, New York.
14. Johnson, S., J. Stedinger, C. Shoemaker, Y. Li, and J. Tejada-Guibert (1993), Numerical-solution of continuous-state dynamic programs using linear and spline interpolation, Oper. Res., 41(3), 484-500.
15. Kaelbling, L., M. Littman, and A. Moore (1996), Reinforcement learning: A survey, J. Artif Intel. Res., 4, 237-285.
16. Kelman, J., R. Stedinger, L. Cooper, E. Hsu, and S.-Q. Yuan (1990), Sampling stochastic dynamic programming applied to reservoir operation, Water Resour. Res., 26(3), 447-454.
17. Klemeš, V. (1977), Discrete representation of storage for stochastic reservoir optimization, Water Resour. Res., 13(1), 149-158.
18. Korea Water Resources Corporation (KWater) (2003), Geum River basin operational guidelines for MODSIM, technical report, Daejeon, South Korea.
19. Labadie, J. (2003), Generalized dynamic programming package CSUDP: Documentation and user guide, version 3.2a, Dep. of Civ. and Environ. Eng., Colo. State Univ., Ft. Collins.
20. Labadie, J. (2004), Optimal operation of multi-reservoir systems: State-of-the-art review, J. Water Resour. Plann. Manage., 130(2), 93-111.
21. Lane, W. (1982), Corrected parameter estimates for disaggregation schemes, in Statistical Analysis of Rainfall and Runoff, edited by V. P. Singh, pp. 505-530, Water Resour. Publ., Littleton, Colo.
22. Lane, W., and D. Frevert (1990), Applied stochastic techniques, users manual, version 5.2, U.S. Bur. of Reclamation, Denver, Colo.
23. Larson, R. (1968), State Increment Dynamic Programming, Elsevier, New York.
24. Lee, J.-H. (2005), Basin-wide multi-reservoir operation using reinforcement learning, Ph.D. dissertation, Dep. of Civ. and Environ. Eng., Colo. State Univ., Ft. Collins.
25. Mejia, J., and J. Rousselle (1976), Disaggregation models in hydrology revisited, Water Resour. Res., 12(2), 185-186.
26. Pereira, M., and L. Pinto (1991), Multi-stage stochastic optimization applied to energy planning, Math. Program., 52, 359-375.
27. Salas, J. (1993), Analysis and modeling of hydrologic time series, in Handbook of Hydrology, edited by D. R. Maidment, pp. 19.5-19.9, McGraw-Hill, New York.
28. Salas, J., O. Sveinsson, W. Lane, and D. Frevert (2006), Stochastic stream-flow simulation using SAMS-2003, J. Irrig. Drain. Eng., 132(2), 112-122.
29. Sherkat, V., R. Campo, K. Moslehi, and E. Lo (1985), Stochastic long-term hydro-thermal optimization for a multireservoir systems, IEEE Trans. Power Apparatus Syst., PAS-104(8), 2040-2050.
30. Shrestha, B., L. Duckstein, and E. Stakhiv (1996), Fuzzy rule-based modeling of reservoir operation, J. Water Resour. Plann. Manage., 122(4), 262-269.
31. Sutton, R., and A. Barto (2000), Reinforcement Learning: An Introduction, MIT Press, Cambridge, Mass.
32. Tejada-Guibert, J., S. Johnson, and J. Stedinger (1995), The value of hydrologic information in stochastic dynamic programming models of a multi-reservoir system, Water Resour. Res., 3(10), 2571-2579.
33. Valencia, D., and J. Schaake (1973), Disaggregation processes in stochastic hydrology, Water Resour. Res., 9(3), 580-585.
34. Wang, D., and B. Adams (1986), Optimization of real-time reservoir operations with Markov decision processes, Water Resour. Res., 22(3), 345-352.
35. Watkins, C., and P. Dayan (1992), Technical note: Q-Learning, Machine Learning, 8, 279-292.
36. Young, G. (1967), Finding reservoir operating rules, J. Hydraul. Div. Am. Soc. Civ. Eng., 93(6), 297-321.