Optimizing long-term hydro-power production using markov decision processes

International Transactions in Operational Research

Volumn 3, Issue 3-4, 1996, Pages 223-241

  Lamond, B F ^a   Boukhtouta, A ^a  

^a UNIVERSITÉ LAVAL   (Canada)

Author keywords

Hydro power; Markov decision processes; Reservoir management; Stochastic dynamic programming; Stochastic optimization

Indexed keywords

EID: 0242692026     PISSN: 09696016     EISSN: 14753995     Source Type: Journal    
DOI: 10.1111/j.1475-3995.1996.tb00049.x     Document Type: Article

References (85)

1. Aho, A. V., Hopcroft, J. E. & Ullman, J. D. (1974). The Design and Analysis of Computer Algorithms. Addison-Wesley, Massachusetts.
2. Archibald, T. W., McKinnon, K. I. M. & Thomas, L. C. (1993). Serial and parallel value iteration algorithms for discounted Markov decision processes. European J: Of Operational Research 67, 188-1203.
3. Archibald, T. W., McKinnon, K. I. M. & Thomas, L. C. (1995). An aggregate stochastic dynamic programming model of multiple reservoir systems. Technical report, Department of Business Studies, The University of Edinburgh, Edinburgh.
4. Arunkumar, S. (1975). Characterization of optimal operating policies for finite dams. J. Math. Anal. Appl. 49, 267-274.
5. Arunkumar, S. (1977): Optimal regulation policies for a multi-purpose reservoir with seasonal input and return function. J: Optim. Theory Appl. 21, 319-328.
6. Arvanitidis, N.V. & Rosing, J. (1970a). Composite representation of a multireservoir hydroelectric power system. IEEE Transactions on Power Apparatus and Systems PAS-89, 319-326.
7. Arvanitidis, N.V. & Rosing, J. (1970b): Optimal operation of multireservoir systems using a composite representation. IEEE Transactions on Power Apparatus and Systems PAS-89, 327-335.
8. Bannister, C. H. & Kaye, R. J. (1991). A rapid method for optimization of linear systems with storage: Operations Research 39, 2210-232.
9. Bather, J. A. (1962): Optimal regulation policies for finite dams. J. Soc. Ind. Appl. Math 10, 395-423.
10. Bellman, R. E. (1954). The theory of dynamic programming. Bull. Amer. Math. Soc. 60, 503-516.
11. Bellman, R. E. (1957). Dynamic Programming. Princeton University Press, Princeton, NJ.
12. Bellman, R. E. & Dreyfus, S. (1962). Applied Dynamic Programming. Princeton University Press, Princeton, NJ.
13. Bertsekas, D. P. (1976). Dynamic Programming and Stochastic Control. Academic Press, New York.
14. Birnbaum, I. & Lapidus, L. (1978). Studies in approximation methods—I. Splines and control via discrete dynamic programming. Chem. Eng. Sci. 33, 415-426.
15. Blackwell, D. (1965). Discounted dynamic programming. Ann. Math. Stat. 36, 226-235.
16. Camacho, R., McLeod, A. I. & Hipel, K. W. (1985). Contemporaneous autoregressive-moving average (CARMA) modelling in water resources. Water Resources Bulletin 21, 709-720.
17. Daniel, J. W. (1976). Splines and efficiency in dynamic programming. J. Math. Anal. Appl. 54, 402-407.
18. Davis, R. E. (1972). Stochastic dynamic programming for multi-reservoir hydro-optimization. Technical Report 15, Syst. Contr., Palo Alto, Calif.
19. Denardo, E. V. (1982). Dynamic Programming: Models and Applications Prentice-Hall, NJ.
20. Drouin, N., Gautier, A., Lamond, B. F. & Lang, P. (1996). Piecewise affine approximations for the control of a one-reservoir hydroelectric system. European J: Of Operational Research 89, 53-69.
21. Eschenback, E. A., Shoemaker, C. A. & Caffey, H. M. (1995). Parallel algorithms for stochastic dynamic programming with continuous state and control variables: ORSA Journal on Computing 7, 388-401.
22. Foufoula-Georgiou, E. & Kitanidis, P. K. (1988). Gradient dynamic programming for stochastic optimal control of multidimensional water resources systems. Water Resources Research 24, 1345-1359.
23. Gal, S. (1979): Optimal management of a multireservoir water supply system. Water Resources Research 15, 737-749.
24. Gal, S. (1989). The parameter iteration method in dynamic programming. Management Science 35, 675-684.
25. Georgakakos, A. P. (1989). Extended linear quadratic gaussian control for the real time operation of reservoir systems. In A. O. Esobue (Ed) Dynamic Programming for Optimal Water Resources Systems Analysis pp. 329-360. Prentice Hall, Englewood Cliffs, NJ.
26. Georgakakos, A. P. & Marks, D. H. (1987). A new method for the real-time operation of reservoir systems. Water Resources Research 23, 1376-1390.
27. Gessford, J. & Karlin, S. (1958): Optimal policy for hydroelectric operations. In Studies on the Mathematical Theory of Inventory and Production pp. 179-200. Stanford University Press, Stanford, Calif.
28. Goulter, I. C. & Tai, F.-K. (1988). Practical implications in the use of stochastic dynamic programming for reservoir operation. Water Resources Bulletin 21, 65-74.
29. Grygier, J. C. & Stedinger, J. R. (1988). Algorithms for optimizing hydropower system operation. Water Resources Research 21, 1-10.
30. Hanscom, M. A., Lafond, L., Lasdon, L. & Pronovost, G. (1980). Modelling and resolution of the medium term energy generation planning problem for a large hydro-electric system. Management Science 26, 659-668.
31. Hinderer, K. (1991). Increasing Lipschitz continuous maximizers of some dynamic programs. Annals of Operations Research 29, 565-586.
32. Jaccard, Y. (1989). Réduction de la complexité dans la gestion d’un grand système hydro-électrique. Master’s thesis, Université Laval, Québec, Canada.
33. Johnson, S. A., Stedinger, J. R. & Staschus, K. (1991). Heuristic operating policies for reservoir system simulation. Water Resources Research 27, 673-685.
34. Johnson, S. A., Stedinger, J. R., Li, Y., Shoemaker, C. A. & Tejada-Guibert, J. A. (1993). Numerical solution of continuous-state dynamic programs using linear and spline interpolation: Operations Research 41, 484-500.
35. Kelman, J., Stedinger, J. R., Cooper, L. A., Hsu, E. & Yuan, S.-Q. (1990). Sampling stochastic dynamic programming applied to reservoir operation. Water Resources Research 26, 447-454.
36. Kitanidis, P. K. & Foufoula-Georgiou, E. (1987). Error analysis of conventional discrete and gradient dynamic programming. Water Resources Research 23, 845-856.
37. Labadie, J. W. & Fontane, D. G. (1989): Objective-space dynamic programming approach to multidimensional problems in water resources. In A: O. Esogbue (Ed) Dynamic Programming for Optimal Water Resources Systems Analysis pp. 388-401. Prentice Hall, Englewood Cliffs, NJ.
38. Lamond, B. F. & Bachar, B. (1996). Une étude numérique de la discrétisation des apports aléatoires pour un réservoir non-saisonnier. Technical report, Faculté des sciences de l’administration, Université Laval, Québec, Canada.
39. Lamond, B. F. & Lang, P. (1996). Lower bounding aggregation and direct computation for an infinite horizon one-reservoir model. European J: Of Operational Research (to appear).
40. Lamond, B. F. & Sobel, M.J. (1995). Exact and approximate solutions of affine reservoir models: Operations Research 43, 771-780.
41. Lamond, B. F., Monroe, S. L. & Sobel, M.J. (1995). A reservoir hydroelectric system: Exactly and approximately optimal policies. European J: Of Operational Research 81, 535-542.
42. Larson, R. E. (1968). State Increment Dynamic Programming. Elsevier, New York.
43. Little, J. D. C. (1955). The use of storage water in a hydroelectric system: Operations Research 3, 187-197.
44. Loaiciga, H. A. & Mariño, M. A. (1985). An approach to parameter estimation and stochastic control in water resources with an application to reservoir operation. Water Resources Research 21, 1575-1584.
45. Loucks, D. P. & Sigvaldason, O.T. (1982). Multiple-reservoir operation in North America. In Z. Kaczmarek and J. Kindler (Eds) The Operation of Multiple Reservoir Systems pp. 2-103. IIASA Laxenburg, Austria.
46. Mariño, M. A. & Loaiciga, H. A. (1985a). Dynamic model for multireservoir operation. Water Resources Research 21, 619-630.
47. Mariño, M. A. & Loaiciga, H. A. (1985b): Optimal operation of a multiple reservoir system. Technical report, California Water Resources Center, University of California, Davis, CA.
48. Mariño, M. A. & Loaiciga, H. A. (1985c). Quadratic model for reservoir management: Application to the central valley project. Water Resources Research 21, 631-641.
49. Massé, P. B. D. (1946). Les Réserves et la Régulation de I’Avenir dans la Vie Économique. Hermann, Paris.
50. McLaughlin, D. & Velasco, H. L. (1990). Real-time control of a system of large hydropower reservoirs. Water Resources Research 26, 623-635.
51. Nemhauser, G. L. (1966). Introduction to Dynamic Programming. Wiley, New York.
52. Pereira, M. V. F. (1989): Optimal stochastic operations scheduling or large hydroelectric systems. Electrical Power and Energy Systems 11, 161-169.
53. Pereira, M. V. F. & Pinto, L. M. V. G. (1985). Stochastic optimization of a multireservoir hydroelectric system: A decomposition approach. Water Resources Research 21, 779-792.
54. Pereira, M. V. F. & Pinto, L. M.V. G. (1991). Multi-stage stochastic optimization applied to energy planning. Mathematical Programming 52, 359-375.
55. Priestley, M. B. (1981). Spectral Analysis and Time Series, Vol. 2. Academic Press.
56. Puterman, M.L. (1994). Markov Decision Processes: Discrete Stochastic Dynamic Programming.
57. Read, E. G. (1989). A dual approach to stochastic programming for reservoir release scheduling. In A: O. Esogbue (Ed) Dynamic Programming for Optimal Water Resources Systems Analysis pp. 361-372. Prentice Hall, Englewood Cliffs, NJ.
58. Read, E. G. & George, J. A. (1990). Dual dynamic programming for linear production/inventory systems. Computers Match. Applic. 19, 29-42.
59. Reznicek, K. & Cheng, T. C. E. (1991). Stochastic modelling of reservoir operations. European J: Of Operational Research 50, 235-248.
60. 1 martingales in stochastic optimization problems. Mathematical Programming Study 2, 170-187.
61. Saad, M. & Turgeon, A. (1988). Application of principal component analysis to long-term reservoir management. Water Resources Research 24, 907-912.
62. Saad, M., Turgeon, A. & Stedinger, J. R. (1992). Censored-data correlation and principal component dynamic programming. Water Resources Research 28, 2135-2140.
63. Saad, M., Turgeon, A., Bigras, P. & Duquette, R. (1994). Learning disaggregation technique for the operation of long-term hydroelectric power systems. Water Resources Research 30, 3195-3202.
64. Salas, J. D., Tabios, III G. Q. & Bartolini, P. (1985). Approaches to multivariate modelling of water resources time series. Water Resources Bulletin 21, 683-708.
65. Sharda, R. & El-Tayeb, M. A. (1989). An adaptive control model for single reservoir operation. In A: O. Esogbue (Ed) Dynamic Programming for Optimal Water Resources Systems Analysis pp. 316-328. Prentice Hall, Englewood Cliffs, NJ.
66. Soares, S. & Carneiro, A. A. F. M. (1990): Optimal operation of reservoirs for electric generation. In Proceedings of the IEEE Power Engineering Society 1990 Summer Meeting, Minneapolis, Minnesota, July.
67. Sobel, M. J. (1975). Reservoir management models. Water Resources Research 11, 767-776.
68. Sobel, M. J. (1989). A multi-reservoir model with a myopic optimum. In A: O. Esogbue (Ed) Dynamic Programming for Optimal Water Resources Systems Analysis pp. 309-315. Prentice-Hall, Englewood Cliffs, NJ.
69. Sobel, M. J. (1990). Higher-order and average reward myopic-affine dynamic models. Mathematics of Operations Research 15, 299-310.
70. Sobel, M. J. (1991). Myopic solutions of affine dynamic models: Operations Research 38, 847-853.
71. Soliman, S. A. & Christensen, G. S. (1988): Optimization of reservoirs in series on a river with a nonlinear storage curve for long-term regulation. J: Of Optimization Theory and Applications 58, 109-126.
72. Stedinger, J. R., Lettenmaier, D. P. & Vogel, R. M. (1985). Multisite ARMA (1,1) and disaggregation models for annual streamflow gener ation. Water Resources Research 21, 497-509.
73. Stedinger, J.R., Sule, B. F. & Loucks, D. P. (1984). Stochastic dynamic programming models for reservoir operation optimization. Water Resources Research 20, 1499-1505.
74. Tejada-Guibert, J. A., Johnson, S. A. & Stedinger, J. R. (1993). Comparison of two approaches for implementing multireservoir operating policies derived using stochastic dynamic programming. Water Resources Research 29, 3969-3980.
75. Terry, L.A., Pereira, M.V.F., Araripe Neto, T.A., Silva, L.F.C.A. & Sales, P.R.H. (1986). Coordinating the energy generation of the Brazilian national hydrothermal electrical generating system. Interfaces 16, 16-38.
76. Trezos, T. & Yeh, W. W.-G. (1987). Use of stochastic dynamic programming for reservoir management. Water Resources Research 23, 983-996.
77. Turgeon, A. (1980): Optimal operation of multireservoir power systems with stochastic inflows. Water Resources Research 16, 275-283.
78. Turgeon, A. (1981). A decomposition method for the long-term scheduling of reservoirs in series. Water Resources Research 17, 1565-1570.
79. Turgeon, A. (1992). PERESE: The new Hydro-Québec optimization model for the long-term scheduling of reservoirs. In E. Broch and D. K. Lysne (Eds) Hydropower 1992: Proceedings of the 2nd International Conference on Hydropower pp. 603-612. Lillehamer, Norway, June. Balkema, Rotterdam.
80. Wald, A. (1947). Sequential Analysis. Wiley, New York.
81. Wasimi, S. A. & Kitanidis, P. K. (1983). Real-time forecasting and daily operation of a multireservoir system during floods by linear quadratic gaussian control. Water Resources Research 19, 1511-1522.
82. White, D. J. (1969). Dynamic Programming. Holden-Day, San Francisco.
83. White, D. J. (1993). Markov Decision Processes Wiley, New York.
84. Yakowitz, S. (1982). Dynamic programming applications in water resources. Water Resources Research 18, 7673-696.
85. Yeh, W. W.-G. (1985). Reservoir management and operations models: A state-of-the-art review. Water Resources Research 21, 1797-1818.