Many-objective reservoir policy identification and refinement to reduce policy inertia and myopia in water management

Water Resources Research

Volumn 50, Issue 4, 2014, Pages 3355-3377

  Giuliani, M ^a   Herman, J D ^b   Castelletti, A ^a   Reed, P ^b  

^a POLITECNICO DI MILANO   (Italy)

^b Cornell University   (United States)

Author keywords

direct policy search; evolutionary algorithm; multiobjective optimization; reservoir operation; visualization

Indexed keywords

COMMERCE; DECISION MAKING; EVOLUTIONARY ALGORITHMS; FLOW VISUALIZATION; MULTIOBJECTIVE OPTIMIZATION; OPHTHALMOLOGY; VISUALIZATION; WATER MANAGEMENT;

DIRECT POLICY SEARCH; HYDRO POWER PRODUCTION; MANAGEMENT EFFORTS; MANY-OBJECTIVE OPTIMIZATIONS; RESERVOIR OPERATION; RIVER BASIN MANAGEMENT; SYSTEM UNCERTAINTIES; URBAN WATER SUPPLY;

RESERVOIRS (WATER);

ALGORITHM; DECISION ANALYSIS; MULTIOBJECTIVE PROGRAMMING; POLICY ANALYSIS; RESERVOIR; TRADE-OFF; VISUALIZATION; WATER DEMAND; WATER MANAGEMENT; WATER PLANNING; WATER SUPPLY;

CONOWINGO RESERVOIR; SUSQUEHANNA RIVER; UNITED STATES;

EID: 84899038091     PISSN: 00431397     EISSN: 19447973     Source Type: Journal    
DOI: 10.1002/2013WR014700     Document Type: Article

References (105)

1. Back, T., D. Fogel, and, Z. Michalewicz, (2000), Handbook of Evolutionary Computation, Oxford Univ. Press, Bristol, U. K.
2. Baglietto, M., C. Cervellera, M. Sanguineti, and, R. Zoppoli, (2006), Water reservoirs management under uncertainty by approximating networks and learning from data, in Topics on System Analysis and Integrated Water Resource Management, edited by, A. Castelletti, and, R. Soncini-Sessa, pp. 117-139, Elsevier, Amsterdam, NL.
3. Balling, R., J. Taber, M. Brown, and, K. Day, (1999), Multiobjective urban planning using genetic algorithm, J. Urban Plann. Dev., 125 (2), 86-99. (Pubitemid 30208721)
4. Barron, A., (1993), Universal approximation bounds for superpositions of a sigmoidal function, IEEE Trans. Inform. Theory, 39 (3), 930-945.
5. Bellman, R., (1957), Dynamic Programming, Princeton Univ. Press, Princeton, NJ.
6. Box, G., and, G. Jenkins, (1970), Time Series Analysis: Forecasting and Control, Holden-Day, San Francisco, Calif.
7. Brill, E., J. Flach, L. Hopkins, and, S. Ranjithan, (1990), MGA: A decision support system for complex, incompletely defined problems, IEEE Trans. Syst. Man Cybern., 20 (4), 745-757. (Pubitemid 20739305)
8. Brown, C., and, M. Carriquiry, (2007), Managing hydroclimatological risk to water supply with option contracts and reservoir index insurance, Water Resour. Res., 43, W11423, doi: 10.1029/2007WR006093.
9. Busoniu, L., D. Ernst, B. De Schutter, and, R. Babuska, (2011), Cross-entropy optimization of control policies with adaptive basis functions, IEEE Trans. Syst. Man Cybern., 41 (1), 196-209, doi: 10.1109/TSMCB.2010.2050586.
10. Castelletti, A., F. Pianosi, and, R. Soncini-Sessa, (2008), Water reservoir control under economic, social and environmental constraints, Automatica, 44 (6), 1595-1607.
11. Castelletti, A., S. Galelli, M. Restelli, and, R. Soncini-Sessa, (2010), Tree-based reinforcement learning for optimal water reservoir operation, Water Resour. Res., 46, W09507, doi: 10.1029/2009WR008898.
12. Castelletti, A., F. Pianosi, X. Quach, and, R. Soncini-Sessa, (2012), Assessing water reservoirs management and development in Northern Vietnam, Hydrol. Earth Syst. Sci., 16 (1), 189-199, doi:10.5194/hess-16-189-2012.
13. Castelletti, A., F. Pianosi, and, M. Restelli, (2013), A multiobjective reinforcement learning approach to water resources systems operation: Pareto frontier approximation in a single run, Water Resour. Res., 49, 3476-3486, doi: 10.1002/wrcr.20295.
14. Celeste, A., and, M. Billib, (2009), Evaluation of stochastic reservoir operation optimization models, Adv. Water Resour., 32 (9), 1429-1443, doi: 10.1016/j.advwatres.2009.06.008.
15. Cervellera, C., V. Chen, and, A. Wen, (2006), Optimization of a large-scale water reservoir network by stochastic dynamic programming with efficient state space discretization, Eur. J. Oper. Res., 171 (3), 1139-1151. (Pubitemid 43132968)
16. Chan Hilton, A. B., and, T. Culver, (2005), Groundwater remediation design under uncertainty using genetic algorithms, J. Water Resour. Plann. Manage., 131 (1), 25-34. (Pubitemid 40106705)
17. Chang, F., L. Chen, and, L. Chang, (2005), Optimizing the reservoir operating rule curves by genetic algorithms, Hydrol. Processes, 19 (11), 2277-2289. (Pubitemid 41052589)
18. Chankong, V., and, Y. Haimes, (1983), Multiobjective Decision Making: Theory and Methodology, North-Holland, New York.
19. Clark, E., (1950), New York control curves, J. Am. Water Works Assoc., 42 (9), 823-827.
20. Clark, E., (1956), Impounding reservoirs, J. Am. Water Works Assoc., 48 (4), 349-354.
21. Coello Coello, C., G. Lamont, and, D. V. Veldhuizen, (2007), Evolutionary Algorithms for Solving Multi-Objective Problems (Genetic Algorithms and Evolutionary Computation), 2nd ed., Springer, New York.
22. Cohon, J. L., and, D. Marks, (1975), A review and evaluation of multiobjective programming techniques, Water Resour. Res., 11 (2), 208-220.
23. Corani, G., A. Rizzoli, A. Salvetti, and, M. Zaffalon, (2009), Reproducing human decisions in reservoir management: The case of lake Lugano, ICSC, paper presented at 4th International ICSC Symposium on Information Technologies in Environmental Engineering, Thessaloniki, Greece.
24. Cui, L., and, G. Kuczera, (2005), Optimizing water supply headworks operating rules under stochastic inputs: Assessment of genetic algorithm performance, Water Resour. Res., 41, W05016, doi: 10.1029/2004WR003517. (Pubitemid 40856466)
25. Draper, A., and, J. Lund, (2004), Optimal hedging and carryover storage value, J. Water Resour. Plann. Manage., 130 (1), 83-87, doi:10.1061/(ASCE)0733- 9496(2004)130:1(83). (Pubitemid 38122781)
26. Exelon (2010), Operations modeling calibration report, Technical Report, Addendum to Conowingo hydroelctric project-RSP 3.11 (405) and Muddy Run Pumped Storage Project-RSP 3.2 (2355), Gomez and Sullivan Eng.
27. Federal Energy Regulatory Commission (1989), Order approving settlement agreement for the Conowingo hydroelectric project, Tech. Rep. Proj. 405-009, Docket EL80-38-000.
28. Fernandez, A., S. Blumsack, and, P. Reed, (2012), Evaluating wind-following and ecosystem services for hydroelectric dams in PJM, J. Regul. Econ., 41 (1), 139-154, doi: 10.1007/s11149-011-9177-9.
29. Fernandez, A., S. Blumsack, and, P. M. Reed, (2013), Operational constraints and hydrologic variability limit hydropower in supporting wind integration, Environ. Res. Lett., 8 (2), 024037, doi:10.1088/1748-9326/8/2/ 024037.
30. Fleming, P., R. Purshouse, and, R. Lygoe, (2005), Many-objective optimization: An engineering design perspective, in Proceedings of the Third International Conference on Evolutionary Multi-Criterion Optimization, edited by, Coello, Aguirre, and, Zitzler, pp. 14-32, Springler-Verlag, Berlin. (Pubitemid 41251753)
31. Gass, S., and, T. Saaty, (1955), Parametric objective function. Part II, J. Oper. Res. Soc. Am., 3, 316-319.
32. Gettys, C., and, S. Fisher, (1979), Hypothesis plausibility and hypothesis generation, Organ. Behavior Hum. Perform., 24 (1), 93-110.
33. Gleick, P., (2002), Water management-Soft water paths, Nature, 418 (6896), 373, doi: 10.1038/418373a. (Pubitemid 34826824)
34. Goldberg, D., (1989), Genetic Algorithms in Search, Optimization and Machine Learning, Addison-Wesley, Boston, Mass.
35. Guariso, G., S. Rinaldi, and, R. Soncini-Sessa, (1986), The management of Lake Como: A multiobjective analysis, Water Resour. Res., 22 (2), 109-120, doi: 10.1029/WR022i002p00109.
36. Guo, X., T. Hu, X. Zeng, and, X. Li, (2013), Extension of parametric rule with the hedging rule for managing multireservoir system during droughts, J. Water Resour. Plann. Manage., 139 (2), 139-148, doi:10.1061/(ASCE)WR.1943-5452. 0000241.
37. Hadka, D., and, P. Reed, (2012), Diagnostic assessment of search controls and failure modes in many-objective evolutionary optimization, Evol. Comput., 20 (3), 423-452.
38. Hadka, D., and, P. Reed, (2013), Borg: An auto-adaptive many-objective evolutionary computing framework, Evol. Comput., 21 (2), 231-259.
39. Haimes, Y., and, W. Hall, (1977), Sensitivity, responsivity, stability and irreversibility as multiple objectives in civil systems, Adv. Water Resour., 1 (2), 71-81.
40. Haimes, Y., K. Tarvainen, T. Shima, and, J. Thadathil, (1990), Hierarchical Multiobjective Analysis of Large-Scale Systems, Hemisphere, New York.
41. Hammond, K., R. Hamm, J. Grassia, and, T. Pearson, (1987), Direct comparison of the efficacy of intuitive and analytical cognition in expert judgment, IEEE Trans. Syst. Man Cybern., 17 (5), 753-770. (Pubitemid 18537455)
42. Hashimoto, T., J. Stedinger, and, D. Loucks, (1982), Reliability, resilience, and vulnerability criteria for water resource system performance evaluation, Water Resour. Res., 18 (1), 14-20, doi: 10.1029/WR018i001p00014. (Pubitemid 12276431)
43. Hejazi, M., X. Cai, and, B. Ruddell, (2008), The role of hydrologic information in reservoir operation-Learning from historical releases, Adv. Water Resour., 31 (12), 1636-1650.
44. Hogarth, R., (1981), Beyond discrete biases: Functional and dysfunctional aspects of judgemental heuristics, Psychol. Bull., 90 (1), 197-217.
45. Howard, C., (1999), Death to Rule Curves, chap. 231, pp. 1-5, ASCE, Tempe, AZ, doi: 10.1061/40430(1999)232.
46. Hurwicz, L., (1951), Optimality criteria for decision making under ignorance, Cowles Comm. Pap. 370.
47. Inselberg, A., (1997), Multidimensional detective, in Proceedings of the IEEE Symposium on Information Visualization, edited by, Dill, and, Gershon, pp. 100-107, IEEE, Phoenix, AZ.
48. Jalali, M., A. Afshar, and, M. Marino, (2006), Reservoir operation by ant colony optimization algorithms, Iran. J. Sci. Technol. Trans. B, 30 (B1), 107-117. (Pubitemid 43870830)
49. Johnson, S., J. Stedinger, and, K. Staschus, (1991), Heuristic operating policies for reservoir system simulation, Water Resour. Res., 27 (5), 673-685.
50. Kasprzyk, J., P. Reed, B. Kirsch, and, G. Characklis, (2009), Managing population and drought risks using many-objective water portfolio planning under uncertainty, Water Resour. Res., 45, W12401, doi: 10.1029/2009WR008121.
51. Kasprzyk, J., P. Reed, G. Characklis, and, B. Kirsch, (2012), Many-objective de Novo water supply portfolio planning under deep uncertainty, Environ. Modell. Software, 34, 87-104, doi: 10.1016/j.envsoft.2011.04.003.
52. Kasprzyk, J., S. Nataraj, P. Reed, and, R. Lempert, (2013), Many-objective robust decision making for complex environmental systems undergoing change, Environ. Modell. Software, 42, 55-71.
53. Keim, D., F. Mansmann, J. Schneidewind, and, H. Ziegler, (2006), Challenges in visual data analysis, in Tenth International Conference on Information Visualization, 2006, IV 2006, edited by, Banissi, Burkhard, Ursyn, Zhang, Bannatyne, Maple, Cowell, Tianm, and, Hou, pp. 9-16, IEEE, London.
54. Klemeš, V., (1977), Value of information in reservoir optimization, Water Resour. Res., 13 (5), 837-850.
55. Kollat, J., and, P. Reed, (2007), A framework for Visually Interactive Decision-making and Design using Evolutionary Multi-objective Optimization (VIDEO), Environ. Modell. Software, 22 (12), 1691-1704. (Pubitemid 47211995)
56. Kollat, J., P. Reed, and, R. Maxwell, (2011), Many-objective groundwater monitoring network design using bias-aware ensemble Kalman filtering, evolutionary optimization, and visual analytics, Water Resour. Res., 47, W02529, doi: 10.1029/2010WR009194.
57. Kormushev, P., and, D. Caldwell, (2012), Direct policy search reinforcement learning based on particle filtering, paper presented at 10th European Workshop on Reinforcement Learning, Edinburg, Scotland.
58. Koutsoyiannis, D., and, A. Economou, (2003), Evaluation of the parameterization-simulation-optimization approach for the control of reservoir systems, Water Resour. Res., 39 (6), 1170, doi: 10.1029/2003WR002148.
59. Kurková, V., and, M. Sanguineti, (2001), Bounds on rates of variable-basis and neural-network approximation, IEEE Trans. Inform. Theory, 47 (6), 2659-2665. (Pubitemid 32949922)
60. Labadie, J., (2004), Optimal operation of multireservoir systems: State-of-the-art review, J. Water Resour. Plann. Manage., 130 (2), 93-111. (Pubitemid 38459486)
61. Lall, U., and, A. Sharma, (1996), A nearest neighbor bootstrap for resampling hydrologic time series, Water Resour. Res., 32 (3), 679-693, doi: 10.1029/95WR02966. (Pubitemid 26351903)
62. Laplace, P., (1951), Essai Philosophique sur les Probabilités (1820), Philosophical Essays on Probabilities [English Transl.], Dover, New York.
63. Lotov, A., and, K. Miettinen, (2008), Visualizing the Pareto frontier, in Multiobjective Optimization, edited by, J. Branke, et al., pp. 213-243, Springer, Schloss Dagstuhl, Germany.
64. Loucks, D., and, O. Sigvaldason, (1982), Multiple-reservoir operation in North America, in The Operation of Multiple Reservoir Systems, IIASA Collab. Proc. Ser., edited by, Z. Kaczmarck, and, J. Kindler, pp. 1-103, IIASA, Laxenburg, Austria.
65. Loucks, D., E. van Beek, J. Stedinger, J. Dijkman, and, M. Villars, (2005), Water Resources Systems Planning and Management: An Introduction to Methods, Models and Applications, United Nations Educational Scientific and Cultural Organization (UNESCO), Paris.
66. Louie, P., W. Yeh, and, N. Hsu, (1984), Multiobjective water resources management planning, J. Water Resour. Plann. Manage., 110 (1), 39-56.
67. Lund, J., and, J. Guzman, (1999), Derived operating rules for reservoirs in series or in parallel, J. Water Resour. Plann. Manage., 125 (3), 143-153. (Pubitemid 29377571)
68. Maass, A., M. Hufschmidt, R. Dorfman, H. Thomas Jr,., S. Marglin, and, G. Fair, (1962), Design of Water-Resource Systems, Harvard Univ. Press, Cambridge, Mass.
69. Menache, I., S. Mannor, and, N. Shimkin, (2005), Basis function adaptation in temporal difference reinforcement learning, Ann. Oper. Res., 134 (1), 215-238, doi: 10.1007/s10479-005-5732-z. (Pubitemid 40550047)
70. Miller, B., and, D. Goldberg, (1996), Optimal sampling for genetic algorithms, in Proceedings of the Artificial Neural Networks in Engineering (ANNIE'96) Conference, vol. 6, edited by, St. Luis, and, Missouri, ASME Press, pp. 291-297. (Pubitemid 126629731)
71. Milly, P., J. Betancourt, M. Falkenmark, R. Hirsch, Z. Kundzewicz, D. Lettenmaier, and, R. Stouffer, (2008), Stationarity is dead, Science, 319, 573-574.
72. Momtahen, S., and, A. Dariane, (2007), Direct search approaches using genetic algorithms for optimization of water reservoir operating policies, J. Water Resour. Plann. Manage., 133 (3), 202-209, doi:10.1061/(ASCE)0733- 9496(2007)133:3(202). (Pubitemid 46605588)
73. Momtahen, S., and, A. Dariane, (2009), Optimization of multireservoir systems operation using modified direct search genetic algorithm, J. Water Resour. Plann. Manage., 135 (3), 141-148, doi:10.1061/(ASCE)0733-9496(2009)135: 3(141).
74. Nalbantis, I., and, D. Koutsoyiannis, (1997), A parametric rule for planning and management of multiple-reservoir systems, Water Resour. Res., 33 (9), 2165-2177. (Pubitemid 27378365)
75. Nicklow, J., et al. (2010), State of the art for genetic algorithms and beyond in water resources planning and management, J. Water Resour. Plann. Manage., 136 (4), 412-432, doi:10.1061/(ASCE)WR.1943-5452.0000053.
76. Nowak, K., J. Prairie, B. Rajagopalan, and, U. Lall, (2010), A nonparametric stochastic approach for multisite disaggregation of annual to daily streamflow, Water Resour. Res., 46, W08529, doi: 10.1029/2009WR008530.
77. Oliveira, R., and, D. P. Loucks, (1997), Operating rules for multi reservoir systems, Water Resour. Res., 33 (4), 839-852.
78. Orlovski, S., S. Rinaldi, and, R. Soncini-Sessa, (1984), A min max approach to reservoir management, Water Resour. Res., 20 (11), 1506-1514.
79. Ostadrahimi, L., M. A. Mariño, and, A. Afshar, (2012), Multi-reservoir operation rules: Multi-swarm pso-based optimization approach, Water Resour. Manage., 26 (2), 407-427.
80. Pianosi, F., X. Quach, and, R. Soncini-Sessa, (2011), Artificial Neural Networks and Multi Objective Genetic Algorithms for water resources management: An application to the Hoabinh reservoir in Vietnam, in Proceedings of the 18th IFAC World Congress, edited by, Bittanti, Cenedese, and, Zampieri, pp. 10579-10584, International Federation of Automatic and Control, Milan, Italy.
81. Piccardi, C., and, R. Soncini-Sessa, (1991), Stochastic dynamic programming for reservoir optimal control: Dense discretization and inflow correlation assumption made possible by parallel computing, Water Resour. Res., 27 (5), 729-741.
82. Powell, W., (2007), Approximate Dynamic Programming: Solving the Curse of Dimensionality, Wiley-Interscience, New York.
83. Rajagopalan, B., J. Salas, and, U. Lall, (2010), Stochastic methods for modeling precipitation and streamflow, in Advances in Data-Based Approaches for Hydrologic Modeling and Forecasting, edited by, B. Sivakumar, and, R. Berndtsson, pp. 17-52, World Sci., Singapore.
84. Randall, D., L. Cleland, C. Kuehne, G. Link, and, D. Sheer, (1997), Water supply planning simulation model using mixed-integer linear programming "Engine," J. Water Resour. Plann. Manage., 123 (2), 116-124, doi:10.1061/(ASCE)0733-9496(1997)123:2(116). (Pubitemid 27429051)
85. Reed, P., and, J. Kollat, (2013), Visual analytics clarify the scalability and effectiveness of massively parallel many-objective optimization: A groundwater monitoring design example, Adv. Water Resour., 56, 1-13.
86. Reed, P., D. Hadka, J. Herman, J. Kasprzyk, and, J. Kollat, (2013), Evolutionary multiobjective optimization in water resources: The past, present, and future, Adv. Water Resour., 51, 438-456.
87. Rosenstein, M., and, A. Barto, (2001), Robot weightlifting by direct policy search, in International Joint Conference on Artificial Intelligence, vol. 17, pp. 839-846, Morgan Kaufmann, San Francisco, Calif.
88. Rückstiess, T., F. Sehnke, T. Schaul, D. Wierstra, Y. Sun, and, J. Schmidhuber, (2010), Exploring parameter space in reinforcement learning, Paladyn J. Behavioral Robotics, 1 (1), 14-24.
89. Savage, L., (1951), The theory of statistical decision, J. Am. Stat. Assoc., 46 (253), 55-67.
90. Schmidhuber, J., (2001), Sequential decision making based on direct search, in Sequence Learning, Lecture Notes in Computer Science, vol. 1828, edited by, R. Sun, and, C. Giles, pp. 213-240, Springer, Berlin, doi:10.1007/3-540-44565-X-10. (Pubitemid 33221931)
91. Shanteau, J., (1992), Competence in experts: The role of task characteristics, Organ. Behavior Hum. Decis. Processes, 53 (2), 252-266.
92. Sheer, D., (2010), Dysfunctional water management: Causes and solutions, J. Water Resour. Plann. Manage., 136 (1), 1-4.
93. Sheer, D., and, A. Dehoff, (2009), Science-based collaboration: Finding better ways to operate the Conowingo Pond, J. Am. Water Works Assoc., 101 (6), 20-22.
94. Smalley, J., B. Minsker, and, D. Goldberg, (2000), Risk-based in situ bioremediation design using a noisy genetic algorithm, Water Resour. Res., 36 (10), 3043-3052.
95. Soncini-Sessa, R., A. Castelletti, and, E. Weber, (2007), Integrated and Participatory Water Resources Management: Theory, Elsevier, Amsterdam.
96. Swartz, P., (2006), Conowingo pond management plan, Tech. Rep. 242, Susquehanna River Basin Comm.
97. Teegavarapu, R., and, S. Simonovic, (2001), Optimal operation of water resource systems: Trade-offs between modeling and practical solutions, in Integrated Water Resources Management, vol. 272, pp. 257-263, edited by, M. A. Mariño, and, S. P. Simonovic, IAHS, Davis, CA.
98. Tejada-Guibert, J., S. Johnson, and, J. Stedinger, (1995), The value of hydrologic information in stochastic dynamic programming models of a multireservoir system, Water Resour. Res., 31 (10), 2571-2579. (Pubitemid 26475099)
99. Thomas, J., and, K. Cook, (2005), Illuminating the Path: The Research and Development Agenda for Visual Analytics, IEEE Press, National Visualization and Analytics Center.
100. Tsitsiklis, J., and, B. Van Roy, (1996), Feature-based methods for large scale dynamic programming, Mach. Learn., 22, 59-94. (Pubitemid 126724363)
101. U.S. Army Corps of Engineers (1977), Reservoir System Analysis for Conservation, Hydrologic Engineering Methods for Water Resources Development, Hydrol. Eng. Cent., Davis, Calif.
102. Woodruff, M., P. Reed, and, T. Simpson, (2013), Many objective visual analytics: Rethinking the design of complex engineered systems, Struct. Multidisc. Optim., 48 (1), 1-19, doi: 10.1007/s00158-013-0891-z.
103. Yates, D., S. Gangopadhyay, B. Rajagopalan, and, K. Strzepek, (2003), A technique for generating regional climate scenarios using a nearest-neighbor algorithm, Water Resour. Res., 39 (7), 1199, doi: 10.1029/2002WR001769.
104. Yeh, W., (1985), Reservoir management and operations models: A state of the art review, Water Resour. Res., 21 (12), 1797-1818.
105. Zoppoli, R., M. Sanguineti, and, T. Parisini, (2002), Approximating networks and extended Ritz method for the solution of functional optimization problems, J. Optim. Theory Appl., 112 (2), 403-440. (Pubitemid 39583388)