A multiagent Q-learning-based optimal allocation approach for urban water resource management system

IEEE Transactions on Automation Science and Engineering

Volumn 11, Issue 1, 2014, Pages 204-214

  Ni, Jianjun ^a   Liu, Minghua ^a   Ren, Li ^b   Yang, Simon X ^c  

^a HOHAI UNIVERSITY   (China)

^b HOHAI UNIVERSITY   (China)

^c UNIVERSITY OF GUELPH   (Canada)

Author keywords

Complex system; Multiagent Q learning; Optimal allocation; Urban water resource management

Indexed keywords

EFFECTIVE APPROACHES; MULTI-AGENT Q-LEARNING; OPTIMAL ALLOCATION; Q-LEARNING ALGORITHMS; RESOURCE OPTIMAL ALLOCATION; URBAN WATER RESOURCES; WATER ENVIRONMENT SYSTEMS; WATERRESOURCE MANAGEMENT;

COMPUTATIONAL METHODS; LARGE SCALE SYSTEMS; MULTI AGENT SYSTEMS; OPTIMIZATION; WATER MANAGEMENT;

LEARNING ALGORITHMS;

EID: 84892442931     PISSN: 15455955     EISSN: None     Source Type: Journal    
DOI: 10.1109/TASE.2012.2229978     Document Type: Article

References (40)

1. G.-H.Wei, F. Liu, and L. Ma, "Fuzzy optimization of water resources project scheme based on improved grey relation analysis," in Proc. 3rd Int. Conf. Comput. Res. Develop., Shanghai, China, 2011, vol. 4, pp. 333-336.
2. W. Sun and Z. Zeng, "City optimal allocation of water resources research based on sustainable development," Adv, Mater. Res., vol. 446-449, pp. 2703-2707, 2012.
3. B. Huang, F. Gui, and X. Zhang, "Study of the optimal water resources allocation scenarios in Pingxiang city," Adv. Intell. Soft Comput., vol. 105, pp. 487-493, 2011.
4. Z. Ren, M.-T. Pham, M. Song, D.-H. Kim, and C. S. Koh, "A robust global optimization algorithm of electromagnetic device utilizing gradient index and multi-objective optimization method," IEEE Trans. Magn., vol. 47, no. 5, pp. 1254-1257, May 2011.
5. W. Y. Zhang, S. Zhang, M. Cai, and J. X. Huang, "A new manufacturing resource allocation method for supply chain optimization using extended genetic algorithm," International J. Adv. Manuf. Technol., vol. 53, no. 9-12, pp. 1247-1260, Apr. 2011.
6. S. Rathinam, R. Sengupta, and S. Darbha, "A resource allocation algorithm for multivehicle systems with nonholonomic constraints," IEEE Trans. Autom. Sci. Eng., vol. 4, no. 1, pp. 98-104, Jan. 2007. (Pubitemid 46113505)
7. A. Gosavi, Simulation-Based Optimization: Parametric Optimization Techniques and Reinforcement Learning. Boston, MA: Kluwer Academic, 2003.
8. D. Vengerov, "A reinforcement learning approach to dynamic resource allocation," Eng. Appl. Artif. Intell., vol. 20, no. 3, pp. 383-390, Apr. 2007. (Pubitemid 46389478)
9. C. Bone and S. Dragi?evi?, "Simulation and validation of a reinforcement learning agent-based model for multi-stakeholder forest management," Comput., Environ. Urban Syst., vol. 34, no. 2, pp. 162-174, 2010.
10. C. Song, X. Guan, Q. Zhao, and Y. Ho, "Machine learning approach for determining feasible plans of a remanufacturing system," IEEE Trans. Autom. Sci. Eng., vol. 2, no. 3, pp. 262-275, Jul. 2005. (Pubitemid 41145975)
11. M. Han, H. Du, X. Yang, and Y. Liu, "Research advances on water resources optimal distribution," Procedia Environ. Sci., vol. 2, pp. 1912-1918, 2010.
12. B. Abolpour,M. Javan, and M. Karamouz, "Water allocation improvement in river basin using adaptive neural fuzzy reinforcement learning approach," Appl. Soft Comput., vol. 7, no. 1, pp. 265-285, Jan. 2007. (Pubitemid 44739432)
13. M. Cunha and L. Ribeiro, "Tabu search algorithms for water network optimization," Eur. J. Oper. Res., vol. 157, no. 3, pp. 746-758, Sept. 16, 2004.
14. I. Montalvo, J. Izquierdo, S. Schwarze, and R. Perez-Garcia, "Multiobjective particle swarm optimization applied to water distribution systems design: An approach with human interaction," Math. Comput. Modeling, vol. 52, no. 7-8, pp. 1219-1227, 2010.
15. L. A. House-Peters and H. Chang, "Urban water demand modeling: Review of concepts, methods, and organizing principles," Water Resources Res., vol. 47, no. 5, 2011.
16. L. Kanta and E. M. Zechman, "A complex adaptive systems approach to develop basin-scale optimal management strategies for water resources systems," in Proc. World Environ. Water Resources Congr., Palm Springs, CA, May 16-20, 2011, pp. 2840-2843.
17. C. Li, F. Wang, X. Wei, and Z. Ma, "Solution method of optimal scheme set for water resources scheduling group decision-making based on multi-agent computation," Intell. Autom. Soft Comput., vol. 17, no. 7, pp. 871-883, 2011, SI.
18. D. Nickel, R. Barthel, and J. Braun, "Large-scale water resources management within the framework of GLOWA-Danube-The water supply model," Phys. Chem. Earth, vol. 30, no. 6-7, pp. 383-388, 2005, SPEC. ISS.. (Pubitemid 41213642)
19. J. Ni,M. Liu, J. Fei, and H. Ma, "Reinforcement learning based multiagent cooperation for water price forecasting decision support system," Information-An International Interdisciplinary J., vol. 15, no. 5, pp. 1889-1899, May 2012.
20. A. Bonarini, A. Lazaric, F.Montrone, andM. Restelli, "Reinforcement distribution in fuzzy Q-learning," Fuzzy Sets and Systems, vol. 160, no. 10, pp. 1420-1443, 2009.
21. R. Ollington and P. Vamplew, "Concurrent Q-learning: Reinforcement learning for dynamic goals and environments," Int. J. Intell. Syst., vol. 20, no. 10, Oct. 2005.
22. U. Kartoun, A. Shapiro, H. Stern, and Y. Edan, "Physical modeling of a bag knot in a robot learning system," IEEE Trans. Autom. Sci. Eng., vol. 7, no. 1, pp. 172-177, 2010.
23. C.-F. Juang and C.-M. Lu, "Ant colony optimization incorporated with fuzzy Q-learning for reinforcement fuzzy control," IEEE Tran. Syst., Man, Cybern. Part A: Syst. Humans, vol. 39, no. 3, pp. 597-608, 2009.
24. H. Cui, O. Turan, and P. Sayer, "Learning-based ship design optimization approach," Comput. Aided Design, vol. 44, no. 3, Mar. 2012.
25. L. Panait and S. Luke, "Cooperative multi-agent learning: The state of the art," Autonomous Agents and Multi-Agent Systems, vol. 11, no. 3, pp. 387-434, Nov. 2005. (Pubitemid 41425094)
26. X.-H. Xia, L.-H. Xu, and X.-Y. Kuang, "A distributed Nash Q-learning approach for optimizing urban traffic," J. Convergence Inf. Technol., vol. 7, no. 2, pp. 92-100, 2012.
27. L. Busoniu, R. Babuka, and B. De Schutter, "A comprehensive survey of multiagent reinforcement learning," IEEE Trans. Syst., Man, Cybern. Part C: Appl. Rev., vol. 38, no. 2, pp. 156-172, 2008. (Pubitemid 351404112)
28. D. Liu, S. Guo, X. Chen, Q. Shao, Q. Ran, X. Song, and Z. Wang, "A macro-evolutionary multi-objective immune algorithm with application to optimal allocation of water resources in Dongjiang River basins, South China," Stochastic Environmental Research and Risk Assessment, vol. 26, no. 4, pp. 491-507, May 2012.
29. C. Watkins, "Learning from delayed rewards," Ph.D. dissertation, King's College, Cambridge, U.K., May 1989.
30. J. Hu andM. P.Wellman, "Nash Q-learning for general-sum stochastic games," J. Mach. Learning Res., vol. 4, no. 6, pp. 1039-1069, 2003.
31. E. Yang and D. Gu, "Multiagent reinforcement learning for multi-robot systems: A survey," Tech. Rep. 2004. [Online]. Available: http://www.essex.ac.uk/csee/research/publications/technicalreports/2004/csm404. pdf
32. J. Wu, X. Xu, P. Zhang, and C. Liu, "A novel multi-agent reinforcement learning approach for job scheduling in Grid computing," Future Generation Comput. Syst., vol. 27, no. 5, pp. 430-439, May 2011.
33. K. Fujita and H. Matsuo, "Multiagent reinforcement learning with the partly high-dimensional state space," Syst. Comput. Japan, vol. 37, no. 9, pp. 22-31, 2006. (Pubitemid 44085938)
34. M. L. Littman, "Value-function reinforcement learning in Markov games," Cognitive Syst. Res., vol. 2, no. 1, pp. 55-66, 2001. (Pubitemid 33718550)
35. X. Xu, D. Hu, and X. Lu, "Kernel-based least squares policy iteration for reinforcement learning," IEEE Trans. Neural Networks, vol. 18, no. 4, pp. 973-992, Jul. 2007. (Pubitemid 47098876)
36. M. Zhou and S. Sun, Theory and Application of Genetic Algorithm. Beijing, China: National Defense Industry Press in China, 1996.
37. H. Wang and Y. Tong, "Optimal allocation models for regional water resources with sustainable development," J. Tsinghua Univ. (Sci. Technol.), vol. 47, no. 9, pp. 1531-1536, Sep. 2007. (Pubitemid 350009718)
38. Z. Zabinsky, Stochastic Adaptive Search for Global Optimization. Boston, MA: Kluwer Academic, 2003.
39. A. Montazar, "A decision tool for optimal irrigated crop planning and water resources sustainability," J. Global Opt. pp. 1-14, 2011. [Online]. Available: http://dx.doi.org/10.1007/s10898-011-9803-1
40. J. R. Kasprzyk, P. M. Reed, G. W. Characklis, and B. R. Kirsch, "Many-objective de Novo water supply portfolio planning under deep uncertainty," Environmental Modelling Software, vol. 34, no. SI, pp. 87-104, Jun. 2012.