Autonomous adaptive and active tuning up of the dissolved oxygen setpoint in a wastewater treatment plant using reinforcement learning

IEEE Transactions on Systems, Man and Cybernetics Part C: Applications and Reviews

Volumn 42, Issue 5, 2012, Pages 768-774

  Hernandez Del Olmo, Felix ^a   Gaudioso, Elena ^a   Nevado, Antonio ^a  

^a UNIVERSIDAD NACIONAL DE EDUCACIÓN A DISTANCIA   (Spain)

Author keywords

Adaptive control; intelligent agent; machine learning; wastewater systems

Indexed keywords

ADAPTIVE CONTROL; DIRECT INTERACTIONS; INFLUENT LOADS; MODEL FREE; PLANT OPERATORS; REINFORCEMENT LEARNING AGENT; SIMULATION MODEL; WASTE WATER SYSTEMS; WASTEWATER TREATMENT PLANTS;

COMPUTER SIMULATION; INTELLIGENT AGENTS; LEARNING SYSTEMS; REINFORCEMENT LEARNING; SEWAGE PUMPING PLANTS; TOXICITY; WATER TREATMENT PLANTS;

AUTONOMOUS AGENTS;

EID: 84865442280     PISSN: 10946977     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSMCC.2011.2162401     Document Type: Article

References (31)

1. L. Belanche, J. J. Valdes, J. Comas, I. R. Roda, and M. Poch, "Prediction of the bulking phenomenon in wastewater treatment plants," Artif. Intell. Eng., vol. 14, no. 4, pp. 307-317, 2000. (Pubitemid 32029223)
2. D. P. Bertsekas, Dynamic Programming and Optimal Control, vol. 2, 3rd ed. Belmont, MA: Athenas Scientific, 2007.
3. L. Busoniu, R. Babuska, and B. De Schutter, "A comprehensive survey of multi-agent reinforcement learning," IEEE Trans. Syst., Man, Cybern. C: Appl. Rev., vol. 38, no. 2, pp. 156-172, Mar. 2008. (Pubitemid 351404112)
4. E. F. Carrasco, J. Rodriguez, A. Punyal, E. Roca, and J. M. Lema, "Rulebased diagnosis and supervision of a pilot-scale wastewater treatment plant using fuzzy logic techniques," Expert Syst. Appl., vol. 22, no. 1, pp. 11-20, 2002. (Pubitemid 33076561)
5. E. F. Carrasco, J. Rodriguez, A. Punyal, E. Roca, and J. M. Lema, "Diagnosis of acidification states in an anaerobic wastewater treatment plant using a fuzzy-based expert system," Control Eng. Practice, vol. 12, no. 1, pp. 59-64, 2004. (Pubitemid 38098423)
6. J. C. Chen, N. B. Chang, andW. K. Shieh, "Assessing wastewater reclamation potential by neural network model," Eng. Appl. Artif. Intell., vol. 16, no. 2, pp. 149-157, 2003.
7. J. Copp, The COST Simulation Benchmark: Description and Simulator Manual. Luxembourg: Office for Official Publications of the European Community, 2002, p. 154.
8. U. Cortes,M. Sanchez-Marre, L. Ceci, I. R-Roda, and M. Poch, "Artificial intelligence and environmental decision support systems," Appl. Intell., vol. 13, no. 1, pp. 77-91, 2000.
9. F. Hernandez Del Olmo, E. Gaudioso, and J. G. Boticario, "A reinforcement approach to achieve unobtrusive and interactive recommendation systems for web-based communities," in Adaptive Hypermedia and Adaptive Web-Based Systems, (Lecture Notes Computer Science Series 3137). New York: Springer-Verlag, 2004, pp. 409-412.
10. F. Hernandez Del Olmo, E. Gaudioso, and E. H. Martin, "The task of guiding in adaptive recommender systems," Expert Syst. Appl., vol. 36, no. 2, pt. 1, pp. 1972-1977, 2009.
11. F. Hernandez Del Olmo and F.H. Llanes, "Artificial intelligence inwastewater systems: An emergent approach," in Proc. 10th IWA Conf. Instrum., Control Autom., 2009. [CD-ROM].
12. F. Hernandez Del Olmo and F. H. Llanes, "An emergent approach for oxigen control in the N-ammonia removal process," in Proc. Int. IWA Conf. Sustainable Solutions Small Water Syst., 2010. [CD-ROM].
13. D. Ernst, M. Glavic, F. Capitanescu, and L. Wehenkel, "Reinforcement learning versus model predictive control:Acomparison on a power system problem," IEEE Trans. Syst., Man, Cybern. B: Cybern., vol. 39, no. 2, pp. 517-529, Apr. 2009.
14. P. Fritzson, Principles of Object-Oriented Modeling and Simulation With Modelica 2.1. Piscataway, NJ: IEEE Press, 2004.
15. E. Gaudioso, M. Montero, L. Talavera, and F. Hernandez Del Olmo, "Supporting teachers in collaborative student modeling: A framework and an implementation," Expert Syst. Appl., vol. 36, no. 2, pp. 2260-2265, 2009.
16. E. Gaudioso and L. Talavera, "Data mining to support tutoring in virtual learning communities: Experiences and challenges," in Data Mining in E-Learning, C. Romero and S. Ventura, Eds., U.K.: WIT Press, 2006, pp. 157-171.
17. M. Huang, J. Wan, Y. Ma, W. Li, X. Sun, and Y. Wan, "A fast predicting neural fuzzy model for on-line estimation of nutrient dynamics in an anoxic/oxic process," Bioresource Technol., vol. 101, no. 6, pp. 1642-1651, 2010.
18. M. Huang, J. Wan, M. Yongwen, W. Yan, L. Weijiang, and S. Xiaofei, "Control rules of aeration in a submerged biofilm wastewater treatment process using fuzzy neural networks," Expert Syst. Appl., vol. 36, no. 7, pp. 10428-10437, 2009.
19. Metcalf-Eddy, Inc, Wastewater Engineering: Treatment, Disposal, Reuse. New York: McGraw-Hill Interamericana, 1998.
20. L. P. Kaelbling, M. L. Littman, and A.W. Moore, "Reinforcement learning: A survey," J. Artif. Intell. Res., vol. 4, pp. 237-285, 1996. (Pubitemid 126646155)
21. S. Krovviddy and W. G. Wee, "Wastewater treatment system from casebased reasoning," Mach. Learning, vol. 10, pp. 341-363, 1993.
22. S. Krovviddy, W. G. Wee, M. Suidan, R. S. Summers, J. J. Coleman, and L. Rossman, "Intelligent sequence planning for wastewater treatment systems," IEEE Expert: Intell. Syst. Appl., vol. 9, no. 6, pp. 15-20, Dec. 1994.
23. S. Liao, "Expert system methodologies and applications: A decade review from 1995 to 2004," Expert Syst. Appl., vol. 28, pp. 93-103, Jan. 2005.
24. G. Olsson, M. Nielsen, Z. Yuan, A. Lynggaard-Jensen, and J. P. Steyer, Instrumentation, Control and Automation inWastewater Systems. Denver, USA: IWA Publishing, 2005.
25. P. Samuelsson, B. Halvarsson, and B. Carlsson, "Cost-efficient operation of a denitrifying activated sludge process," Water Res., vol. 41, pp. 2325-2332, 2007. (Pubitemid 46670195)
26. M. Sanchez, U. Cortes, J. Lafuente, I. R. Roda, and M. Poch, "DAIDEPUR: An integrated and distributed architecture for wastewater treatment plants supervision," Artif. Intell. Eng., vol. 10, no. 3, pp. 275-285, 1996. (Pubitemid 126358813)
27. P. Serra, M. Sanchez-Marre, J. Lafuente, U. Cortes, and M. Poch, "DEPUR: A knowledge based tool for wastewater treatment plants," Eng. Appl. Artif. Intell., vol. 7, pp. 23-30, 1994.
28. A. Stare, D. Vrecko, N. Hvala, and S. Strmcnik, "Comparison of control strategies for nitrogen removal in activated sludge process in terms of operating costs: A simulation study," Water Res., vol. 41, pp. 2004-2014, 2007. (Pubitemid 46528973)
29. R. Sutton and A. Barto, Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press, 1998.
30. S. Syafiie, F. Tadeo, and E. Martinez, "Model-free learning control of neutralization processes using reinforcement learning," Eng. Appl. Artif. Intell., vol. 20, no. 6, pp. 767-782, 2007. (Pubitemid 46962288)
31. C. H. Wen and C. A. Vassiliadis, "Applying hybrid artificial intelligence in wastewater treatment," Eng. Appl. Artif. Intell., vol. 11, pp. 685-705, 1998. (Pubitemid 128409032)