Approximate value iteration in the reinforcement learning context. application to electrical power system control.

International Journal of Emerging Electric Power Systems

Volumn 3, Issue 1, 2005, Pages 1-35

  Ernst, Damien ^a   Glavic, Mevludin ^a   Geurts, Pierre ^a   Wehenkel, Louis ^a  

^a UNIVERSITY OF LIÈGE   (Belgium)

Author keywords

Approximate value iteration; Electrical power oscillations damping; Power system control; reinforcement learning; TCSC control

Indexed keywords

APPROXIMATION ALGORITHMS; DISCRETE TIME CONTROL SYSTEMS; ELECTRIC POWER SYSTEMS; INTELLIGENT AGENTS; ITERATIVE METHODS; LEARNING ALGORITHMS; OPTIMAL CONTROL SYSTEMS; POWER CONTROL; REINFORCEMENT LEARNING;

DISCRETE-TIME OPTIMAL CONTROL PROBLEM; ELECTRICAL POWER; FOUR-MACHINE POWER SYSTEM; POWER SYSTEM CONTROLS; POWER SYSTEM OSCILLATIONS; THYRISTOR CONTROLLED SERIES CAPACITOR; VALUE ITERATION; VALUE ITERATION ALGORITHM;

ELECTRIC POWER SYSTEM CONTROL;

EID: 81355166317     PISSN: 21945756     EISSN: 1553779X     Source Type: Journal    
DOI: 10.2202/1553-779X.1066     Document Type: Review

References (41)

1. R. Bellman. Dynamic Programming. Princeton University Press, 1957.
2. R. Bellman, R. Kalaba, and B. Kotkin. Polynomial approximation-A new computational technique in dynamic programming: Allocation processes. Mathematical Computation, 17:155-161, 1973.
3. K. H. Chan, L. Jiang, P. Tilloston, and Q. H. Wu. Reinforcement learning for the control of large-scale power systems. In Proceedings of EIS'2000, Paisley, UK, 2000.
4. A. Diu and L. Wehenkel. EXaMINE-Experimentation of a monitoring and control system for managing vulnerabilities of the European infrastructure for electrical power exchange. In Proceedings of the IEEE PES Summer Meeting, panel session on power system security in the new market environment, Chicago, USA, 2002.
5. D. Ernst. Near Optimal Closed-Loop Control. Application to Electric Power Systems. PhD thesis, University of Liége, Belgium, March 2003.
6. D. Ernst. Selecting concise sets of samples for a reinforcement learning agent. Submitted, 2005.
7. D. Ernst, P. Geurts, and L. Wehenkel. Iteratively extending time horizon reinforcement learning. In N. Lavra, L. Gamberger, and L. Todorovski, editors, Proceedings of the 14th European Conference on Machine Learning, pages 96-107, Dubrovnik, Croatia, September 2003. Springer-Verlag Heidelberg.
8. D. Ernst, P. Geurts, and L. Wehenkel. Tree-based batch mode reinforcement learning. Journal of Machine Learning Research, 6:503-556, April 2005.
9. D. Ernst, M. Glavic, and L. Wehenkel. Power system stability control: reinforcement learning framework. IEEE Transactions on Power Systems, 19:427-435, February 2004.
10. D. Ernst and L. Wehenkel. FACTS devices controlled by means of reinforcement learning algorithms. In Proceedings of the 14th Power System Computation Conference, Sevilla, Spain, June 2002.
11. P. Geurts. Contributions to Decision Tree Induction: Bias/Variance Tradeoff and Time Series Classification. PhD thesis, University of Liége, Belgium, May 2002.
12. P. Geurts, D. Ernst, and L. Wehenkel. Extremely randomized trees. Submitted, 2004.
13. M. Ghandhari. Control Lyapunov Functions: A Control Strategy for Damping of Power Oscillations in Large Power Systems. PhD thesis, Royal Instituate of Technology, Dept. of Electric Power Engineering, Electric Power Systems, 2000.
14. M. Glavic, D. Ernst, and L. Wehenkel. Combining a stability and a performance oriented control in power systems. IEEE Transactions on Power Systems, 20(1):525-525, 2005.
15. G.J. Gordon. Approximate Solutions to Markov Decision Processes. PhD thesis, Carnegie Mellon University, June 1999.
16. O. Herńandez-Lerma and B. Lasserre. Discrete-Time Markov Control Processes. Springer, New-York, 1996.
17. N.G. Hingorani and L. Gyugyi. Understanding FACTS. IEEE press, 2000.
18. J. Jung, C.C. Liu, S.L. Tanimoto, and V. Vittal. Adaptation in load shedding under vulnerable operating conditions. IEEE Transactions on Power Systems, 17(4):1199-1205, 2002.
19. L.P. Kaelbling, M.L. Littman, and A.R. Cassandra. Planning and acting in partially observable stochastic domains. Artificial Intelligence, 101, 1998.
20. L.P. Kaelbling, M.L. Littman, and A.W. Moore. Reinforcement learning: A survey. Journal of Artificial Intelligence Research, 4:237-285, 1996.
21. P. Kundur. Power System Stability and Control. McGraw-Hill, 1994.
22. B. H. Li and Q. H. Wu. Learning coordinated fuzzy logic control of dynamic quadrature boosters in multimachine power systems. IEE Part C-Generation, Transmission, and Distribution, 146(6):577-585, 1999.
23. C.C. Liu, J. Jung, G.T. Heydt, and V. Vittal. The strategic power infrastructure defense (SPID) system. IEEE Control System Magazine, 20: 40-52, 2000.
24. A.W. Moore and C.G. Atkeson. Prioritized sweeping: reinforcement learning with less data and less real time. Machine Learning, 13:103-130, 1993.
25. D. Ormoneit and P. Glynn. Kernel-based reinforcement learning in average-cost problems. IEEE Transactions on Automatic Control, 47 (10):1624-1636, 2002.
26. D. Ormoneit and S. Sen. Kernel-based reinforcement learning. Machine Learning, 49(2-3):161-178, 2002.
27. M. Pavella and P.G.Murthy. Transient Stability of Power Systems: Theory and Practice. John Wiley & Sons, 1994.
28. G. Rogers. Power System Oscillations. Kluwer Academic Publishers, 2000.
29. J. Rust. Using randomization to break the curse of dimensionality. Econometrica, 65(3):487-516, 1997.
30. S. Singh and D. Bertsekas. Reinforcement learning for dynamical channel allocation in cellular telephone systems. In M.C. Mozer, M.I. Jordan, and T. Petsche, editors, Advances in Neural Information Processing Systems, volume 9, pages 974-980. The MIT Press, 1997.
31. R.S. Sutton. Integrated architectures for learning, planning and reacting based on approximating dynamic programming. In Proceedings of the Seventh International Conference on Machine Learning, pages 216-224, San Mateo, CA, 1990. Morgan Kaufmann.
32. R.S. Sutton, D. McAllester, S. Singh, and Y. Mansour. Policy gradient methods for reinforcement learning with function approximation. Advances in Neural Information Processing Systems, 12:1057-1063, 2000.
33. C.W. Taylor. Response-based, feedforward wide-Area control. Position paper for NSF/DOE/EPRI Sponsored Workshop on Future Research Directions for Complex Interactive Networks, Washington DC, USA, 16-17 November 2000., 2000.
34. G.J. Tesauro. TD-Gammon, a self-Teaching backgammon program, achieves master-level play. Neural Computation, 6(2):215-219, 1994.
35. J.N. Tsitsiklis and B. Van Roy. Feature-based methods for large-scale dynamic programming. Machine Learning, 22:59-94, 1996.
36. J.N. Tsitsiklis and B. Van Roy. Actor-critic algorithms. Advances in Neural Information Processing Systems, 12, 2000.
37. G.K. Venayagamoorthy, R.G. Harley, and D.C. Wunsch. Implementation of adaptive critic-based neurocontrollers for turbogenerators in a multimachine power system. IEEE Transactions on Neural Networks, 14(5): 1047-1064, 2003.
38. C.J.C.H. Watkins. Learning from Delayed Rewards. PhD thesis, Cambridge University, Cambridge, England, 1989.
39. L. Wehenkel. Emergency control and its strategies. In Proceedings of the 13-Th PSCC, pages 35-48, Trondheim, Norway, 1999.
40. L. Wehenkel, M. Glavic, and D. Ernst. New developments in the application of automatic learning to power system control. In Proceedings of the 15th Power System Computation Conference, 2005.
41. R.J. Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine Learning, 8:229-256, 1992.