Stochastic optimal relaxed automatic generation control in non-Markov environment based on multi-step Q(λ) learning

IEEE Transactions on Power Systems

Volumn 26, Issue 3, 2011, Pages 1272-1282

  Yu, Tao ^a   Zhou, Bin ^b   Chan, Ka Wing ^b   Chen, Liang ^c   Yang, Bo ^a  

^a SOUTH CHINA UNIVERSITY OF TECHNOLOGY   (China)

^b HONG KONG POLYTECHNIC UNIVERSITY   (Hong Kong)

^c ELECTRIC POWER RESEARCH INSTITUTE   (China)

Author keywords

AGC; CPS; multi step Q( ) learning; non Markov environment; relaxed control; stochastic optimization

Indexed keywords

AGC; CPS; MULTI-STEP; NON-MARKOV ENVIRONMENT; RELAXED CONTROL; STOCHASTIC OPTIMIZATIONS;

LEARNING ALGORITHMS; OPTIMIZATION; STATE FEEDBACK; STOCHASTIC SYSTEMS; USER INTERFACES;

DELAY CONTROL SYSTEMS;

EID: 79960889687     PISSN: 08858950     EISSN: None     Source Type: Journal    
DOI: 10.1109/TPWRS.2010.2102372     Document Type: Article

References (31)

1. N. Jaleeli et al., "Understanding automatic generation control," IEEE Trans. Power Syst., vol. 7, no. 3, pp. 1106-1122, Aug. 1992. (Pubitemid 23563457)
2. N. Jaleeli and L. S. VanSlyck, "NERC's new control performance standards," IEEE Trans. Power Syst., vol. 14, no. 3, pp. 1092-1099, Aug. 1999.
3. G. A. Chown and B.Wigdorowitz, "A methodology for the redesign of frequency control for AC networks," IEEE Trans. Power Syst., vol. 19, no. 3, pp. 1546-1554, Aug. 2004.
4. T. F. Garrity, "Getting smart: Innovation and trends for future electric power systems," IEEE Power Energy Mag., vol. 6, no. 2, pp. 38-45, Mar./Apr. 2008.
5. G. Gross and J. W. Lee, "Analysis of load frequency control performance assessment criteria," IEEE Trans. Power Syst., vol. 16, no. 3, pp. 520-525, Aug. 2001. (Pubitemid 32680752)
6. T. Sasaki and K. Enomoto, "Dynamic analysis of generation control performance standards," IEEE Trans. Power Syst., vol. 17, no. 3, pp. 806-811, Aug. 2002. (Pubitemid 35151761)
7. N. B. Hoonchareon, C. M. Ong, and R. A. Kramer, "Feasibility of decomposing ACE to identify the impact of selected loads on CPS1 and CPS2," IEEE Trans. Power Syst., vol. 17, no. 3, pp. 752-756, Aug. 2002. (Pubitemid 35151753)
8. Ibraheem, P. Kumar, and D. P. Kothari, "Recent philosophies of automatic generation control strategies in power systems," IEEE Trans. Power Syst., vol. 20, no. 1, pp. 346-357, Feb. 2005. (Pubitemid 40259454)
9. Z. H. Gao, X. L. Teng, and L. Q. Tu, "Hierarchical AGC mode and CPS control strategy for interconnected power systems," (in Chinese) Autom. Elect. Power Syst., vol. 28, no. 1, pp. 78-81, Jan. 2004.
10. T. Yu, L. Chen, and G. L. Cai, "CPS statistic information self-learning methodology based adaptive automatic generation control," (in Chinese) Proc. CSEE, vol. 28, no. 10, pp. 82-87, Apr. 2008. (Pubitemid 351648457)
11. M. Yao, R. R. Shoults, and R. Kelm, "AGC logic based on NERC's new control performance standard and disturbance control standard," IEEE Trans. Power Syst., vol. 15, no. 2, pp. 852-857, May 2000.
12. A. Feliachi and D. Rerkpreedapong, "NERC compliant load frequency control design using fuzzy rules," Elect. Power Syst. Res., vol. 73, no. 2, pp. 101-106, Feb. 2005. (Pubitemid 40114231)
13. B. Tyagi and S. C. Srivastava, "A decentralized automatic generation control scheme for competitive electricity markets," IEEE Trans. Power Syst., vol. 21, no. 1, pp. 312-320, Feb. 2006. (Pubitemid 43210238)
14. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press, 1998.
15. Q. Y. Hu and W. Y. Yue, Markov Decision Processes With Their Applications. New York: Springer, 2008.
16. P. Jing and R. J.Williams, "Incremental multi-Step Q-learning," Mach. Learn., vol. 22, no. 1-3, pp. 283-290, Jan. 1996. (Pubitemid 126724369)
17. J. C. H.Watkins and P. Dayan, "Q-learning," Mach. Learn., vol. 8, no. 3-4, pp. 279-292, May 1992.
18. R. S. Sutton, "Learning to predict by the methods of temporal differences," Mach. Learn., vol. 3, no. 1, pp. 9-44, Aug. 1988.
19. J. N. Tsitsiklis, "Asynchronous stochastic approximation and Q-learning," Mach. Learn., vol. 16, no. 3, pp. 185-202, Sep. 1994.
20. The Operation Mode of China Southern Power Grid in 2009 (in Chinese,) China Southern Power Grid Co. Ltd.
21. S. D. Whiteheada and L. J. Lin, "Reinforcement learning of non-Markov decision processes," Artif. Intell., vol. 73, no. 1-2, pp. 271-306, Feb. 1995.
22. T. P. I. Ahamed, P. S. N. Rao, and P. S. Sastry, "A reinforcement learning approach to automatic generation control," Elect. Power Syst. Res., vol. 63, no. 1, pp. 9-26, Aug. 2002. (Pubitemid 34862972)
23. B. H. Bakken and H. H. Faanes, "Technical and economic aspects of using a long submarine HVDC connection for frequency control," IEEE Trans. Power Syst., vol. 12, no. 3, pp. 1252-1258, Aug. 1997. (Pubitemid 127771502)
24. S. P. Bhattacharyya, A. Datta, and L. H. Keel, Linear Control Theory: Structure, Robustness, and Optimization. Boca Raton, FL: CRC, 2009.
25. M. L. Kothari, J. Nanda, D. P. Kothari, and D. Das, "Discrete-mode automatic generation control of a two-area reheat thermal system with new area control error," IEEE Trans. Power Syst., vol. 4, no. 2, pp. 730-738, May 1989. (Pubitemid 20604027)
26. O. I. Elgerd, Electric Energy Systems Theory: An Introduction, 2nd ed. New York: McGraw-Hill, 1982.
27. D. Ernst, M. Glavic, and L. Wehenkel, "Power systems stability control: Reinforcement learning framework," IEEE Trans. Power Syst., vol. 19, no. 1, pp. 427-435, Feb. 2004.
28. J. A. Stuller, An Introduction to Signals and Systems. Toronto, ON, Canada: Thomson, 2007.
29. Y. Manichaikul, Industrial Electric Load Modeling. Cambridge, MA: MIT Press, 1978.
30. Q. H.Wu and A. C. Pugh, "Reinforcement learning control of unknown dynamic systems," Proc. Inst. Elect. Eng. D, Control Theory Appl., vol. 140, no. 5, pp. 313-322, Sep. 1993. (Pubitemid 23703616)
31. M. Lopes and J. S. Victor, "A developmental roadmap for learning by imitation in robots," IEEE Trans. Syst., Man, Cybern. B: Cybern., vol. 37, no. 2, pp. 308-321, Apr. 2007. (Pubitemid 46523221)