Using smart devices for system-level management and control in the smart grid: A reinforcement learning framework

2012 IEEE 3rd International Conference on Smart Grid Communications, SmartGridComm 2012

Volumn , Issue , 2012, Pages 85-90

  Kara, Emre Can ^a   Berges, Mario ^a   Krogh, Bruce ^a   Kar, Soummya ^a  

^a CARNEGIE MELLON UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ADAPTIVE CONTROL STRATEGY; ANCILLARY SERVICE; LINEAR TIME-INVARIANT; MANAGEMENT AND CONTROLS; MARKOV DECISION PROCESSES; Q-LEARNING APPROACH; REFERENCE-TRACKING; THERMOSTATICALLY CONTROLLED LOADS;

LEARNING ALGORITHMS; MARKOV PROCESSES; SMART POWER GRIDS;

REINFORCEMENT LEARNING;

EID: 84876067125     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/SmartGridComm.2012.6485964     Document Type: Conference Paper

References (21)

1. Y. G. Rebours, D. S. Kirschen, M. Trotignon, and S. Rossignol, "A survey of frequency and voltage control ancillary services-Part I: Technical features, " IEEE Trans. Power Sys. , vol. 22, pp. 350-357, 2007.
2. L. Xie, P. M. S. Carvalho, L. A. F. M. Ferreira, J. Liu, B. H. Krogh, N. Popli, and M. D. Ilic, "Wind integration in power systems: Operational challenges and possible solutions, " Proceedings of the IEEE, vol. 99, no. 1, pp. 214-232, Jan. 2011.
3. A. H. R. Faruqi, G. S. S. , J. Bode, P. Mangasarian, I. Rohmund, G. Wikler, and S. Glosh, D. Yoshida, "A national assesment of demand response potential, " Federal Energy Regulatory Commission (FERC), Tech. Rep. , June 2009. [Online]. Available: www. ferc. gov/legal/staff-reports/06-09-demand- response. pdf
4. A. Brooks, E. Lu, D. Reicher, C. Spirakis, and B. Weihl, "Demand dispatch, " IEEE Power and Energy Magazine, vol. 8, no. 3, pp. 20-29, June 2010.
5. D. Hammerstrom, J. Brous, G. R. Horst, R. Kajfasz, P. Michie, T. Oliver, T. Carlon, C. Eustis, O. Jarvegren, W. Marek, R. Munson, and R. Pratt, "Pacific Northwest GridWise Testbed Demonstration Projects Part II. Grid Friendly Appliance Project, " Pacific Northwest Nat. Lab. , Richland, WA. , Tech. Rep. , Oct. 2007.
6. U. S. Energy Information Administration, "Residential energy consumption survey (RECS), " U. S. Dept. Energy, Washington, DC, Tech. Rep. , 2001.
7. D. S. Callaway and I. A. Hiskens, "Achieving controllability of electric loads, " Proceedings of the IEEE, vol. 99, no. 1, pp. 184-199, Jan. 2011.
8. C. Duncan S. , "Tapping the energy storage potential in electric loads to deliver load following and regulation, with application to wind energy, " Energy Conversion and Management, vol. 50, no. 5, pp. 1389-1400, May 2009.
9. D. Callaway, S. Koch, and J. Mathieu, "Modelling of control of aggregated heterogenous thermostatically controlled loads for ancillary services, " Power Systems Computation Conference, Aug. 2011.
10. T. Facchinetti and M. Della Vedova, "Real-time modeling for direct load control in cyber-physical power systems, " Industrial Informatics, IEEE Transactions on, vol. 7, no. 4, pp. 689-698, Nov. 2011.
11. N. Lu, D. Chassin, and S. Widergren, "Modeling uncertainties in aggregated thermostatically controlled loads using a state queueing model, " IEEE Transactions on Power Systems, vol. 20, no. 2, pp. 725-733, May 2005.
12. J. Short, D. Infield, and L. Freris, "Stabilization of grid frequency through dynamic demand control, " Power Systems, IEEE Transactions on, vol. 22, no. 3, pp. 1284-1293, Aug. 2007.
13. K. Turitsyn, S. Backhaus, M. Ananyev, and M. Chertkov, "Smart finite state devices: A modeling framework for demand response technologies, " Mar. 2011. [Online]. Available: http://arxiv. org/abs/1103. 2750
14. S. Bashash and H. Fathy, "Modeling and control insights into demandside energy management through setpoint control of thermostatic loads, " in American Control Conference (ACC), 2011, 2011, pp. 4546-4553.
15. R. Sutton, A. Barto, and R. Williams, "Reinforcement learning is direct adaptive optimal control, " IEEE Control Systems, vol. 12, no. 2, pp. 19-22, Apr. 1992.
16. R. Sutton, "First results with DYNA, an intergrated architecture for learning, " in Planning, and Reacting, " In AAAI Spring Symposium on Planning in Uncertain, Unpredictable, or Changing Environments, 1990.
17. A. M. Leslie Pack Kaelbling, Michael Littman, "Reinforcement learning: A survey, " Journal of Artificial Intelligence Research, vol. 4, pp. 237-285, 1996.
18. M. Sato, K. Abe, and H. Takeda, "Learning control of finite markov chains with an explicit trade-off between estimation and control, " IEEE Transactions on Systems, Man, and Cybernetics, vol. 18, no. 5, pp. 677-684, Oct. 1988.
19. P. Kumar andW. Lin, "Optimal adaptive controllers for unknown markov chains, " IEEE Transactions on Automatic Control, vol. 27, no. 4, pp. 765-774, Aug. 1982.
20. C. J. C. H. Watkins and P. Dayan, "Q-learning, " Machine Learning, vol. 8, pp. 279-292, 1992.
21. M. L. Littman and C. Szepesvri, "A generalized reinforcement- learning model: Convergence and applications, " in In Proceedings of the 13th International Conference on Machine Learning. Morgan Kaufmann, 1996, pp. 310-318.