Conjectural variation-based bidding strategies with Q-learning in electricity markets

Proceedings of the 42nd Annual Hawaii International Conference on System Sciences, HICSS

Volumn , Issue , 2009, Pages

  Wang, Jianhui ^a  

^a ARGONNE NATIONAL LABORATORY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

AGENTS; EDUCATION; GAME THEORY; LEARNING ALGORITHMS; LEARNING SYSTEMS; MARKETING; REINFORCEMENT;

ADAPTIVE AGENTS; AGENT-BASED SIMULATIONS; BIDDING BEHAVIORS; BIDDING STRATEGIES; CONJECTURAL VARIATIONS; ELECTRICITY MARKETS; GENERATION COMPANIES; LEARNING PROCESS; MARKET CLEARINGS; MARKET PARTICIPANTS; METROPOLIS CRITERIONS; Q-LEARNING; Q-LEARNING ALGORITHMS; REPEATED GAMES; STAGE GAMES;

ELECTRIC GENERATORS;

EID: 63349110376     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/HICSS.2009.128     Document Type: Conference Paper

References (24)

1. Y. Smeers, "Computable Equilibrium Models and the Restructuring of the European Electricity and Gas Markets," Energy Journal, vol. 18, no. 4, pp. 1-31, 1997.
2. B.F. Hobbs, C. Metzler, and J.S. Pang, "Strategic Gaming Analysis for Electric Power Networks: an MPEC Approach," IEEE Transactions on Power Systems, vol. 15, no. 2, pp. 638-645, 2000.
3. E. Kahn, "Numerical Techniques for Analyzing Market Power in Electricity," Electricity Journal, vol. 11, no. 6, pp. 34-43, 1998.
4. J. Wang, A. Botterud, G. Conzelmann, and V. Koritarov, "Market Power Analysis in the EEX Electricity Market: An Agent-Based Simulation Approach," accepted, IEEE PES General Meetings 2008.
5. J. Wang, A. Botterud, G. Conzelmann, and V. Koritarov, "Multi-Agent System for Short and Long-term Power Market Simulations," accepted, PSCC08 conference.
6. A. Botterud, M.R. Mahalik, T.D. Veselka, H-S. Ryu, K-W. Sohn, "Multi-Agent Simulation of Generation Expansion in Electricity Markets," Proceedings of the IEEE PES General Meeting, 2007.
7. A.E. Roth and I. Erev, "Learning in Extensive Form Games: Experimental Data and Simple Dynamic Models in the Intermediate Term," Games Econom. Beh., vol. 8, pp. 164-212, 1995.
8. I. Erev and A.E. Roth, "Predicting How People Play Games with Unique, Mixed-Strategy Equilibria," Amer. Econom. Rev., vol. 88, pp. 848-881, 1998.
9. D.W. Bunn and S. Oliveira, "Agent-Based Simulation: An Application to the New Electricity Trading Arrangements of England and Wales," IEEE Transactions on Evolutionary Computation, vol. 5, pp. 493-503, 2001.
10. A.J. Bagnall and G.D. Smith, "A Multiagent Model of the UK Market in Electricity Generation," IEEE Transactions on Evolutionary Computation, vol. 9, pp. 522-536, 2005.
11. James Nicolaisen, Valentin Petrov, and Leigh Tesfatsion, "Market Power and Efficiency in a computational Electricity Market with Discriminatory Double-Auction Pricing," IEEE Transactions on Evolutionary Computation, vol.5, pp.504-523, 2001.
12. A.C. Tellidou and A.G. Bakirtzis, "Agent-Based Analysis of Capacity Withholding and Tacit Collusion in Electricity Markets," IEEE Transactions on Power Systems, vol. 22, no.4, Nov. 2007, pp. 1735-1742.
13. M. Guo, Y. Liu, and J. Malec, "A New Q-Learning Algorithm Based on the Metropolis Criterion," IEEE Trans. Syst., Man, Cybern. B: Cybern., vol. 34, no. 5, pp. 2140-2143, Oct. 2004.
14. Nanpeng Yu, Chen-Ching Liu, and L. Tesfatsion, "Modeling of Suppliers' Learning Behaviors in an Electricity Market Environment," International Conference on Intelligent Systems Applications to Power Systems, 5-8 Nov. 2007, pp. 1-6.
15. T. Krause, G. Andersson, D. Ernst, E.V. Beck, R. Cherkaoui, and A. Germond, "Nash Equilibria and Reinforcement Learning for Active Decision Maker Modeling in Power Markets," Proc. 6th IAEE Conf. - Modeling in Energy Economics, Zurich, Switzerland, Sept. 2004.
16. T. Krause, G. Andersson, D. Ernst, E.V. Beck, R. Cherkaoui, and A. Germond, "A Comparison of Nash Equilibria Analysis and Agent-Based Modeling for Power Markets," Proc. 15th Power Systems Computation Conf. (PSCC), Liege, Belgium, Aug. 22-26, 2005.
17. Y. Song, Y. Ni, F. Wen, Z. Hou, and F.F. Wu, "Conjectural Variation Based Bidding Strategy in Spot Markets: Fundamentals and Comparison with Classical Game Theoretical Bidding Strategies," Electric Power Syst Res 2003; 67(1):45-51.
18. A. García Alcalde, M. Ventosa, M. Rivier, A. Ramos, G. Relaño, "Fitting Electricity Market Models: a Conjectural Variations Approach," Proc. 14th Power Systems Computation Conf. (PSCC), Seville, 2002.
19. C.J. Day, B.F. Hobbs, and J. Pang, "Oligopolistic Competition in Power Networks: A Conjectured Supply Function Approach," IEEE Trans. Power Syst. 17 (2002), pp. 597-06.
20. J. Church, R. Ware, Industrial Organization: A Strategic Approach, Irwin McGraw-Hill, Boston, 2000.
21. C.J.C.H. Watkins and P. Dayan, "Technical Note: Q-learning," Machine Learning, vol. 8, pp. 279-292, 1992.
22. R. Axelrod, The Evolution of Cooperation, New York: Penguin Books, 1990.
23. V.P. Gountis and A.G. Bakirtzis, "Bidding Strategies for Electricity Producers in a Competitive Electricity Marketplace," IEEE Trans. Power Syst., vol. 19, Issue 1, pp.356-365, Feb. 2004.
24. R. Gibbons, A Primer in Game Theory, Harvester Wheatsheaf, 1992.