Competition, risk and learning in electricity markets: An agent-based simulation study

Applied Energy

Volumn 195, Issue , 2017, Pages 1000-1011

  Esmaeili Aliabadi, Danial ^a   Kaya, Murat ^a   Sahin, Guvenc ^a  

^a SABANCI UNIVERSITY   (Turkey)

Author keywords

Agent based simulation; Electricity markets; Imperfect competition; Q learning; Risk aversion

Indexed keywords

COMPETITION; COSTS; PROFITABILITY; REINFORCEMENT LEARNING;

AGENT BASED SIMULATION; GENERATION COMPANIES; IMPERFECT COMPETITION; LEARNING BEHAVIOR; OLIGOPOLISTIC ELECTRICITY MARKETS; Q-LEARNING; RISK AVERSION; TRANSMISSION GRIDS;

POWER MARKETS;

COMPETITION (ECONOMICS); ELECTRICITY GENERATION; ENERGY MARKET; LEARNING; RISK FACTOR; SIMULATION;

EID: 85016412154     PISSN: 03062619     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.apenergy.2017.03.121     Document Type: Article

References (65)

1. [1] David, A.K., Wen, F., Strategic bidding in competitive electricity markets: a literature survey. Power engineering society summer meeting 2000, vol. 4, 2000, IEEE, 2168–2173.
2. [2] Hobbs, B.F., Metzler, C.B., Pang, J.S., Strategic gaming analysis for electric power systems: an MPEC approach. IEEE Trans Power Syst 15:2 (2000), 638–645.
3. [3] Conejo, A.J., Carrión, M., Morales, J.M., Decision making under uncertainty in electricity markets, vol. 1, 2010, Springer.
4. [4] Li, G., Shi, J., Qu, X., Modeling methods for GenCo bidding strategy optimization in the liberalized electricity spot market – a state-of-the-art review. Energy 36:8 (2011), 4686–4700.
5. [5] Krause, T., Andersson, G., Ernst, D., Vdovina-Beck, E., Cherkaoui, R., Germond, A., Nash equilibria and reinforcement learning for active decision maker modelling in power markets. Proceedings of the 6th IAEE European conference: modelling in energy economics and policy, 2004, 1–6.
6. [6] Krause, T., Beck, E.V., Cherkaoui, R., Germond, A., Andersson, G., Ernst, D., A comparison of Nash equilibria analysis and agent-based modelling for power markets. Int J Electr Power Energy Syst 28:9 (2006), 599–607.
7. [7] Aliabadi, D.E., Kaya, M., Şahin, G., An agent-based simulation of power generation company behavior in electricity markets under different market-clearing mechanisms. Energy Policy 100 (2017), 191–205.
8. [8] Ventosa, M., Baıllo, A., Ramos, A., Rivier, M., Electricity market modeling trends. Energy Policy 33:7 (2005), 897–913.
9. [9] Ruiz, C., Conejo, A., García-Bertrand, R., Some analytical results pertaining to Cournot models for short-term electricity markets. Electr Power Syst Res 78:10 (2008), 1672–1678.
10. [10] Bunn, D.W., Oliveira, F.S., Evaluating individual market power in electricity markets via agent-based simulation. Ann Oper Res 121:1–4 (2003), 57–77.
11. [11] Li, T., Shahidehpour, M., Strategic bidding of transmission-constrained GenCos with incomplete information. IEEE Trans Power Syst 20:1 (2005), 437–447.
12. [12] Day, C.J., Hobbs, B.F., Pang, J.S., Oligopolistic competition in power networks: a conjectured supply function approach. IEEE Trans Power Syst 17:3 (2002), 597–607.
13. [13] Díaz, C.A., Villar, J., Campos, F.A., Reneses, J., Electricity market equilibrium based on conjectural variations. Electr Power Syst Res 80:12 (2010), 1572–1579.
14. [14] Ruiz, C., Conejo, A.J., Arcos, R., Some analytical results on conjectural variation models for short-term electricity markets. IET Gener Transm Distrib 4:2 (2010), 257–267.
15. [15] Weidlich, A., Engineering interrelated electricity markets: an agent-based computational approach. 2008, Springer Science & Business Media.
16. [16] Ruiz, C., Kazempour, S.J., Conejo, A.J., Equilibria in futures and spot electricity markets. Electr Power Syst Res 84:1 (2012), 1–9.
17. [17] Kardakos, E.G., Simoglou, C.K., Bakirtzis, A.G., Optimal bidding strategy in transmission-constrained electricity markets. Electr Power Syst Res 109 (2014), 141–149.
18. [18] Weidlich, A., Veit, D., A critical survey of agent-based wholesale electricity market models. Energy Econ 30:4 (2008), 1728–1759.
19. [19] Bunn, D.W., Oliveira, F.S., Agent-based simulation-an application to the new electricity trading arrangements of England and Wales. IEEE Trans Evol Comput 5:5 (2001), 493–503.
20. [20] Veit, D.J., Weidlich, A., Krafft, J.A., An agent-based analysis of the German electricity market with transmission capacity constraints. Energy Policy 37:10 (2009), 4132–4144.
21. [21] Li, G., Shi, J., Agent-based modeling for trading wind power with uncertainty in the day-ahead wholesale electricity markets of single-sided auctions. Appl Energy 99 (2012), 13–22.
22. [22] Erev, I., Roth, A.E., Predicting how people play games: reinforcement learning in experimental games with unique, mixed strategy equilibria. Am Econ Rev 88:4 (1998), 848–881.
23. [23] Watkins, C.J., Dayan, P., Q-learning. Mach Learn 8:3–4 (1992), 279–292.
24. [24] Sutton, R.S., Learning to predict by the methods of temporal differences. Mach Learn 3:1 (1988), 9–44.
25. [25] Nicolaisen, J., Petrov, V., Tesfatsion, L., Market power and efficiency in a computational electricity market with discriminatory double-auction pricing. IEEE Trans Evol Comput 5:5 (2001), 504–523.
26. [26] Li, H., Tesfatsion, L., Co-learning patterns as emergent market phenomena: an electricity market illustration. J Econ Behav Org 82:2 (2012), 395–419.
27. [27] Sharbafi, M.A., Azidehak, A., Hoshyari, M., Babarsad, O.B., Aliabadi, D.E., Zareian, A., et al. MRL extended team description 2011. Proceedings of the 15th international RoboCup symposium, Istanbul, Turkey, 2011, 1–29.
28. [28] Wang, J., Conjectural variation-based bidding strategies with Q-learning in electricity markets. 42nd Hawaii international conference on system sciences, 2009, HICSS’09, 2009, IEEE, 1–10.
29. [29] Yu, N.P., Liu, C.C., Price, J., Evaluation of market rules using a multi-agent system method. IEEE Trans Power Syst 25:1 (2010), 470–479.
30. [30] Guo, M., Liu, Y., Malec, J., A new q-learning algorithm based on the metropolis criterion. IEEE Trans Syst Man Cybernet Part B (Cybernet) 34:5 (2004), 2140–2143.
31. [31] Dahlgren, R., Liu, C.C., Lawarree, J., Risk assessment in energy trading. IEEE Trans Power Syst 18:2 (2003), 503–511.
32. [32] Bathurst, G.N., Weatherill, J., Strbac, G., Trading wind generation in short term energy markets. IEEE Trans Power Syst 17:3 (2002), 782–789.
33. [33] Ni, E., Luh, P.B., Rourke, S., Optimal integrated generation bidding and scheduling with risk management under a deregulated power market. IEEE Trans Power Syst 19:1 (2004), 600–609.
34. [34] Conejo, A.J., Garcia-Bertrand, R., Carrion, M., Caballero, Á, de Andres, A., Optimal involvement in futures markets of a power producer. IEEE Trans Power Syst 23:2 (2008), 703–711.
35. [35] Morales, J.M., Conejo, A.J., Pérez-Ruiz, J., Short-term trading for a wind power producer. IEEE Trans Power Syst 25:1 (2010), 554–564.
36. [36] Morales, J.M., Conejo, A.J., Madsen, H., Pinson, P., Zugno, M., Trading stochastic production in electricity pools. Integrating renewables in electricity markets, 2014, Springer, 205–242.
37. [37] Zheng, Q.P., Wang, J., Liu, A.L., Stochastic optimization for unit commitment – a review. IEEE Trans Power Syst 30:4 (2015), 1913–1924.
38. [38] Conejo, A.J., Nogales, F.J., Arroyo, J.M., García-Bertrand, R., Risk-constrained self-scheduling of a thermal power producer. IEEE Trans Power Syst 19:3 (2004), 1569–1574.
39. [39] García-González, J., Parrilla, E., Mateo, A., Risk-averse profit-based optimal scheduling of a hydro-chain in the day-ahead electricity market. Eur J Oper Res 181:3 (2007), 1354–1369.
40. [40] Dicorato, M., Forte, G., Trovato, M., Caruso, E., Risk-constrained profit maximization in day-ahead electricity market. IEEE Trans Power Syst 24:3 (2009), 1107–1114.
41. [41] Ghadikolaei, H.M., Ahmadi, A., Aghaei, J., Najafi, M., Risk constrained self-scheduling of hydro/wind units for short term electricity markets considering intermittency and uncertainty. Renew Sustain Energy Rev 16:7 (2012), 4734–4743.
42. [42] Jiang, R., Wang, J., Guan, Y., Robust unit commitment with wind power and pumped storage hydro. IEEE Trans Power Syst 27:2 (2012), 800–810.
43. [43] Nojavan, S., Zare, K., Risk-based optimal bidding strategy of generation company in day-ahead electricity market using information gap decision theory. Int J Electr Power Energy Syst 48 (2013), 83–92.
44. [44] Chen, J., Zhuang, Y., Li, Y., Wang, P., Zhao, Y., Zhang, C., Risk-aware short term hydro-wind-thermal scheduling using a probability interval optimization model. Appl Energy 189 (2017), 534–554.
45. [45] Batlle, C., Otero-Novas, I., Alba, J., Meseguer, C., Barquín, J., A model based in numerical simulation techniques as a tool for decision-making and risk management in a wholesale electricity market. Part I: General structure and scenario generators. PMAPS00 Conference, 2000, 1–7.
46. [46] Gountis, V.P., Bakirtzis, A.G., Bidding strategies for electricity producers in a competitive electricity marketplace. IEEE Trans Power Syst 19:1 (2004), 356–365.
47. [47] Caruso, E., Dicorato, M., Minoia, A., Trovato, M., Supplier risk analysis in the day-ahead electricity market. IEE Proc Gener Transm Distrib 153:3 (2006), 335–342.
48. [48] Cabero, J., Ventosa, M.J., Cerisola, S., Baillo, A., Modeling risk management in oligopolistic electricity markets: a Benders decomposition approach. IEEE Trans Power Syst 25:1 (2010), 263–271.
49. [49] Pousinho, H.M., Contreras, J., Bakirtzis, A.G., Catalão, J.P., Risk-constrained scheduling and offering strategies of a price-maker hydro producer under uncertainty. IEEE Trans Power Syst 28:2 (2013), 1879–1887.
50. [50] Chin, D., Siddiqui, A., Capacity expansion and forward contracting in a duopolistic power sector. Comput Manage Sci 11:1–2 (2014), 57–86.
51. [51] Egging, R., Pichler, A., Kalvø, Ø.I., Walle-Hansen, T.M., Risk aversion in imperfect natural gas markets. Eur J Oper Res 259:1 (2017), 367–383.
52. [52] Fleten, S.E., Wallace, S.W., Ziemba, W.T., Hedging electricity portfolios via stochastic programming. Decision making under uncertainty, 2002, Springer, 71–93.
53. [53] Vehviläinen, I., Keppo, J., Managing electricity market price risk. Eur J Oper Res 145:1 (2003), 136–147.
54. [54] Sahin, C., Shahidehpour, M., Erkmen, I., Generation risk assessment in volatile conditions with wind, hydro, and natural gas units. Appl Energy 96 (2012), 4–11.
55. [55] Gielis, F., Potential effects of risk aversion on technology choices and security of supply: researched with an agent-based model of a liberalised electricity market. Ph.D. thesis, 2016, Delft University of Technology, TU Delft.
56. [56] Di Lorenzo, G., Pilidis, P., Witton, J., Probert, D., Monte-carlo simulation of investment integrity and value for power-plants with carbon-capture. Appl Energy 98 (2012), 467–478.
57. [57] Liu, Y., Wu, F., Risk management of generators’ strategic bidding in dynamic oligopolistic electricity market using optimal control. IET Gener Transm Distrib 1:3 (2007), 388–398.
58. [58] Rahimiyan, M., Mashhadi, H.R., An adaptive-learning algorithm developed for agent-based computational modeling of electricity market. IEEE Trans Syst Man Cybernet Part C (Appl Rev) 40:5 (2010), 547–556.
59. [59] Sun, J., Tesfatsion, L., et al. DC optimal power flow formulation and solution using QuadProgJ. Proceedings, IEEE power and energy society general meeting, Tampa, Florida, 2007, 1–62.
60. [60] Sutton, R.S., Barto, A.G., Reinforcement learning: an introduction, vol. 1, 1998, MIT Press, Cambridge.
61. [61] Shoham Y, Powers R, Grenager T. Multi-agent reinforcement learning: a critical survey. Tech rep; 2003.
62. [62] Boonchuay, C., Ongsakul, W., Optimal risky bidding strategy for a generating company by self-organising hierarchical particle swarm optimisation. Energy Convers Manage 52:2 (2011), 1047–1053.
63. [63] Ma, X., Wen, F., Ni, Y., Liu, J., Towards the development of risk-constrained optimal bidding strategies for generation companies in electricity markets. Electr Power Syst Res 73:3 (2005), 305–312.
64. [64] Liu, A.L., Hobbs, B.F., Tacit collusion games in pool-based electricity markets under transmission constraints. Math Program 140:2 (2013), 351–379.
65. [65] Aliabadi, D.E., Kaya, M., Şahin, G., Determining collusion opportunities in deregulated electricity markets. Electr Power Syst Res 141 (2016), 432–441.