Multiagent reinforcement learning for integrated network of adaptive traffic signal controllers (marlin-atsc): Methodology and large-scale application on downtown toronto

IEEE Transactions on Intelligent Transportation Systems

Volumn 14, Issue 3, 2013, Pages 1140-1150

  El Tantawy, Samah ^a   Abdulhai, Baher ^a   Abdelgawad, Hossam ^a  

^a UNIVERSITY OF TORONTO   (Canada)

Author keywords

Adaptive traffic signal control; game theory; microsimulation modeling; multi agent reinforcement learning; multi agent system; reinforcement learning

Indexed keywords

ADAPTIVE TRAFFIC SIGNAL CONTROL; LARGE-SCALE APPLICATIONS; MICROSIMULATION MODELING; MULTI-AGENT REINFORCEMENT LEARNING; TRAFFIC FLUCTUATIONS; TRAFFIC SIGNAL CONTROLLERS; TRAVEL TIME SAVINGS; URBAN TRAFFIC CONGESTION;

FERTILIZERS; GAME THEORY; MOTOR TRANSPORTATION; MULTI AGENT SYSTEMS; STREET TRAFFIC CONTROL; TIMING CIRCUITS; TRAFFIC CONGESTION; TRAFFIC SIGNALS;

REINFORCEMENT LEARNING;

EID: 84883772487     PISSN: 15249050     EISSN: None     Source Type: Journal    
DOI: 10.1109/TITS.2013.2255286     Document Type: Article

References (35)

1. W. R. McShane, R. P. Roess, and E. S. Prassas, Traffic Engineering. Englewood Cliffs, NJ, USA: Prentice-Hall, 1998.
2. P. B. Hunt, D. I. Robertson, R. D. Bretherton, and R. I. Winton, SCOOT-A traffic responsive method of coordinating signals, Transp. Road Res. Lab., Crowthorne, U.K., Tech. Rep., 1981.
3. C. Diakaki, M. Papageorgiou, and K. Aboudolas, A multivariable regulator approach to trafficresponsive network-Wide signal control, Control Eng. Pract., vol. 10, no. 2, pp. 183-195, Feb. 2002.
4. A. G. Sims and K. W. Dobinson, SCAT-The Sydney co-ordinated adaptive traffic system: Philosophy and benefits, presented at the Int. Symp. Traffic Control Systems, Berkeley, CA, USA, 1979.
5. J. L. Farges, J. J. Henry, and J. Tufal, The PRODYN real-Time traffic algorithm, presented at the 4th IFAC/IFIP/IFORS Symp. Control Transp. Syst., Baden-Baden, Germany, 1983.
6. N. H. Gartner, OPAC: A demand-responsive strategy for traffic signal control, Transp. Res. Rec., J. Transp. Res. Board, vol. 906, pp. 75-81, 1983.
7. K. L. Head, P. B.Mirchandani, and D. Sheppard, Hierarchical framework for real-Time traffic control, Transp. Res. Rec., vol. 1360, pp. 82-88, 1992.
8. A. L. C. Bazzan, Opportunities for multiagent systems and multiagent reinforcement learning in traffic control, Autonomous Agents Multi-Agent Syst., vol. 18, no. 3, pp. 342-375, Jun. 2009.
9. B. Abdulhai and L. Kattan, Reinforcement learning: Introduction to theory and potential for transport applications, Can. J. Civil Eng., vol. 30, no. 6, pp. 981-991, Dec. 2003.
10. S. El-Tantawy and B. Abdulhai, An agent-Based learning towards decentralized and coordinated traffic signal control, in Proc. 13th IEEE ITSC, 2010, pp. 665-670.
11. S. El-Tantawy and B. Abdulhai, Towards multi-Agent reinforcement learning for integrated network of optimal traffic controllers (MARLINOTC), Transp. Lett.: Int. J. Transp. Res., vol. 2, pp. 89-110, Apr. 2010.
12. C. Watkins and P. Dayan, Q-learning, Mach. Learn., vol. 8, pp. 279-292, 1992.
13. R. S. Sutton and A. G. Barto, Introduction to Reinforcement Learning. Cambridge, MA, USA: MIT Press, 1998.
14. T. Basar and G. J. Olsder, Dynamic Noncooperative Game Theory, 2nd ed. London, U.K: Classics Appl. Math., 1999.
15. L. Busoniu, R. Babuska, and B. De Schutter, A comprehensive survey of multiagent reinforcement learning, IEEE Trans. Syst., Man, Cybern. C, Appl. Rev., vol. 38, no. 2, pp. 156-172, Mar. 2008.
16. C. Claus and C. Boutilier, The dynamics of reinforcement learning in cooperative multiagent systems, in Proc. 15th Nat. Conf. Artif. Intell./10th Conf. Innov. Appl. Artif. Intell., Madison, WI, USA, 1998, pp. 746-752.
17. M. Weinberg and J. S. Rosenschein, Best-response multiagent learning in non-Stationary environments, in Proc. 3rd Int. Joint Conf. Autonom. Agents Multiagent Syst., 2004, pp. 506-513.
18. T. Thorpe, Vehicle traffic light control using sarsa,M.S. thesis, Comput. Sci. Dept., Colo. St. Univ., Fort Collins, CO, USA, 1997.
19. M.Wiering, Multi-Agent reinforcement learning for traffic light control, in Proc. 17th Int. Conf. Mach. Learn., 2000, pp. 1151-1158.
20. B. Abdulhai, R. Pringle, and G. J. Karakoulas, Reinforcement learning for true adaptive traffic signal control, J. Transp. Eng., vol. 129, no. 3, pp. 278-285, Apr. 2003.
21. E. Camponogara and W. Kraus, Jr., Distributed learning agents in urban traffic control, in Proc. 11th Portuguese Conf. Artif. Intell., 2003, pp. 324-335.
22. D. De Oliveira, A. L. C. Bazzan, B. C. da Silva, E. W. Basso, L. Nunes, R. Rossetti, E. De Oliveira, R. da Silva, and L. Lamb, Reinforcement learning-Based control of traffic lights in non-Stationary environments: A case study in a microscopic simulator, in Proc. EUMAS, 2006, pp. 31-42.
23. S. Richter, D. Aberdeen, and J. Yu, Natural actor-Critic for road traffic optimisation, in Advances in Neural Information Processing Systems. Cambridge, MA, USA: MIT Press, 2007.
24. I. Arel, C. Liu, T. Urbanik, and A. G. Kohls, Reinforcement learningbased multi-Agent system for network traffic signal control, IET Intell. Transp. Syst., vol. 4, no. 2, pp. 128-135, Jun. 2010.
25. T. Li, D. B. Zhao, and J. Q. Yi, Adaptive dynamic programming formultiintersections traffic signal intelligent control, in Proc. 11th Int. IEEE Conf. Intell. Transp. Syst., 2008, pp. 286-291.
26. A. Salkham, R. Cunningham, A. Garg, and V. Cahill, A collaborative reinforcement learning approach to urban traffic control optimization, in Proc. IEEE/WIC/ACM Int. Conf. Web Intell. Intell. Agent Technol., 2008, pp. 560-566.
27. J. C. Medina and R. F. Benekohal, Q-learning and approximate dynamic programming for traffic control-A case study for an oversaturated network, presented at the Transp. Res. Board Annu. Meet.,Washington, DC, USA, 2012, Paper 12-4103.
28. L. Shoufeng, L. Ximin, and D. Shiqiang, Q-Learning for adaptive traffic signal control based on delay minimization strategy, in Proc. IEEE Int. Conf. Netw. Sens. Control, 2008, pp. 687-691.
29. L. Kuyer, S. Whiteson, B. Bakker, and N. Vlassis, Multiagent reinforcement learning for urban traffic control using coordination graph, in Proc. 19th Eur. Conf. Mach. Learn., 2008, pp. 656-671.
30. A. L. C. Bazzan, A distributed approach for coordination of traffic signal agents, Autonom. Agents Multi-Agent Syst., vol. 10, no. 1, pp. 131-164, Jan. 2005.
31. R. Nair, P. Varakantham, M. Tambe, and M. Yokoo, Networked distributed POMDPs: A synthesis of distributed constraint optimization and POMDPs, in Proc. 20th Nat. Conf. Artif. Intell., 2005, pp. 133-139. (Pubitemid 43006767)
32. N. Ono and K. Fukumoto, Multi-Agent reinforcement learning: A modular approach, in Proc. 2nd Int. Conf. Multi-Agent Syst., 1996, pp. 252-258.
33. Quadstone Paramics, Paramics Microscopic Traffic Simulation Software, 2012. [Online]. Available: http://www.paramics-online.com
34. S. El-Tantawy and B. Abdulhai, Comprehensive analysis of reinforcement learning methods and parameters for adaptive traffic signal control, presented at the Transp. Res. Board, Washington, DC, USA, 2011.
35. S. El-Tantawy and B. Abdulhai, Neighborhood coordination-Based multi-Agent reinforcement learning for coordinated adaptive traffic signal control, presented at the Transp. Res. Board, Washington, DC, USA, 2012.