Reinforcement learning with function approximation for traffic signal control

IEEE Transactions on Intelligent Transportation Systems

Volumn 12, Issue 2, 2011, Pages 412-421

  Prashanth, L A ^a   Bhatnagar, Shalabh ^a  

^a INDIAN INSTITUTE OF SCIENCE   (India)

Author keywords

Q learning with full state representation (QTLC FS); Q learning with function approximation (QTLC FA); reinforcement learning (RL); traffic signal control

Indexed keywords

CURSE OF DIMENSIONALITY; FEATURE-BASED; FUNCTION APPROXIMATION; HIGH-DIMENSIONAL; ORDERS OF MAGNITUDE; OTHER ALGORITHMS; PERFORMANCE COMPARISON; PRIORITIZATION; Q-LEARNING; QUEUE ALGORITHMS; QUEUE LENGTHS; REINFORCEMENT LEARNING WITH FUNCTION APPROXIMATIONS; ROAD NETWORK; STATE REPRESENTATION; TRAFFIC SIGNAL CONTROL;

APPROXIMATION ALGORITHMS; COMPUTATIONAL COMPLEXITY; HIGHWAY TRAFFIC CONTROL; INFORMATION DISSEMINATION; LEARNING ALGORITHMS; MOTOR TRANSPORTATION; QUEUEING THEORY; REINFORCEMENT LEARNING; ROADS AND STREETS; STREET TRAFFIC CONTROL; TRAFFIC CONGESTION;

TRAFFIC SIGNALS;

EID: 79958173163     PISSN: 15249050     EISSN: None     Source Type: Journal    
DOI: 10.1109/TITS.2010.2091408     Document Type: Conference Paper

References (29)

1. B. Abdulhai, R. Pringle, and G. Karakoulas, "Reinforcement learning for true adaptive traffic signal control", J. Transp. Eng., vol. 129, no. 3, pp. 278-285, May/Jun. 2003. (Pubitemid 36594889)
2. S. Cools, C. Gershenson, and B. D'Hooghe, "Self-organizing traffic lights: A realistic simulation", in Advances in Applied Self-Organizing Systems. New York: Springer-Verlag, 2008, pp. 41-50.
3. J. Spall and D. Chin, "Traffic-responsive signal timing for systemwide traffic control", Transp. Res. Part C: Emerging Technol., vol. 5, no. 3/4, pp. 153-163, Aug. 1997. (Pubitemid 127438592)
4. R. Smith and D. Chin, "Evaluation of an adaptive traffic control technique with underlying system changes", in Proc. Winter Simul. Conf., 1995, pp. 1124-1130.
5. D. Chin, J. Spall, and R. Smith, "Evaluation of systemwide traffic signal control using stochastic optimization and neural networks", in Proc. Amer. Control Conf., 1999, vol. 3, pp. 2188-2194. (Pubitemid 32078817)
6. D. Srinivasan, M. Choy, and R. Cheu, "Neural networks for real-time traffic signal control", IEEE Trans. Intell. Transp. Syst., vol. 7, no. 3, pp. 261-272, Sep. 2006. (Pubitemid 44376113)
7. W. Lin and C. Wang, "An enhanced 0-1 mixed-integer LP formulation for traffic signal control", IEEE Trans. Intell. Transp. Syst., vol. 5, no. 4, pp. 238-245, Dec. 2004.
8. B. Gokulan and D. Srinivasan, "Distributed geometric fuzzy multiagent urban traffic signal control", IEEE Trans. Intell. Transp. Syst., vol. 11, no. 3, pp. 714-727, Sep. 2010.
9. M. Girianna and R. Benekohal, "Using genetic algorithms to design signal coordination for oversaturated networks", J. Intell. Transp. Syst., vol. 8, no. 2, pp. 117-129, Apr. 2004. (Pubitemid 44112511)
10. J. Sanchez-Medina, M. Galan-Moreno, and E. Rubiyo-Royo, "Traffic signal optimization in "La Almozara" district in Saragossa under congestion conditions, using genetic algorithms, traffic microsimulation, and cluster computing", IEEE Trans. Intell. Transp. Syst., vol. 11, no. 1, pp. 132-141, Mar. 2010.
11. X. Yu and W. Recker, "Stochastic adaptive control model for traffic signal systems", Transp. Res. Part C: Emerging Technol., vol. 14, no. 4, pp. 263-282, Aug. 2006. (Pubitemid 44490300)
12. D. Robertson, TRANSYT: A Traffic Network Study Tool. Crowthorne, U. K.: Road Res. Lab., 1969.
13. D. Robertson and R. Bretherton, "Optimizing networks of traffic signals in real time-the SCOOT method", IEEE Trans. Veh. Technol., vol. 40, pt. 2, no. 1, pp. 11-15, Feb. 1991.
14. D. de Oliveira, A. Bazzan, B. da Silva, E. Basso, L. Nunes, R. Rossetti, E. de Oliveira, R. da Silva, and L. Lamb, "Reinforcement learning based control of traffic lights in nonstationary environments: A case study in a microscopic simulator", in Proc. 4th EUMAS, 2006, pp. 31-42.
15. T. Li, D. Zhao, and J. Yi, "Adaptive dynamic programming for multiintersections traffic signal intelligent control", in Proc. 11th Int. IEEE ITSC, 2008, pp. 286-291.
16. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction (Adaptive Computation and Machine Learning). Cambridge, MA: MIT Press, Mar. 1998.
17. D. P. Bertsekas and J. N. Tsitsiklis, Neurodynamic Programming (Optimization and Neural Computation Series 3). Belmont, MA: Athena Scientific, May 1996.
18. C. J. Watkins and P. Dayan, "Q-learning", Mach. Learn., vol. 8, no. 3, pp. 279-292, May 1992. [Online]. Available: http://dx.doi.org/10.1007/ BF00992698
19. G. Abu-Lebdeh and R. Benekohal, "Design and evaluation of dynamic traffic management strategies for congested conditions", Transp. Res. Part A: Policy Pract., vol. 37, no. 2, pp. 109-127, Feb. 2003. (Pubitemid 36059578)
20. I. Yun and B. Park, "Application of stochastic optimization method for an urban corridor", in Proc. 38th Winter Simul. Conf., 2006, pp. 1493-1499.
21. F. Melo and M. Ribeiro, "Q-learning with linear function approximation", in Proc. Learn. Theory, 2007, pp. 308-322.
22. M. Wiering, J. Vreeken, J. van Veenen, and A. Koopman, "Simulation and optimization of traffic in a city", in Proc. IEEE Intell. Veh. Symp., Jun. 2004, pp. 453-458.
23. P. Ravikumar, Area traffic control system for heterogeneous traffic having limited lane discipline", Department of Civil Engineering, Indian Institute of Technology Bombay, Mumbai, India, Tech. Rep., 2009.
24. M. Puterman, Markov Decision Processes: Discrete Stochastic Dynamic Programming. New York: Wiley, 1994.
25. J. Tsitsiklis, "Asynchronous stochastic approximation and Q-learning", Mach. Learn., vol. 16, no. 3, pp. 185-202, Sep. 1994.
26. K. Kwong, R. Kavler, R. Rajagopal, and P. Varaiya, "Real-time measurement of link vehicle count and travel time in a road network", IEEE Trans. Intell. Transp. Syst., vol. 11, no. 4, pp. 814-825, Dec. 2010.
27. V. Konda and J. Tsitsiklis, "On actor-critic algorithms", SIAM J. Control Optim., vol. 42, no. 4, pp. 1143-1166, 2004.
28. S. Bhatnagar, R. Sutton, M. Ghavamzadeh, and M. Lee, "Natural actorcritic algorithms", Automatica, vol. 45, no. 11, pp. 2471-2482, Nov. 2009.
29. J. Pauwelussen and P. Feenstra, "Driver behavior analysis during ACC activation and deactivation in a real traffic environment", IEEE Trans. Intell. Transp. Syst., vol. 11, no. 2, pp. 329-338, Jun. 2010.