Threshold tuning using stochastic optimization for graded signal control

IEEE Transactions on Vehicular Technology

Volumn 61, Issue 9, 2012, Pages 3865-3880

  Prashanth, L A ^a   Bhatnagar, Shalabh ^a  

^a INDIAN INSTITUTE OF SCIENCE   (India)

Author keywords

Deterministic perturbation sequences; intelligent transportation systems; simultaneous perturbation stochastic approximation (SPSA); stochastic optimization; threshold tuning; traffic signal control

Indexed keywords

DETERMINISTIC PERTURBATION; INTELLIGENT TRANSPORTATION SYSTEMS; SIMULTANEOUS PERTURBATION STOCHASTIC APPROXIMATION; STOCHASTIC OPTIMIZATIONS; THRESHOLD TUNING; TRAFFIC SIGNAL CONTROL;

APPROXIMATION ALGORITHMS; INTELLIGENT SYSTEMS; LEARNING ALGORITHMS; MOTOR TRANSPORTATION; OPTIMIZATION; STOCHASTIC SYSTEMS; STREET TRAFFIC CONTROL; TRAFFIC CONGESTION; TRAFFIC SIGNALS; TUNING; VEHICLE LOCATING SYSTEMS;

CLUSTERING ALGORITHMS;

EID: 84869419928     PISSN: 00189545     EISSN: None     Source Type: Journal    
DOI: 10.1109/TVT.2012.2209904     Document Type: Article

References (37)

1. L. A. Prashanth and S. Bhatnagar, "Reinforcement learning with function approximation for traffic signal control," IEEE Trans. Intell. Transp. Syst., vol. 12, no. 2, pp. 412-421, Jun. 2011.
2. S. Bhatnagar, M. Fu, S. Marcus, and I. Wang, "Two-timescale simultaneous perturbation stochastic approximation using deterministic perturbation sequences," ACM Trans. Model. Comput. Simul., vol. 13, no. 2, pp. 180-209, Apr. 2003.
3. M. Papageorgiou, C. Diakaki, V. Dinopoulou, A. Kotsialos, and Y. Wang, "Review of road traffic control strategies," Proc. IEEE, vol. 91, no. 12, pp. 2043-2067, Dec. 2003.
4. M. Papageorgiou, M. Ben-Akiva, J. Bottom, P. Bovy, S. Hoogendoorn, N. Hounsell, A. Kotsialos, and M. McDonald, "ITS and traffic management," in Handbooks in Operations Research and Management Science. Amsterdam, The Netherlands: North-Holland, 2007, pp. 715-774.
5. D. Robertson, TRANSYT: A Traffic Network Study Tool. Crowthorne, U.K.: Road Res., 1969.
6. D. Robertson and R. Bretherton, "Optimizing networks of traffic signals in real time: The SCOOT method," IEEE Trans. Veh. Technol., vol. 40, pt. 2, no. 1, pp. 11-15, Feb. 1991.
7. A. Sims and K. Dobinson, "The Sydney coordinated adaptive traffic (SCAT) system philosophy and benefits," IEEE Trans. Veh. Technol., vol. 29, no. 2, pp. 130-137, May 1980.
8. M. Girianna and R. Benekohal, "Using genetic algorithms to design signal coordination for oversaturated networks," J. Intell. Transp. Syst., vol. 8, no. 2, pp. 117-129, 2004.
9. J. Spall and D. Chin, "Traffic-responsive signal timing for systemwide traffic control," Transp. Res. C, vol. 5, no. 3/4, pp. 153-163, Aug.-Oct. 1997.
10. D. Srinivasan, M. Choy, and R. Cheu, "Neural networks for real-time traffic signal control," IEEE Trans. Intell. Transp. Syst., vol. 7, no. 3, pp. 261-272, Sep. 2006.
11. M. Patel and N. Ranganathan, "IDUTC: An intelligent decision-making system for urban traffic-control applications," IEEE Trans. Veh. Technol., vol. 50, no. 3, pp. 816-829, May 2001.
12. J. Wei, A. Wang, and N. Du, "Study of self-organizing control of traffic signals in an urban network based on cellular automata," IEEE Trans. Veh. Technol., vol. 54, no. 2, pp. 744-748, Mar. 2005.
13. X. Yu andW. Recker, "Stochastic adaptive control model for traffic signal systems," Transp. Res. C: Emerg. Technol., vol. 14, no. 4, pp. 263-282, Aug. 2006.
14. N. Gartner, OPAC: A demand-responsive strategy for traffic signal control. Washington, DC: Transp. Res. Rec., 1983.
15. J. Henry, J. Farges, and J. Tuffal, "The PRODYN real-time traffic algorithm," in Proc. IFAC Conf., 1984, pp. 305-310.
16. S. Sen and K. Head, "Controlled optimization of phases at an intersection," Transp. Sci., vol. 31, no. 1, pp. 5-17, Feb. 1997.
17. B. Abdulhai, R. Pringle, and G. Karakoulas, "Reinforcement learning for true adaptive traffic signal control," J. Transp. Eng., vol. 129, no. 3, pp. 278-285, May 2003.
18. E. Mueller, "Aspects of the history of traffic signals," IEEE Trans. Veh. Technol., vol. VT-19, no. 1, pp. 6-17, Feb. 1970.
19. E. Kosmatopoulos, M. Papageorgiou, A. Vakouli, and A. Kouvelas, "Adaptive fine-tuning of nonlinear control systems with application to the urban traffic control strategy TUC," IEEE Trans. Control Syst. Technol., vol. 15, no. 6, pp. 991-1002, Nov. 2007.
20. E. Kosmatopoulos and A. Kouvelas, "Large-scale nonlinear control system fine-tuning through learning," IEEE Trans. Neural Netw., vol. 20, no. 6, pp. 1009-1023, Jun. 2009.
21. A. Kouvelas, K. Aboudolas, E. Kosmatopoulos, and M. Papageorgiou, "Adaptive performance optimization for large-scale traffic control systems," IEEE Trans. Intell. Transp. Syst., vol. 12, no. 4, pp. 1434-1445, Dec. 2011.
22. N. Yung and A. Lai, "An effective video analysis method for detecting red light runners," IEEE Trans. Veh. Technol., vol. 50, no. 4, pp. 1074-1084, Jul. 2001.
23. C. Li and S. Shimamoto, "An open traffic light control model for reducing vehicles CO2 emissions based on ETC vehicles," IEEE Trans. Veh. Technol., vol. 61, no. 1, pp. 97-110, Jan. 2012.
24. S. Oh, J. Lee, and D. Choi, "A new reinforcement learning vehicle control architecture for vision-based road following," IEEE Trans. Veh. Technol., vol. 49, no. 3, pp. 997-1005, May 2000.
25. J. Spall, "Multivariate stochastic approximation using a simultaneous perturbation gradient approximation," IEEE Trans. Autom. Control, vol. 37, no. 3, pp. 332-341, Mar. 1992.
26. J. Spall, "A one-measurement form of simultaneous perturbation stochastic approximation," Automatica, vol. 33, no. 1, pp. 109-112, Jan. 1997.
27. J. Spall, "Adaptive stochastic approximation by the simultaneous perturbation method," IEEE Trans. Autom. Control, vol. 45, no. 10, pp. 1839- 1853, Oct. 2000.
28. S. Bhatnagar, "Adaptive multivariate three-timescale stochastic approximation algorithms for simulation based optimization," ACM Trans. Model. Comput. Simul., vol. 15, no. 1, pp. 74-107, Jan. 2005.
29. S. Bhatnagar, "Adaptive Newton-based multivariate smoothed functional algorithms for simulation optimization," ACM Trans. Model. Comput. Simul., vol. 18, no. 1, pp. 1-35, Dec. 2007.
30. P. J. Schweitzer, "Perturbation theory and finite Markov chains," J. Appl. Probab., vol. 5, no. 2, pp. 401-413, Aug. 1968.
31. V. Borkar, Stochastic Approximation: A Dynamical Systems Viewpoint. Cambridge, U.K.: Cambridge Univ. Press, 2008.
32. D. P. Bertsekas and J. N. Tsitsiklis, Neuro-Dynamic Programming. Belmont, MA: Athena Scientific, May 1996.
33. M. Wiering, J. Vreeken, J. van Veenen, and A. Koopman, "Simulation and optimization of traffic in a city," in Proc. IEEE Intell. Veh. Symp., Jun. 2004, pp. 453-458.
34. S. Cools, C. Gershenson, and B. D'Hooghe, "Self-organizing traffic lights: A realistic simulation," in Advances in Applied Self-Organizing Systems. New York: Springer-Verlag, 2008, pp. 41-50.
35. V. S. Borkar, Probability Theory: An Advanced Course. New York: Springer-Verlag, 1995.
36. M. W. Hirsch, "Convergent activation dynamics in continuous-time networks," Neural Netw., vol. 2, no. 5, pp. 331-349, 1989.
37. H. Kushner and D. Clark, Stochastic Approximation Methods for Constrained and Unconstrained Systems. New York: Springer-Verlag, 1978.