Soilse: A decentralized approach to optimization of fluctuating urban traffic using reinforcement learning

IEEE Conference on Intelligent Transportation Systems, Proceedings, ITSC

Volumn , Issue , 2010, Pages 531-538

  Salkham, AS'ad ^a   Cahill, Vinny ^a,b  

^a TRINITY COLLEGE DUBLIN   (Ireland)

^b University of Limerick   (Ireland)

Author keywords

[No Author keywords available]

Indexed keywords

AVERAGE NUMBERS; BALANCING ALGORITHMS; BASE-LINE PERFORMANCE; DECENTRALIZED APPROACH; DEVELOPED COUNTRIES; DYNAMIC NATURE; ENVIRONMENTAL COSTS; NON-PARAMETRIC; OPTIMIZATION SCHEME; PATTERN CHANGE; TRAFFIC PATTERN; URBAN AREAS; URBAN TRAFFIC; URBAN TRAFFIC CONTROL; WAITING-TIME;

INTELLIGENT SYSTEMS; OPTIMIZATION; REINFORCEMENT LEARNING;

TRAFFIC CONGESTION;

EID: 78650467018     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ITSC.2010.5625145     Document Type: Conference Paper

References (49)

1. OECD, OECD Factbook 2008: Economic, Environmental and Social Statistics, 2008.
2. European Commission, "Memo - towards a new culture for urban mobility," 2007.
3. D. Cosgrove and D. Gargett, "Estimating urban traffic and congestion cost trends for australian cities," 2007.
4. D. Schrank and T. Lomax, "2009 annual urban mobility report," 2009.
5. European Commission, "Green paper - towards a new culture for urban mobility sec(2007) 1209," vol. COM/2007/0551 final, 2007.
6. United States FHWA, "Managing our congested streets and highways," Washington, DC, 2001.
7. United States DOT, "Intelligent transportation systems for traffic signal control - deployment benefits and lessons learned," 2007.
8. P. R. Lowrie, "Scats: The sydney co-ordinated adaptive traffic system-principles, methodology, algorithms," in Proceedings of the IEE International Conference on Road Traffic Signalling, 1982, pp. 67-70.
9. A. Sims and K. Dobinson, "The sydney coordinated adaptive traffic (scat) system philosophy and benefits," Vehicular Technology, IEEE Transactions on, vol. 29, no. 2, pp. 130-137, May 1980.
10. D. I. Robertson and R. D. Bretherton, "Optimizing networks of traffic signals in real time-the scoot method," Vehicular Technology, IEEE Transactions on, vol. 40, no. 1, pp. 11-15, 1991.
11. P. Mirchandani and F.Y. Wang, "Rhodes to intelligent transportation systems," Intelligent Systems, IEEE, vol. 20, no. 1, pp. 10-15, jan.-feb. 2005.
12. V. Mauro and C. Di Taranto, "Utopia," Control Computers Communications in Transportation, pp. 245-252, 1990, pergamon Press, Oxford.
13. I. Porche and S. Lafortune, "Dynamic traffic control: Decentralized and coordinated methods," in Proceedings of the IEEE Conference on ITS, 1997.
14. J. L. Farges, J. J. Henry, and J. Tuffal, "The prodyn real-time traffic algorithm," in Proceedings of the IEE International Conference on Road Traffic Signalling, 1983, pp. 307-312.
15. V. Dinopoulou, C. Diakaki, and M. Papageorgiou, "Applications of the urban traffic control strategy tuc." European Journal of Operational Research, vol. 175, no. 3, pp. 1652-1665, 2006.
16. A. Di Febbraro, D. Giglio, and N. Sacco, "Urban traffic control structure based on hybrid petri nets," Intelligent Transportation Systems, IEEE Transactions on, vol. 5, no. 4, pp. 224-237, Dec. 2004.
17. A. L. C. Bazzan, "A distributed approach for coordination of traffic signal agents," AAMAS, vol. 10, no. 1, pp. 131-164, 2004.
18. G. Felici, G. Rinaldi, A. Sforza, and K. Truemper, "A logic programming based approach for on-line traffic control," Transportation Research Part C: Emerging Technologies, vol. 14, no. 3, pp. 175-189, 2006.
19. B. De Schutter, "Optimal traffic light control for a single intersection," in Proc. American Control Conference the 1999, vol. 3, 1999, pp. 2195-2199 vol.3.
20. B. Abdulhai, P. Pringle, and G. J. Karakoulas, "Reinforcement learning for true adaptive traffic signal control," in ASCE Journal of Transportation Engineering, vol. 129(3), 2003, pp. 278-284.
21. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press, 1998.
22. C. J. C. H. Watkins and P. Dayan, "Q-learning" Machine Learning, vol. 8, no. 3-4, pp. 279-292, 1992.
23. A. Salkham, R. Cunningham, A. Garg, and V. Cahill, "A collaborative reinforcement learning approach to urban traffic control optimization," Web Intelligence and Intelligent Agent Technology, IEEF/WIC/ACM International Conference on, vol. 2, pp. 560-566, 2008.
24. M. Papageorgiou, C. Diakaki, V. Dinopoulou, A. Kotsialosa, and Y. Wang, "Review of road traffic control strategies," in Proceedings of the IEEE, vol. 91, 2003, pp. 2043-2067.
25. United States FHWA, "Signal timing under saturated conditions," FHWA-HOP-09-008, November 2008.
26. P. Mirchandani and L. Head, "A real-time traffic signal control system: architecture, algorithms, and analysis," Transportation Research Part C: Emerging Technologies, vol. 9, pp. 415-432, December 2001.
27. N. Gartner, "Opac: A demand-responsive strategy for traffic signal control," U.S. Dept. Transportation, Transp. Res. Record 906, 1983.
28. K. Dresner and P. Stone, "Multiagent traffic management: A reservation-based intersection control mechanism," in AAMAS '04. Washington, DC, USA: IEEE Computer Society, 2004, pp. 530-537.
29. E. Camponogara and K. J. Werner, "Distributed learning agents in urban traffic control." in LNCS, vol. 2902. Springer, 2003, pp. 324-335.
30. S. Richter, D. Aberdeen, and J. Yu, "Natural actor-critic for road traffic optimisation," in Advances in Neural Information Processing Systems, vol. 19. The MIT Press, Cambridge, MA, 2007.
31. J. Peters, S. Vijayakumar, and S. Schaal, "Natural actor-critic." in ECML, 2005, pp. 280-291.
32. I. Dusparic and V. Cahill, "Distributed W-Learning: Multi-policy optimization in self-organizing systems," in Third IEEE SASO, 2009.
33. M. Humphrys, "Action selection methods using reinforcement learning," Ph.D. dissertation, University of Cambridge, 1996.
34. Z.-s. Yang, X. Chen, Y.-s. Tang, and J.-p. Sun, "Intelligent cooperation control of urban traffic networks," in Proceedings of 2005 ICMLC, vol. 3, 2005, pp. 1482-1486.
35. D. Srinivasan, M. C. Choy, and R. Cheu, "Neural networks for real-time traffic signal control," Intelligent Transportation Systems, IEEE Transactions on, vol. 7, no. 3, pp. 261-272, Sept. 2006.
36. M. Wiering, "Multi-agent reinforcement learning for traffic light control," in Proceedings of the 17th ICML. Morgan Kaufmann, San Francisco, CA, 2000, pp. 1151-1158.
37. M. Steingröver, R. Schouten, S. Peelen, E. Nijhuis, and B. Bakker, "Reinforcement learning of traffic light controllers adapting to traffic congestion." in BNAIC, 2005, pp. 216-223.
38. H. Prothmann, F. Rochner, S. Tomforde, J. Branke, C. Müller-Schloer, and H. Schmeck, "Organic control of traffic lights," pp. 219-233, 2008.
39. D. d. Oliveira, A. L. C. Bazzan, B. C. d. Silva, E. W. Basso, and L. Nunes, "Reinforcement learning based control of traffic lights in nonstationary environments: A case study in a microscopic simulator," in CEUR Workshop Proceedings, vol. 223. CEUR-WS.org, 2006.
40. S. Tomforde, H. Prothmann, F. Rochner, J. Branke, J. Hahner, C. Muller-Schloer, and H. Schmeck, "Decentralised progressive signal systems for organic traffic control," in Proc. Second IEEE SASO '08, 20-24 Oct. 2008, pp. 413-422.
41. K. Nagel and M. Schreckenberg, "A cellular automaton model for freeway traffic," Journal de Physique 1, no. 115, p. 2221, 1992.
42. J. C. Spall, Introduction to Stochastic Search and Optimization: Estimation, Simulation, and Control. Hoboken, NJ: Wiley,, 2003.
43. M. Thottan and C. Ji, "Anomaly detection in ip networks," IEEE Transactions on Signal Processing, vol. 51, no. 8, pp. 2191-2204, 2003.
44. K. J. Oh, M. S. Moon, and T. Y. Kim, "Variance change point detection via artificial neural networks for data separation," Neurocomputing, vol. 68, pp. 239-250, 2005.
45. H. Wang, D. Zhang, and K. G. Shin, "Detecting syn flooding attacks," in In Proceedings of the IEEE Infocom. IEEE, 2002, pp. 1530-1539.
46. v. A. Siris and F. Papagalou, "Application of anomaly detection algorithms for detecting syn flooding attacks," Computer Communications, vol. 29, no. 9, pp. 1433-1442, 2006, iCON 2004 - 12th IEEE International Conference on Network 2004.
47. G. C. I. Lin and S. V. Nagalingam, CIM justification and optimisation. London, UK: Taylor & Francis, 2000.
48. V. Reynolds, V. Cahill, and A. Senart, "Requirements for an ubiquitous computing simulation and emulation environment," in InterSense'06. New York, NY, USA: ACM, 2006.
49. S. Richter, "Learning traffic control - towards practical traffic control using policy gradients - diplomarbeit." Albert-Ludwigs- Universität Freiburg, 2006.