Neural networks for continuous online learning and control

IEEE Transactions on Neural Networks

Volumn 17, Issue 6, 2006, Pages 1511-1531

  Choy, Min Chee ^a   Srinivasan, Dipti ^a   Cheu, Ruey Long ^a  

^a NATIONAL UNIVERSITY OF SINGAPORE   (Singapore)

Author keywords

Distributed control; Hybrid model; Neural control; Online learning; Traffic signal control

Indexed keywords

COMPUTER SIMULATION; EVOLUTIONARY ALGORITHMS; LEARNING ALGORITHMS; LEARNING SYSTEMS; MATHEMATICAL MODELS; ONLINE SYSTEMS;

HYBRID MODEL; NEURAL CONTROL; ONLINE LEARNING; TRAFFIC SIGNAL CONTROL;

NEURAL NETWORKS;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; AUTOMATED PATTERN RECOGNITION; INFORMATION RETRIEVAL; METHODOLOGY; SIGNAL PROCESSING; TRAFFIC AND TRANSPORT;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; INFORMATION STORAGE AND RETRIEVAL; NEURAL NETWORKS (COMPUTER); PATTERN RECOGNITION, AUTOMATED; SIGNAL PROCESSING, COMPUTER-ASSISTED; TRANSPORTATION;

EID: 34447547289     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2006.881710     Document Type: Article

References (32)

1. A. G. Barto and S. Mahadevan, "Recent advances in hierarchical reinforcement learning," Special Issue Reinforcement Learn., Discrete Event Syst. J., vol. 13, pp. 41-77, 2003.
2. M. Kearns and S. Singh, "Near-optimal reinforcement learning in polynomial time," in Proc. Int. Conf. Mach. Learn., 1999, pp. 260-268.
3. M. Littman and C. Szepesvari, "A generalized reinforcement learning model: Convergence and applications," in Proc. 13th Int. Conf. Mach. Learn., 1996, pp. 310-318.
4. R. E. Parr, "Hierarchical control and learning for Markov decision processes," Ph.D. dissertation, Univ. California Berkeley, Berkeley, CA, 1998.
5. G. Rummery and M. Niranjan, Online Q-learning using connectionist system Cambridge Univ., Eng. Dept., Tech. Rep. CUED/F-INFENG/TR 166, 1994.
6. I. Johnson and M. D. Plumbley, "On-line connectionist Q-learning produces unreliable performance with a synonym finding task," in Proc. Int. Joint Conf. Neural Netw., Jul. 2000, pp. 24-27.
7. R. Jaksa, P. Majernik, and P. Sincak, Reinforcement learning based on back propagation for mobile robot navigation Computational Intelligence Group, Dept. Cybern. Artif. Intell., Technical Univ., Kosice, Slovakia, 2000.
8. S. Chiu and S. Chand, "Self-organizing traffic control via fuzzy logic," in Proc. 32nd IEEE Conf. Decision Control, 1993, pp. 1987-1902.
9. G. Nakamiti and F. Gomide, "Fuzzy sets in distributed traffic control," in Proc. 5th IEEE Int. Conf. Fuzzy Syst., 1996, pp. 1617-1623.
10. S. Mikami and Y. Kakazu, "Genetic reinforcement learning for cooperative traffic signal control," in Proc. 1st IEEE Conf. Evol. Comput., 1994, vol. 1, pp. 223-228.
11. W. Wei and Y. Zhang, "FL-FN based traffic signal control," in Proc. 2002 IEEE Int. Conf. Fuzzy Syst., May 2002, vol. 1, no. 12-17, pp. 296-300.
12. J. C. Spall and D. C. Chin, "Traffic-responsive signal timing for system-wide traffic control," Transpn. Res.- C, vol. 5, no. 3/4, pp. 153-163, 1997.
13. E. Bingham, "Reinforcement learning in neural fuzzy traffic signal control," Euro. J. Operation Res., vol. 131, no. 2, pp. 232-241, 2001.
14. N. J. Garber and L. A. Hoel, Traffic and Highway Engineering, 2nd ed. Boston, MA: PWS-Kent, 1997, pp. 281-329.
15. M. Baglietto, T. Parisini, and R. Zoppoli, "Distributed-information neural control: The case of dynamic routing in traffic networks," IEEE Trans. Neural Netw., vol. 12, no. 3, pp. 485-502, May 2001.
16. T. Parisini, M. Sanguineti, and R. Zoppoli, "Nonlinear stabilization by receding-horizon neural regulators," Int. J. Control, vol. 70, no. 3, pp. 341-362, 1998.
17. J. J. Buckley and U. Hayashi, "Hybrid fuzzy neural nets are universal approximators," in Proc. 3rd IEEE Conf. Fuzzy Syst. IEEE World Congr. Comput. Intell., Jun. 1994, vol. 1, pp. 238-243.
18. A. V. Wouwer, C. Renotte, and M. Remy, "On the use of simultaneous perturbation stochastic approximation for neural network training," in Proc. Amer. Control Conf., 1999, pp. 388-392.
19. E. Gomez-Ramirez, P. L. Najim, and E. Ikonen, "Stochastic learning control for nonlinear systems," in Proc. Int. Joint Conf. Neural Netw. (IJCNN'02), May 2002, vol. 1, pp. 171-176.
20. T. Jaakkola, M. I. Jordan, and S. P. Singh, "On the convergence of stochastic iterative dynamic programming algorithms," Neural Comput., vol. 6, no. 6, pp. 1185-1201, 1994.
21. M. Littman and C. Szepesvari, "A generalized reinforcement learning model: Convergence and applications," in Proc. 13th Int. Conf. Mach. Learn., 1996, pp. 310-318.
22. J. C. Spall, "Multivariate stochastic approximation using a simultaneous perturbation gradient approximation," IEEE Trans. Autom. Control, vol. 37, no. 3, pp. 332-341, Mar. 1992.
23. H. Robbins and S. Monro, "A stochastic approximation method," Ann. Math. Statist., vol. 25, pp. 382-386, 1951.
24. J. Kiefer and J. Wolfowitz, "Stochastic estimation of a regression function," Ann. Math. Stat., vol. 23, pp. 462-466, 1952.
25. R. Jaksa, P. Majernik, and P. Sincak, "Reinforcement learning based on back propagation for mobile robot navigation," in Proc. Comput. Intell. Modeling, Control, Autom. (CIMCA), Vienna, Austria, 1999.
26. R. S. Sutton and A. Barto, Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press, 1998.
27. R. A. Jacobs, "Increased rates of convergence through learning rate adaptation," Neural Netw., vol. 1, pp. 295-307, 1988.
28. Z. Luo, "On the convergence of the LMS algorithm with adaptive learning rate for linear feedforward networks," Neural Comput., vol. 3, pp. 226-245, 1991.
29. T. Kohonen, Self-Organizing Maps, 2nd ed. Berlin, Germany: Springer-Verlag, 1997.
30. G. Yan, F. Yang, T. Hickey, and M. Goldstein, "Coordination of exploration and exploitation in a dynamic environment," in Proc. Int. Joint Conf. Neural Netw. (IJCNN), 2001, pp. 1014-1018.
31. Quadstone, PARAMICS Modeller v4.0 User Guide and Reference Manual. Edinburgh, U.K.: Quadstone Ltd., 2002.
32. P. B. Wolshon and W. C. Taylor, "Analysis of intersection delay under real-time adaptive signal control," Transportation Res. Part C, Emerging Technol., vol. 7C, no. 1, pp. 53-72, Feb. 1999.