Asynchronous stochastic approximation algorithms for networked systems: Regime-switching topologies and multiscale structure

Multiscale Modeling and Simulation

Volumn 11, Issue 3, 2013, Pages 813-839

  Yin, G ^a   Yuan, Quan ^a   Wang, Le Yi ^b  

^a WAYNE STATE UNIVERSITY   (United States)

^b Wayne State University   (United States)

Author keywords

Asynchronous algorithm; Consensus; Nonstationarity; Random computation time; Stochastic approximation; Switching model

Indexed keywords

ASYNCHRONOUS ALGORITHMS; COMPUTATION TIME; CONSENSUS; NON-STATIONARITIES; STOCHASTIC APPROXIMATIONS; SWITCHING MODEL;

APPROXIMATION ALGORITHMS; APPROXIMATION THEORY; MARKOV PROCESSES; MULTI AGENT SYSTEMS; ORDINARY DIFFERENTIAL EQUATIONS; TOPOLOGY;

STOCHASTIC CONTROL SYSTEMS;

EID: 84887459277     PISSN: 15403459     EISSN: 15403467     Source Type: Journal    
DOI: 10.1137/120871614     Document Type: Article

References (40)

1. I. Akyildiz, W. Su, Y. Sankarasubramniam, and E. Cayirci, A survey on sensor networks, IEEE Commun. Mag., 40 (2002), pp. 102-114.
2. P. Billingsley, Convergence of Probability Measures, John Wiley, New York, 1968.
3. R. A. Brualdi and H. J. Ryser, Combinatorial Matrix Theory, Cambridge University Press, Cambridge, UK, 1991.
4. J. Cort́es and F. Bullo, Coordination and geometric optimization via distributed dynamical systems, SIAM J. Control Optim., 44 (2005), pp. 1543-1574.
5. S. N. Ethier and T. G. Kurtz, Markov Processes: Characterization and Convergence, John Wiley, New York, 1986.
6. S. Haykin, Digital Communications, 4th ed., John Wiley, New York, 2001.
7. M. Huang and J. H. Manton, Stochastic approximation for consensus seeking: Mean square and almost sure convergence, in Proceedings of the 46th IEEE Conference on Decision and Control (CDC), New Orleans, LA, 2007, pp. 306-311. (Pubitemid 351277170)
8. M. Huang, S. Dey, G. N. Nair, and J. H. Manton, Stochastic consensus over noisy networks with Markovian and arbitrary switches, Automatica J. IFAC, 46 (2010), pp. 1571-1583.
9. A. Jadbabaie, J. Lin, and A. S. Morse, Coordination of groups of mobile autonomous agents using nearest neighbor rules, IEEE Trans. Automat. Control, 48 (2003), pp. 988-1000.
10. S. Kar and J. M. F. Moura, Distributed consensus algorithms in sensor networks with imperfect communication: Link failures and channel noise, IEEE Trans. Signal Process., 57 (2009), pp. 355-369.
11. H. J. Kushner, Approximation and Weak Convergence Methods for Random Processes, with Applications to Stochastic Systems Theory, MIT Press, Cambridge, MA, 1984.
12. H. J. Kushner and G. Yin, Asymptotic properties of distributed and communicating stochastic approximation algorithms, SIAM J. Control Optim., 25 (1987), pp. 1266-1290.
13. H. J. Kushner and G. Yin, Stochastic approximation algorithms for parallel and distributed processing, Stochastics, 22 (1987), pp. 219-250.
14. H. J. Kushner and G. Yin, Stochastic Approximation and Recursive Algorithms and Applications, 2nd ed., Springer-Verlag, New York, 2003.
15. T. Li and J. F. Zhang, Consensus conditions of multi-Agent systems with time-varying topologies and stochastic communication noises, IEEE Trans. Automat. Control, 55 (2010), pp. 2043-2057.
16. Y. Liu, K. Passino, and M. M. Polycarpou, Stability analysis of M-dimensional asynchronous swarms with a fixed communication topology, IEEE Trans. Automat. Control, 48 (2003), pp. 76-95.
17. N. A. Lynch, Distributed Algorithms, Morgan Kaufmann, San Francisco, 1997.
18. T. K. Moon, Error Correction Coding, Mathematical Methods and Algorithms, John Wiley, New York, 2005.
19. L. Moreau, Stability of multiagent systems with time-dependent communication links, IEEE Trans. Automat. Control, 50 (2005), pp. 169-182. (Pubitemid 40319348)
20. R. Olfati-Saber and R. M. Murray, Consensus problems in networks of agents with switching topology and time-delays, IEEE Trans. Automat. Control, 49 (2004), pp. 1520-1533.
21. R. Olfati-Saber, J. A. Fax, and R. M. Murray, Consensus and cooperation in networked multi-Agent systems, IEEE Proc., 95 (2007), pp. 215-233.
22. P. Ogren, E. Fiorelli, and N. E. Leonard, Cooperative control of mobile sensor networks: Adaptive gradient climbing in a distributed environment, IEEE Trans. Automat. Control, 49 (2005), pp. 1292-1302.
23. W. Ren and R.W. Beard, Consensus seeking in multiagent systems under dynamically changing interaction topologies, IEEE Trans. Automat. Control, 50 (2005), pp. 655-661. (Pubitemid 40784437)
24. C. W. Reynolds, Flocks, herds, and schools: A distributed behavioral model, Comput. Graphics, 21 (1987), pp. 25-34.
25. J. Toner and Y. Tu, Flocks, herds, and schools: A quantitative theory of flocking, Phys. Rev. E, 58 (1998), pp. 4828-4858. (Pubitemid 128596420)
26. J. N. Tsitsiklis, D. P. Bertsekas, and M. Athans, Distributed asynchronous deterministic and stochastic gradient optimization algorithms, IEEE Trans. Automat. Control, 31 (1986), pp. 803-812. (Pubitemid 16629805)
27. R. S. Varga, Matrix Iterative Analysis, Spring-Verlag, Berlin, 2000.
28. T. Viseck, A. Czirook, E. Ben-Jacob, O. Cohen, and I. Shochet, Novel type of phase transition in a system of self-driven particles, Phys. Rev. Lett., 75 (1995), pp. 1226-1229.
29. L. Xiao, S. Boyd, and S. J. Kim, Distributed average consensus with least-mean-square deviation, J. Parallel Distrib. Comput., 67 (2007), pp. 33-46. (Pubitemid 44828440)
30. G. Yin, C. Ion, and V. Krishnamurthy, How does a stochastic optimization/approximation algorithm adapt to a randomly evolving optimum/root with jump Markov sample paths, Math. Programming Ser. B, 120 (2009), pp. 67-99.
31. G. Yin, V. Krishnamurthy, and C. Ion, Regime switching stochastic approximation algorithms with application to adaptive discrete stochastic optimization, SIAM J. Optim., 14 (2004), pp. 1187-1215.
32. G. Yin and V. Krishnamurthy, Least mean square algorithms with Markov regime switching limit, IEEE Trans. Automat. Control, 50 (2005), pp. 577-593. (Pubitemid 40784430)
33. G. Yin, Y. Sun, and L. Y. Wang, Asymptotic properties of consensus-Type algorithms for networked systems with regime-switching topologies, Automatica J. IFAC, 47 (2011), pp. 1366-1378.
34. G. Yin, L. Y. Wang, and Y. Sun, Stochastic recursive algorithms for networked systems with delay and random switching: Multiscale formulations and asymptotic properties, Multiscale Model. Simul., 9 (2011), pp. 1087-1112.
35. G. Yin, C. Z. Xu, and L. Y. Wang, Optimal remapping in dynamic bulk synchronous computations via a stochastic control approach, IEEE Trans. Parallel Distrib. Systems, 14 (2003), pp. 51-62.
36. G. Yin and Q. Zhang, Singularly perturbed discrete-Time Markov chains, SIAM J. Appl.Math., 61 (2000), pp. 834-854.
37. G. Yin and Q. Zhang, Discrete-Time Markov Chains: Two-Time-Scale Methods and Applications, Springer, New York, 2005.
38. G. Yin, Q. Zhang, and G. Badowski, Discrete-Time singularly perturbed Markov chains: Aggregation, occupation measures, and switching diffusion limit, Adv. in Appl. Probab., 35 (2003), pp. 449-476.
39. G. Yin and C. Zhu, Hybrid Switching Diffusions: Properties and Applications, Springer, New York, 2010.
40. G. Yin and Y. M. Zhu, On w.p.1 convergence of a parallel stochastic approximation algorithm, Probab. Engrg. Inform. Sci., 3 (1989), pp. 55-75.