A distributed actor-critic algorithm and applications to mobile sensor network coordination problems

IEEE Transactions on Automatic Control

Volumn 55, Issue 2, 2010, Pages 492-497

  Pennesi, Paris ^a   Paschalidis, Ioannis Ch ^b  

^a RBS Global Banking and Markets   (United Kingdom)

^b Center for Information and Systems Engineering   (United States)

Author keywords

Actor critic methods; Consensus; Markov decision processes (MDP); Multi agent coordination; Sensor networks

Indexed keywords

ACTOR-CRITIC ALGORITHM; ACTOR-CRITIC METHODS; COMMUNICATION CONSTRAINTS; COORDINATION PROBLEMS; GENERAL CLASS; KNOWN ENVIRONMENTS; MARKOV DECISION PROBLEM; MARKOV DECISION PROCESSES; MARKOV DECISION PROCESSES (MDP); MOBILE SENSOR NETWORKS; MULTI-AGENT COORDINATIONS; MULTIPLE AGENTS; SINGLE-AGENT;

MARKOV PROCESSES; MULTI AGENT SYSTEMS; REINFORCEMENT LEARNING; WIRELESS NETWORKS;

SENSOR NETWORKS;

EID: 76949102585     PISSN: 00189286     EISSN: None     Source Type: Journal    
DOI: 10.1109/TAC.2009.2037462     Document Type: Article

References (10)

1. V. R. Konda and J. N. Tsitsiklis, "Actor-critic algorithms," SIAM J. Control Optim., vol.42, no.4, pp. 1143-1166, 2003.
2. J. N. Tsitsiklis, "Problems in Decentralized Decision Making and Computation," Ph.D. dissertation, Dept. Elect. Eng. Comp. Sci., MIT, Cambridge, MA, 1984.
3. A. Jadbabaie, J. Lin, and A. S. Morse, "Coordination of groups of mobile autonomous agents using nearest neighbor rules," IEEE Trans. Autom. Control, vol.48, no.6, pp. 988-1001, Jun. 2003.
4. V. Blondel, J. Hendrickx, A. Olshevsky, and J. Tsitsiklis, "Convergence in multiagent coordination, consensus, and flocking," in Proc. 44th IEEE Conf. Decision Control Eur. Control Conf. (CDC-ECC'05), Seville, Spain, 2005, pp. 2996-3000.
5. W. Li and C. G. Cassandras, "Distributed cooperative coverage control of sensor networks," in Proc. 44th IEEE Conf. Decision Control, 2005, pp. 2542-2547.
6. J. Cortes, S. Martinez, T. Karatas, and F. Bullo, "Coverage control for mobile sensing networks," IEEE Trans. Robot. Autom., vol.20, no.2, pp. 243-255, Apr. 2004.
7. P. Pennesi and I. C. Paschalidis, "Solving sensor network coverage problems by distributed asynchronous actor-critic methods," in Proc. 46th IEEE Conf. Decision Control, New Orleans, LA, Dec. 2007, pp. 5300-5305.
8. P. Marbach and J. Tsitsiklis, "Simulation-based optimization of Markov reward processes," IEEE Trans. Autom. Control, vol.46, no.2, pp. 191-209, Feb. 2001.
9. J. Tsitsiklis and B. V. Roy, "An analysis of temporal-difference learning with function approximation," IEEE Trans. Autom. Control, vol.42, no.5, pp. 674-690, May 1997.
10. J. Tsitsiklis, D. Bertsekas, and M. Athans, "Distributed asynchronous deterministic and stochastic gradient optimization algorithms," IEEE Trans. Autom. Control, vol.AC-31, no.9, pp. 803-812, Sep. 1986.