Maximizing Reward in a Non-Stationary Mobile Robot Environment

Autonomous Agents and Multi-Agent Systems

Volumn 6, Issue 3, 2003, Pages 287-316

  Goldberg, Dani ^a   Matarić, Maja J ^b  

^a CARNEGIE MELLON UNIVERSITY   (United States)

^b University of Southern California ^*  (United States)

Author keywords

Collection tasks; Mobile robots; Non stationary environments; On line modeling; Reward maximization

Indexed keywords

COLLECTION TASKS; NON-STATIONARY ENVIRONMENTS; ON-LINE MODELING;

ALGORITHMS; AUTONOMOUS AGENTS; MARKOV PROCESSES; MOBILE ROBOTS; STATE ESTIMATION; STOCHASTIC CONTROL SYSTEMS;

MULTI AGENT SYSTEMS;

EID: 0347132453     PISSN: 13872532     EISSN: None     Source Type: Journal    
DOI: 10.1023/A:1022935725296     Document Type: Article

References (47)

1. R. C. Arkin, Behavior-Based Robotics, The MIT Press: Cambridge, Massachusetts, 1998.
2. R. C. Arkin and K. S. Ali, "Reactive and Telerobotic Control in Multi-Agent Systems," in From Animals to Animats 3: Proceedings of the Third International Conference on Simulation of Adpalive Behavior, Brighton, England, pp. 473-478, 1994.
3. R. C. Arkin, T. Balch, and E. Nitz, "Communication of Behavioral State in Multi-Agent Retrieval Tasks," in IEEE International Conference on Robotics and Automation, Atlanta, pp. 588-594, 1993.
4. T. Balch, "Hierarchical Social Entropy: An Information Theoretic Measure of Robot Group Diversity," Autonomous Robots, vol. 8, pp. 209-237, 2000.
5. R. D. Beer, "A Dynamical Systems Perspective on Agent-Environment Interaction," Artificial Intelligence, vol. 72, pp. 173-215, 1993.
6. R. A. Brooks, "The Behavior Language: User's Guide," Technical Report AIM-1227, MIT AI Laboratory 1990.
7. R. A. Brooks, "Intelligence Without Reason," in Proceedings of the Twelfth International Joint Conference on Artificial Intelligence (IJCAI-91), Sydney, Australia, pp. 569-590, 1991.
8. C. R. Blyth, "Approximate Binomial Confidence Limits," Journal of the American Statistical Association, vol. 81, no. 395, pp. 843-855, 1986.
9. Y. U. Cao, A. S. Fukunaga, and A. B. Kahng, "Cooperative Mobile Robotics: Antecedents and Directions," Autonomous Robots, vol. 4, pp. 1-23, 1997.
10. A. R. Cassandra, L. P. Kaelbling, and J. A. Kurien, "Acting under Uncertainty: Discrete Bayesian Models for Mobile-Robot Navigation," in Proceedings of the 1996 IEEE/RSJ International Conference on Intelligent Robots and Systems, vol. 2, pp. 963-972, 1996.
11. H. Chernoff and S. Zacks, "Estimating the Current Mean of a Normal Distribution which is Subjected to Changes in Time," Annals of Mathematical Statistics, vol. 35, no. 3, pp. 999-1018, 1964.
12. T. H. Cormen, C. E. Leiserson, and R. L. Rivest, Introduction to Algorithms, McGraw-Hill Book Company, 1990.
13. L. Chrisman, "Reinforcement Learning with Perceptual Aliasing: The Perceptual Distinctions Approach," in W. Swartout (ed.), Proceedings of the 10th National Conference on Artificial Intelligence, San Jose, CA, pp. 183-188, 1992.
14. M. S. Fontán and M. J. Matarić, "Territorial Multi-Robot Task Division," IEEE Transactions on Robotics and Automation, vol. 14, no. 5, pp. 815-822, 1998.
15. J. E. Freund, Mathematical Statistics, Fifth Edition, Prentice Hall, 1992.
16. E. Gat, "On Three-Layer Architectures," in D. Kortenkamp, R. P. Bonnasso, and R. Murphy (eds.), Artificial Intelligence and Mobile Robotics: Case Studies of Successful Robot Systems, AAAI Press, pp. 195-210, 1998.
17. D. Goldberg, "Evaluating the Dynamics of Agent Environment Interaction," Ph.D. thesis, University of Southern California, 2001.
18. D. Goldberg and M. J. Matarić, "Coordinating Mobile Robot Group Behavior Using a Model of Interaction Dynamics," in Proceedings, The Third International Conference on Autonomous Agents (Agents '99), Seattle, Washington, pp. 100-107, 1999.
19. D. Goldberg and M. J. Matarić, "Detecting Regime Changes with a Mobile Robot using Multiple Models," in Proceedings of the 2001 IEEE/RSJ International Conference on Intelligent Robots and Systems, Maui, Hawaii, pp. 619-624, 2001.
20. K. Han and M. Veloso, "Automated Robot Behavior Recognition Applied to Robotic Soccer," in Robotics Research: the Ninth International Symposium, Snowbird, Utah, pp. 249-256, 2000.
21. S. J. Hanson, "Meiosis Networks," in D. S. Touretzky (ed.), Advances in Neural Information Processing Systems 2, San Mateo, CA, pp. 533-541, 1990.
22. L. P. Kaelbling, M. L. Littman, and A. R. Cassandra, "Planning and Acting in Partially Observable Stochastic Domains," Artificial Intelligence, vol. 101, no. 1-2, pp. 99-134, 1998.
23. J. G. Kemeny, J. L. Snell, and A. W. Knapp, Denumerable Markov Chains, D. Van Nostrand Company, Inc., 1966.
24. S. Koenig and R. G. Simmons, "Unsupervised Learning of Probabilistic Models for Robot Navigation," in Proceedings of the IEEE International Conference on Robotics and Automation, vol. 3, pp. 2301-2308, 1996.
25. J. Košecká and R. Bajcsy, "Discrete Event Systems for Autonomous Mobile Agents," in Proceedings of the First Workshop on Intelligent Robotic Systems, Zakopane, Poland, pp. 21-31, 1993.
26. T. L. Lai, "Sequential Changepoint Detection in Quality Control and Dynamical Systems," Journal of the Royal Statistical Society, Series B (Methodological), vol. 57, no. 4, pp. 613-658, 1995.
27. 2, and F Tail Probabilities," Journal of the American Statistical Association, vol. 73, no. 362, pp. 274-283, 1978.
28. S. Mahadevan and G. Theocharous, "Optimizing Production Manufacturing using Reinforcement Learning," in Proceedings of the Eleventh International FLAIRS Conference, Sanibel Island, Florida, pp. 372-377, 1998.
29. M. J. Matarić, "Behavior-Based Systems: Key Properties and Implications," in IEEE International Conference on Robotics and Automation, Workshop on Architectures for Intelligent Control Systems, Nice, France, pp. 46-54, 1992.
30. M. J. Matarić, "Issues and Approaches in the Design of Collective Autonomous Agents," Robotics and Autonomous Systems, vol. 16, no. 2-4, pp. 321-331, 1995.
31. M. J. Matarić, "Behavior-Based Control: Examples from Navigation, Learning, and Group Behavior," Journal of Experimental and Theoretical Artificial Intelligence, vol. 9, no. 2-3, pp. 323-336, 1997.
32. A. K. McCallum, "Reinforcement Learning with Selective Perception and Hidden State," Ph.D. thesis, University of Rochester, Department of Computer Science, 1996.
33. F. Michaud and M. J. Matarić, "Learning from History for Behavior-Based Mobile Robots in Nonstationary Conditions," Autonomous Robots, vol. 5, no. 3-4, pp. 335-354, 1998.
34. T. M. Mitchell, Machine Learning, The McGraw-Hill Companies, Inc., 1997.
35. L. E. Parker, "Heterogeneous Multi-Robot Cooperation," Ph.D. thesis, MIT, 1994.
36. D. B. Peizer and J. W. Pratt, "A Normal Approximation for Binomial, F, Beta, and Other Common, Related Tail Probabilities, 1," Journal of the American Statistical Association, vol. 63, no. 324, pp. 1416-1456, 1968.
37. P. Pirjanian, "Multiple Objective Action Selection & Behavior Fusion using Voting," Ph.D. thesis, Institute of Electronic Systems, Alborg University, Denmark, 1998.
38. W. H. Press, S. A. Teukolsky, W. T. Vetterling, and B. P. Flannery, Numerical Recipes in C: The Art of Scientific Computing, Cambridge University Press, 1992.
39. L. R. Rabiner, "A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition," Proceedings of the IEEE, vol. 77, no. 2, pp. 257-285, 1989.
40. F. S. Roberts, Discrete Mathematical Models: With Applications to Social, Biological, and Environmental Problems, Prentice-Hall, Inc., 1976.
41. S. M. Ross, Applied Probability Models with Optimization Applications, New York: Dover Publications, Inc., 1992.
42. K. Seymore, A. McCallum, and R. Rosenfeld, "Learning Hidden Markov Model Structure for Information Extraction," in Proceedings of the Sixteenth National Conference on Artificial Intelligence: Workshop on Machine Learning for Information Extraction, Orlando, FL, pp. 37-42, 1999.
43. T. Smithers, "What the Dynamics of Adaptive Behavior and Cognition Might Look Like in Agent-Environment Interaction Systems," in Practice and Future of Autonomous Agents, Mt. Verita, Switzerland, 1995.
44. A. Stolcke and S. Omohundro, "Hidden Markov Model Induction by Bayesian Model Merging," in S. J. Hanson, J. D. Cowan, and C. L. Giles (eds.), Advances in Neural Information Processing Systems, vol. 5. pp. 11-18, 1993.
45. R. S. Sutton, D. Precup, and S. Singh, "Between MDPs and Semi-MDPs: A Framework for Temporal Abstraction in Reinforcement Learning," Artificial Intelligence, vol. 112, pp. 181-211, 1999.
46. G. Wang and S. Mahadevan, "Hierarchical Optimization of Policy-Coupled Semi-Markov Decision Processes," in Proceedings of the Sixteenth International Conference on Machine Learning, Bled, Slovenia, pp. 464-473, 1999.
47. S. D. Whitehead and L.-J. Lin, "Reinforcement Learning of Non-Markov Decision Processes," Artificial Intelligence, vol. 73, no. 1-2, pp. 271-306, 1995.