Personalizing influence diagrams: Applying online learning strategies to dialogue management

User Modeling and User-Adapted Interaction

Volumn 17, Issue 1-2, 2007, Pages 71-91

  Chickering, David Maxwell ^a   Paek, Tim ^a  

^a MICROSOFT RESEARCH   (United States)

Author keywords

Dialogue management; Influence diagrams; Personalization; Planning; Speech recognition; User model adaptation

Indexed keywords

ALGORITHMS; DECISION THEORY; LEARNING SYSTEMS; MARKOV PROCESSES; MATHEMATICAL MODELS; PARAMETER ESTIMATION; PLANNING; PROBLEM SOLVING; SPEECH RECOGNITION;

DIALOGUE MANAGEMENT; INFLUENCE DIAGRAMS; MARKOV DECISION PROCESSES; USER-MODEL ADAPTATION;

ONLINE SYSTEMS;

EID: 33847317508     PISSN: 09241868     EISSN: 15731391     Source Type: Journal    
DOI: 10.1007/s11257-006-9020-7     Document Type: Article

References (25)

1. Albrecht, D., Zukerman, I., Nicholson, A.: Bayesian models for keyhole plan recognition in an adventure game. User Model. User-Adapted Interaction, Special Issue Machine Learning User Model. 8(1-2), 5-47 (1998)
2. Auer, P.: Using confidence bounds for exploitation-exploration trade-offs. J. Machine Learn. Res. 3, 397-422 (2002)
3. Auer, P., Cesa-Bianchi, M., Freund, Y., Schapire, R.: Gambling in a rigged casino: the adversarial multi-armed bandit problem. In: In Proceedings of the 36th Annual Symposium on Foundations of Computer Science, pp. 322-331. IEEE Computer Society Press, Los Alamitos, CA (1995)
4. Berry, D., Fristedt, B.: Bandit Problems: Sequential Allocation of Experiments Chapman and Hall, London (1985)
5. Boutilier, C., Dean, T., Hanks, S.: Decision-theoretic planning: structural assumptions and computational leverage. I Aritif. Intell. Res. 1, 1-93 (1999)
6. Chickering, D.M.: The winmine toolkit. Technical Report MSR-TR-2002-103, Microsoft Redmond, WA (2002)
7. Cooper, G.F.: A method for using belief networks as influence diagrams. In: Heckerman, D., Mamdani, A., (eds.), Proceedings of the Ninth Conference on Uncertainty in Artificial Intelligence, Washington, DC, pp. 55-63, Morgan Kaufmann (1993)
8. Dearden, R., Friedman, N., Russell, S.: Bayesian Q-learning. In: Proceedings of the Fifteenth National Conference on Artificial Intelligence, pp. 761-768. Madison, WI (1998)
9. Heckerman, D.: A Bayesian approach for learning causal networks. In: Hanks, S., Besnard, P. (eds.), Proceedings of the Eleventh Conference on Uncertainty in Artificial Intelligence, Montreal, QU. Morgan Kaufmann (1995)
10. Heckerman, D.: A tutorial on learning Bayesian networks. Technical Report MSR-TR-95-06, Microsoft Research (1996)
11. Horvitz, E., Breese, J., Heckerman, D., Hovel, D., Rommelse, K.: The lumiere project: Bayesian user modeling for inferring the goals and needs of software users. In: Proceedings of the Fourteenth Conference on Uncertainty in Artificial Intelligence, pp. 256-265. Madison, Wisconsin (1998)
12. Howard, R., Matheson, J.: Influence diagrams. In: Readings on the Principles and Applications of Decision Analysis, Vol. II, pp. 721-762. Strategic Decisions Group, Menlo Park, CA (1981)
13. Kaelbling, L.P.: Learning in Embedded Systems. The MIT Press, Cambridge, MA (1993)
14. Kaelbling, L.P., Littman, M.L., Moore, A.W.: Reinforcement learning: a survey. J. Artif. Intell. Res. 4, 237-285 (1996)
15. Kakade, S.M., Ng, A.Y.: Online bounds for bayesian algorithms. In: Saul, L.K., Weiss, Y., Bottou, L. (eds.), Advances in Neural Information Processing Systems, Vol. 17, pp. 641-648. MIT Press, Cambridge, MA (2005)
16. Lauritzen, S.L., Nilsson, D.: Representing and solving decision problems with limited information. Manage. Sci. 47(9), 1235-1251 (2001)
17. Roy, N., Pineau, J., Thrun, S.: Spoken dialogue management using probabilistic reasoning. In: Proceedings of ACL-2000, pp. 93-100. Hong Kong, China (2000)
18. Shachter, R., Peot, M.: Decision making using probabilistic inference methods. In: Proceedings of the 8th Annual Conference on Uncertainty in Artificial Intelligence, pp. 276-283. San Mateo, CA, Morgan Kaufmann Publishers (1992)
19. Singh, S., Litman, D., Kearns, M., Walker, M.: Optimizing dialogue management with reinforcement learning: experiments with the njfun system. J. Artif. Intell. Res. 16, 105-133 (2002)
20. Sutton, R., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press (1998)
21. Tatman, J.A., Shachter, R.D.: Dynamic programming and influence diagrams. IEEE Trans. Syst. Man Cybernet. 20(2), 365-379 (1990)
22. Thompson, W.R.: On the likelihood that one unknown probability exceeds another in view of the evidence of two samples. Biometricka. 25, 285-294 (1993)
23. Wyatt, J.: Exploration and Inference in Learning from Reinforcement. PhD thesis, University of Edinburgh (1997)
24. Young, S.: Probabilistic methods in spoken dialogue systems. Philos. Trans. Roy. Soc. (Ser A) 358(1769), 1389-1402 (2000)
25. Zukerman, I., Albrecht, D.: Predictive statistical models for user modeling. User Model. User-Adapted Interact. 11(1), 5-18 (2001)