Value-directed human behavior analysis from video using partially observable Markov decision processes

IEEE Transactions on Pattern Analysis and Machine Intelligence

Volumn 29, Issue 7, 2007, Pages 1118-1132

  Hoey, Jesse ^a   Little, James J ^b  

^a UNIVERSITY OF DUNDEE   (United Kingdom)

^b UNIVERSITY OF BRITISH COLUMBIA   (Canada)

Author keywords

Clustering algorithms; Control theory; Dynamic programming; Face and gesture recognition; Machine learning; Motion; Parameter learning; Statistical models; Video analysis

Indexed keywords

BEHAVIORAL RESEARCH; CLUSTERING ALGORITHMS; CONTROL THEORY; DECISION MAKING; DYNAMIC PROGRAMMING; EXPERT SYSTEMS; FACE RECOGNITION; GESTURE RECOGNITION; MARKOV PROCESSES; OPTIMIZATION; STATISTICAL METHODS;

DYNAMIC BAYESIAN NETWORK; HUMAN BEHAVIOR ANALYSIS; LEARNING DECISION THEORETIC MODEL; MARKOV DECISION PROCESS; VIDEO ANALYSIS;

LEARNING SYSTEMS;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; AUTOMATED PATTERN RECOGNITION; BEHAVIOR; COMPUTER ASSISTED DIAGNOSIS; DECISION SUPPORT SYSTEM; HUMAN; IMAGE ENHANCEMENT; METHODOLOGY; PHYSIOLOGY; PROBABILITY; REPRODUCIBILITY; SENSITIVITY AND SPECIFICITY; VIDEORECORDING;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; BEHAVIOR; DECISION SUPPORT TECHNIQUES; HUMANS; IMAGE ENHANCEMENT; IMAGE INTERPRETATION, COMPUTER-ASSISTED; MARKOV CHAINS; PATTERN RECOGNITION, AUTOMATED; REPRODUCIBILITY OF RESULTS; SENSITIVITY AND SPECIFICITY; VIDEO RECORDING;

EID: 34249650859     PISSN: 01628828     EISSN: None     Source Type: Journal    
DOI: 10.1109/TPAMI.2007.1145     Document Type: Article

References (57)

1. K.J. Áström, "Optimal Control of Markov Decision Processes with Incomplete State Estimation," J. Math. Analysis and Applications, vol. 10, pp. 174-205, 1965.
2. M.S. Bartlett, G. Littlewort, B. Braathen, T.J. Sejnowski, and J.R. Movellan, "A Prototype for Automatic Recognition of Spontaneous Facial Actions," Advances in Neural Information Processing Systems, vol. 15, pp. 382-386, 2003.
3. P.N. Belhumeur, J.P. Hespanha, and D.J. Kriegman, "Eigenfaces versus Fisherfaces: Recognition Using Class Specific Linear Projections," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 711-720, July 1997.
4. R.E. Bellman, Dynamic Programming. Princeton Univ. Press, 1957.
5. Y. Bengio and P. Frasconi, "Input-Output HMMs for Sequence Processing," IEEE Trans. Neural Networks, vol. 7, no. 5, pp. 1231-1249, Sept. 1996.
6. M. Black and Y. Yacoob, "Tracking and Recognizing Rigid and Nonrigid Facial Motions Using Local Parametric Models of Image Motions," Int'l J. Computer Vision, vol. 25, no. 1, pp. 23-48, 1997.
7. A.F. Bobick and J.W. Davis, "The Recognition of Human Movement Using Temporal Templates," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 3, pp. 257-267, Mar. 2001.
8. J. Boger, J. Hoey, P. Poupart, C. Boutilier, G. Fernie, and A. Mihailidis, "A Planning System Based on Markov Decision Processes to Guide People with Dementia through Activities of Daily Living," IEEE Trans. Information Technology in Biomedicine, vol. 10, no. 2, pp. 323-333, Apr. 2006.
9. C. Boutilier, T. Dean, and S. Hanks, "Decision Theoretic Planning: Structural Assumptions and Computational Leverage," J. Artificial Intelligence Research, vol. 11, pp. 1-94, 1999.
10. M. Brand, "Learning Concise Models of Human Activity from Ambient Video via a Structure-Inducing M-Step Estimator," Technical Report TR-97-25, Mitsubishi Electric Research Laboratory, Nov. 1997.
11. M. Brand, N. Oliver, and A. Pentland, "Coupled Hidden Markov Models for Complex Action Recognition," Proc. Int'l Conf. Computer Vision and Pattern Recognition, pp. 994-999, 1997.
12. C. Bregler, "Learning and Recognising Human Dynamics in Video Sequences," Proc. IEEE Conf. Computer Vision and Pattern Recognition pp. 568-574, 1997.
13. P. Carbonetto, N. de Freitas, P. Gustafson, and N. Thompson, "Bayesian Feature Weighting for Unsupervised Learning with Application to Object Recognition," Proc. Ninth Int'l Workshop Artificial Intelligence and Statistics, pp. 122-128, Jan. 2003.
14. Embodied Conversational Agents, J. Cassell et al., eds. MIT Press, 2000.
15. L. Chrisman, "Reinforcement Learning with Perceptual Aliasing: The Perceptual Distinctions Approach," Proc. 10th Nat'l Conf. Artificial Intelligence, pp. 183-188, 1992.
16. B. Clarkson and A. Pentland, "Unsupervised Clustering of Ambulatory Audio and Video," Proc. Int'l Conf. Acoustics, Speech, and Signal Processing, 1999.
17. T. Darrell and A.P. Pentland, "Active Gesture Recognition Using Partially Observable Markov Decision Processes," Proc. 13th IEEE Int'l Conf. Pattern Recognition, 1996.
18. T.J. Darrell, I.A. Essa, and A.P. Pentland, "Task-Specific Gesture Analysis in Real-Time Using Interpolated Views," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 18, no. 12, pp. 1236-1242, Dec. 1996.
19. A.P. Dempster, N.M. Laird, and D.B. Rubin, "Maximum Likelihood from Incomplete Data Using the EM Algorithm," J. Royal Statistical Soc. B vol. 39, pp. 1-38, 1977.
20. G. Donato, M.S. Bartlett, J.C. Hager, P. Ekman, and T.J. Sejnowski, "Classifying Facial Actions," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 21, no. 10, pp. 974-989, Oct. 1999.
21. I.A. Essa and A.P. Pentland, "Coding Analysis, Interpretation, and Recognition of Facial Expressions," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 757-763, July 1997.
22. S. Fine, Y. Singer, and N. Tishby, "The Hierarchical Hidden Markov Model: Analysis and Applications," Machine Learning, vol. 32, no. 1, pp. 41-62, 1998.
23. D.J. Fleet, M.J. Black, Y. Yacoob, and A.D. Jepson, "Design and Use of Linear Models for Image Motion Analysis," Int'l J. Computer Vision vol. 36, no. 3, pp. 171-193, 2000.
24. A.J. Fridlund, Human Facial Expression: An Evolutionary View. Academic Press, 1994.
25. B. Galantucci, "An Experimental Study of the Emergence of Human Communication Systems," Cognitive Science, vol. 29, pp. 737-767, 2005.
26. J. Hoey, "Decision Theoretic Learning of Human Facial Displays and Gestures," PhD thesis, Univ. of British Columbia, 2004.
27. J. Hoey and J.J. Little, "Representation and Recognition of Complex Human Motion," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 752-759, June 2000.
28. J. Hoey and J.J. Little, "Bayesian Clustering of Optical Flow Fields," Proc. Int'l Conf. Computer Vision, pp. 1086-1093, Oct. 2003.
29. J. Hoey and P. Poupart, "Solving POMDPs with Continuous or Large Discrete Observation Spaces," Proc. Int'l Joint Conf. Artificial Intelligence, pp. 1332-1338, July 2005.
30. J. Hoey, P. Poupart, C. Boutilier, and A. Mihailidis, "POMDP Models for Assistive Technology," Proc. AAAI Fall Symp. Caring Machines: AI in Eldercare, 2005.
31. J. Hoey, R. St-Aubin, A. Hu, and C. Boutilier, "SPUDD: Stochastic Planning Using Decision Diagrams," Proc. Uncertainty in Artificial Intelligence, pp. 279-288, 1999.
32. E. Hunter, J. Schlenzig, and R. Jain, "Posture Estimation in Reduced-Model Gesture Input Systems," Proc. Int'l Workshop Automatic Face- and Gesture-Recognition, pp. 290-295, 1995.
33. T. Jebara and A.P. Pentland, "Action Reaction Learning: Automatic Visual Analysis and Synthesis of Interactive Behaviour," Proc. Int'l Conf. Vision Systems, pp. 273-292, 1999.
34. R.M. Krauss and S. Glucksberg, "Social and Nonsocial Speech," Scientific Am., vol. 236, pp. 100-105, 1977.
35. A. Lanitis, C.J. Taylor, and T.F. Cootes, "Automatic Interpretation and Coding of Face Images Using Flexible Models," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 743-756, July 1997.
36. C. Li and G. Biswas, "Temporal Pattern Generation Using Hidden Markov Model Based Unsupervised Classification," Advances in Intelligent Data Analysis, 1999.
37. S.X. Liao and M. Pawlak, "On the Accuracy of Zernike Moments for Image Analysis," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 12, pp. 1358-1364, Dec. 1998.
38. D. McNeill, Hand and Mind: What Gestures Reveal about Thought. Univ. of Chicago Press, 1992.
39. K.P. Murphy, "Dynamic Bayesian Networks: Representation, Inference and Learning," PhD thesis, Computer Science Division, Univ. of California Berkeley, July 2002.
40. N. Oliver, A. Garg, and E. Horvitz, "Layered Representations for Learning and Inferring Office Activity from Multiple Sensory Channels," Int'l J. Computer Vision and Image Understanding, vol. 96, pp. 163-180, 2004.
41. T. Paek and E. Horvitz, "Conversation as Action under Uncertainty," Proc. Conf. Uncertainty in Artificial Intelligence, June 2000.
42. A. Pentland, "Looking at People: Sensing for Ubiquitous and Wearable Computing," IEEE Trans. Pattern Analysis and Machine Intelligence vol. 22, no. 1, pp. 107-119, Jan. 2000.
43. P. Poupart and C. Boutilier, "Value-Directed Compression of POMDPs," Advances in Neural Information Processing Systems, vol. 15, pp. 1547-1554, 2003.
44. A. Prata and W.V.T. Rusch, "Algorithm for Computation of Zernike Polynomials Expansion Coefficients," Applied Optics, vol. 28, no. 4, pp. 749-754, Feb. 1989.
45. L.R. Rabiner and B.H. Huang, Fundamentals of Speech Recognition. Prentice Hall, 1993.
46. J.A. Russell and J.M. Fernández-Dols, "What Does Facial Expression Mean?" The Psychology of Facial Expression, pp. 3-30, 1997.
47. J. Schlenzig, E. Hunter, and R. Jain, "Vision Based Hand Gesture Interpretation Using Recursive Estimation," Proc. Asilomar Conf. Signals, Systems, and Computation, pp. 394-399, Oct. 1994.
48. E.P. Simoncelli, E.H. Adelson, and D.J. Heeger, "Probability Distributions of Optical Flow," Proc. Int'l Conf. Computer Vision and Pattern Recognition, pp. 310-315, 1991.
49. P. Smyth, "Clustering Sequences with Hidden Markov Models," Advances in Neural Information Processing Systems, vol. 10, 1997.
50. M.T.J. Spaan and N. Vlassis, "Perseus: Randomized Point-Based Value Iteration for POMDPs," J. Artificial Intelligence Research, vol. 24, pp. 195-220, 2005.
51. T. Starner and A.P. Pentland, "Visual Recognition of American Sign Language Using Hidden Markov Models," Proc. Int'l Workshop Automatic Face and Gesture Recognition, pp. 189-194, 1995.
52. R. Sutton and A.G. Barto, Reinforcement Learning: An Introduction. MIT Press, 1998.
53. C.-H. Teh and R.T. Chin, "On Image Analysis by the Methods of Moments," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 10, no. 4, pp. 496-513, July 1988.
54. S. Thrun, "Probabilistic Algorithms in Robotics," AI Magazine, vol. 21, no. 4, pp. 93-109, 2000.
55. Y. Tian, T. Kanade, and J.F. Cohn, "Recognizing Action Units for Facial Expression Analysis," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 2, Feb. 2001.
56. M. Turk and A.P. Pentland, "Eigenfaces for Recognition," J. Cognitive Neuroscience, vol. 3, no. 1, pp. 71-86, 1991.
57. J. Williams, P. Poupart, and S. Young, "Factored Partially Observable Markov Decision Processes for Dialogue Management," Proc. IJCAI Workshop Knowledge and Reasoning in Practical Dialogue Systems, pp. 76-82, Aug. 2005.