Concept-based evidential reasoning for multimodal fusion in human-computer interaction

Applied Soft Computing Journal

Volumn 10, Issue 2, 2010, Pages 567-577

  Reddy, B Srinath ^a   Basir, Otman A ^a  

^a UNIVERSITY OF WATERLOO   (Canada)

Author keywords

Conceptual reasoning; Dempster Shafer theory; Evidential reasoning; Human computer interaction; Multimodal fusion; Transferable belief models

Indexed keywords

BRAIN COMPUTING; CONCEPT-BASED; CONCEPTUAL GRAPH; DEMPSTER-SHAFER THEORY; DOMAIN-SPECIFIC KNOWLEDGE; EVIDENTIAL REASONING; FACIAL GESTURES; HAND-GESTURE RECOGNITION; HCI SYSTEM; HUMAN COMPUTER INTERFACES; HUMAN GESTURES; MINIMAL COST; MULTI-MODAL FUSION; MULTIMODAL SYSTEM; MULTIPLE MODALITIES; SEMANTIC FUSION; TRANSFERABLE BELIEF MODEL;

ELECTRIC CURRENTS; FORMAL LOGIC; GESTURE RECOGNITION; INTERFACES (COMPUTER); KNOWLEDGE BASED SYSTEMS; KNOWLEDGE MANAGEMENT; MULTI AGENT SYSTEMS; SEMANTICS; SENSORS;

HUMAN COMPUTER INTERACTION;

EID: 70649097285     PISSN: 15684946     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.asoc.2009.08.026     Document Type: Article

References (36)

1. Wu Y., Chang E.Y., Chang K.C.-C., and Smith J.R. Optimal multimodal fusion for multimedia data analysis. Proceedings of the ACM International Conference on Multimedia (MM). New York, October (2004) 572-579
2. Malik S., and Laszlo J. Visual touchpad: a two-handed gestural input device. Proceedings of the 6th International Conference on Multimodal Interfaces (ICMI'04). State College, PA, USA, October (2004) 289-296
3. K. Seong-Pal, M. Tordon, J. Katupitiya, Curvature based hand shape recognition for a virtual wheelchair control interface, in: Proceedings of the IEEE International Conference on Robotics and Automation, 2004 (ICRA'04), vol. 2, April 26-May 9, 2004, pp. 2049-2054.
4. Y. Liu, Y. Jia, A robust hand tracking for gesture-based interaction of wearable computers, in: Eighth International Symposium on Wearable Computers, 2004 (ISWC 2004), vol. 1, October 31-November 3, 2004, pp. 22-29.
5. E.C. Paraiso, J.-P.A. Barthes, An intelligent speech interface for personal assistants in RD projects, in: Proceedings of the (IEEE) Ninth International Conference on Computer Supported Cooperative Work in Design, vol. 2, May 24-26, 2005, pp. 804-809.
6. Xie S.Q., Gao C., Yang Z.L., and Wang R.Y. Computer-brain interface. Proceedings of the (IEEE) First International Conference on Neural Interface and Control. Wuhan, China, May 26-28 (2005) 32-36
7. Holzapfel H., Nickel K., and Stiefelhagen R. Implementation and evaluation of a constraint-based multimodal fusion system for speech and 3D pointing gestures. Proceedings of the (ACM) 6th International Conference on Multimodal Interfaces, 2005. State College, PA, USA, October (2004)
8. S. Bangalore, M. Johnston, Integrating multimodal language processing with speech recognition, in: International Conference on Speech and Language Processing, vol. 2, 2000, pp. 126-129.
9. Kaiser E., Olwal A., McGee D., Benko H., Corradini A., Li X., Cohen P., and Feiner S. Mutual disambiguation of 3D multimodal interaction in augmented and virtual reality. Proceedings of the 5th International Conference on Multimodal Interfaces (ICMI 2003). Vancouver, BC, Canada, November (2003) 12-19
10. S.M. Chu, V. Libal, E. Marcheret, C. Neti, G. Potamianos, Multistage information fusion for audio-visual speech recognition, in: Proceedings of the (IEEE) International Conference on Multimedia and Expo, 2004 (ICME'04), June 27-30, 2004, vol. 3, pp. 1651-1654.
11. Corradini A., Mehta M., Bernsen N.O., Martin J.C., and Abrilian S. Multimodal input fusion in human-computer interaction on the example of the on-going NICE project. Proceedings of the NATO-ASI Conference on Data Fusion for Situation Monitoring, Incident Detection, Alert and Response Management, August 18-29 (2003)
12. C. Yang, B. Blyth, An evidential reasoning approach to composite combat identification (CCID), in: Proceedings of the (IEEE) Aerospace Conference, vol. 3, March 6-13, 2004, pp. 2042-2053.
13. B. Miners, O.A. Basir, M. Kamel, Knowledge-based disambiguation of hand gestures, in: Proceedings of the (IEEE) International Conference on Systems, Man and Cybernetics, vol. 5, October 6-9, 2002, pp. 201-206.
14. Miners W.B., Basir O.A., and Kamel M.S. Understanding hand gestures using approximate graph matching. IEEE Transactions on Systems, Man, and Cybernetics-Part A 35 March (2) (2005) 239-248
15. Green A., and Eklundh K.S. Designing for learnability in human-robot communication. IEEE Transactions on Industrial Electronics 50 August (4) (2003) 644-650
16. Smets P. The combination of evidence in transferable belief model. IEEE Transactions on Pattern Analysis and Machine Intelligence 12 (1990) 447-458
17. Mason S.G., and Birch G.E. A general framework for brain-computer interface design. IEEE Transactions on Neural Systems Rehabilitation Engineering 11 March (1) (2003) 70-85
18. Gupta L., Chung B., Srinath M.D., Molfese D.L., and Kook H. Multichannel fusion models for the parametric classification of differential brain activity. IEEE Transactions on Bio Medical Engineering 52 November (11) (2005) 1869-1881
19. Veeramachaneni K., Osadciw L.A., and Varshney P.K. An adaptive multimodal biometric management algorithm. IEEE Transactions on Systems, Man and Cybernetics-Part C 35 August (3) (2005) 344-356
20. Gunatilaka A.H., and Baertlein B.A. Feature-level and decision-level fusion of noncoincidently sampled sensors for land mine detection. IEEE Transactions on Pattern Analysis and Machine Intelligence 23 June (6) (2001) 577-589
21. Lo D., Goubran R.A., and Dansereau R.M. Robust joint audio-video talker localization in video conferencing using reliability information-II: Bayesian network fusion. IEEE Transactions on Instrument Measurement 54 August (4) (2005) 1541-1547
22. Ruthven I., and Lalmas M. Using Dempster-Shafer's theory of evidence to combine aspects of information use. Journal of Intelligent Information Systems 19 3 (2002) 267-302
23. Basir O., and Yuan X. Engine fault diagnosis based on multi-sensor information fusion using Dempster-Shafer evidence theory. Information Fusion 8 October (4) (2007) 379-386
24. Zadeh L. A simple view of the Dempster-Shafer theory of evidence and its implication for the rule of combination. AI Magazine 7 2 (1986) 85-90
25. Pal N.R., Bezdek J.C., and Hemasinha R. Uncertainy measures for evidential reasoning II: a new measure of total uncertainty. International Journal of Approximate Reasoning 8 1 (1993) 1-16
26. K. Sentz, S. Ferson, Combination of evidence in Dempster-Shafer theory, Tech. Report, Sandia National Laboratories, CA, SAND 2002-0835, April 2002.
27. M. Chein, M. Mugnier, Conceptual graphs are also graphs, LIRMM, Universit'e Montpellier II, Tech. Rep. 95003, 1995.
28. Wille R. Conceptual graphs and formal concept analysis. International Conference on Conceptual Structures (1997) 290-303
29. Shafer G. A Mathematical Theory of Evidence (1976), Princeton University Press, New Jersey, USA
30. P. Smets, Belief functions and the transferable belief model, belief.pdf, http://ippserv.rug.ac.be/documentation/belief/belief.pdf, August 2000.
31. D. Harmenac, Measures of uncertainty and information, ippinfom.pdf, http://ensmain.rug.ac.be/ipp, 1999.
32. Sowa J.F. Knowledge Representation Logical, Philosophical, and Computational Foundations (2000), Brooks Cole Publishing Co, Pacific Grove CA
33. Braun J. Demspter-Shafer theory and Bayesian reasoning in multisensor data fusion. Sensor Fusion: Architectures, Algorithms and Applications IV; Proceedings of SPIE 4051 (2000) 255-266
34. Chen Y.M. Information fusion in data association applications. Applied Soft Computing 6 August (4) (2006) 394-405
35. Buche P., Dibie-Barthelemy J., and Ibanescu L. Using fuzzy conceptual graphs to map ontologies. Supplementary Proceedings of the 16th International Conference on conceptual structure (2008) 17-24
36. Cohen L., Avrahami G., Last M., and Kandel A. Info-fuzzy algorithms for mining dynamic data streams. Applied Soft Computing 8 September (4) (2008) 1283-1294