A system for medical consultation and education using multimodal human/machine communication

IEEE Transactions on Information Technology in Biomedicine

Volumn 2, Issue 4, 1998, Pages 282-291

  Akay, Metin ^a,b,c   Marsic, Ivan ^b,c   Medl, Attila ^b,c   Bu, Guangming ^b,c  

^a IEEE   (United States)

^b DARTMOUTH COLLEGE   (United States)

^c RUTGERS UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; COMPUTER INTERFACE; EDUCATION; HUMAN; MAN MACHINE INTERACTION; MEDICAL EDUCATION; METHODOLOGY; TELECONSULTATION;

EDUCATION, DISTANCE; EDUCATION, MEDICAL; HUMANS; MAN-MACHINE SYSTEMS; REMOTE CONSULTATION; USER-COMPUTER INTERFACE;

DISTRIBUTED COMPUTER SYSTEMS; HEALTH CARE; HUMAN COMPUTER INTERACTION; IMAGE ANALYSIS; INTERACTIVE COMPUTER SYSTEMS; MAMMOGRAPHY; MEDICAL EDUCATION; MEDICAL IMAGING; USER INTERFACES; VIDEO CONFERENCING;

MEDICAL CONSULTATION; MULTIMODAL INTERFACES;

COMPUTER AIDED DIAGNOSIS;

EID: 0032216442     PISSN: 10897771     EISSN: None     Source Type: Journal    
DOI: 10.1109/4233.737584     Document Type: Article

References (53)

1. J. K. Aggarwal, Ed., Multisensor Fusion for Computer Vision. New York: Springer-Verlag, 1993.
2. M. Akay, Time-Frequency and Wavelets in Biomédical Signal Processing. Piscataway, NJ: IEEE, 1997.
3. J. Allen, Natural Language Understanding. Los Angeles, CA: Benjamin/Cummings, 1994.
4. M. Andre, G. V. Popescu, A. Shaikh, A. Medl, I. Marsic, C. A. Kulikowski, and J. L. Flanagan, "Integration of speech and gesture for multimodal human-computer interaction," in Proc. 2nd Int. Conf. Cooperative Multimodal Commun., Tilburg, The Netherlands, Jan. 1998, pp. 20-27.
5. R. A. Bolt, "Put-that-there: Voice and gesture at the graphic interface," Comput. Graph., vol. 14, pp. 262-270, Aug. 1980.
6. D. Brzakovic, X. M. Luo, and P. Brzakovic, "An approach to automated detection of tumors in mammograms," IEEE Trans. Med. Imag., vol. MI-9, pp. 232-241, Feb. 1990.
7. H. P. Chan et al., "Computer-aided detection of microcalcifications in mammograms: Methodology and preliminary clinical study," Investigate Radial., vol. 23, no. 9, pp. 664-671, 1988.
8. A. Cheyer and L. Julia, "Speech: A privileged modality," in Proc. EU-ROSPEECH'97, Rhodes, Greece.
9. A. Cheyer, L. Julia, and J. C. Martin, "A theoretical framework for multimodal user studies," in Proc. 2nd Int. Conf. Cooperative Multimodal Commun., Tilburg, The Netherlands, Jan. 1998.
10. CMU Interactive Syst. Labs, Gesture and Speech Integration, Carnegie-Mellon Univ., Pittsburgh, PA, 1998, available at http://www.is.ca.cmu.edu/ISL.multimodal.gesture.html.
11. P. R. Cohen, M. Johnston, D. McGee, S. Oviatt, and J. Pittman, "QuickSet: Multimodal interaction for simulation set-up and control," in Proc. 5th Appl. Natural Language Processing Meeting, Washington, DC, 1997.
12. A. P. Dhawan, G. Buelloni, and R. Gordon, "Enhancement of mammographie features by optimal adaptive neighborhood image processing," IEEE Trans. Med. Imag., vol. MI-5, p. 8, Jan. 1986.
13. A. P. Dhawan, Y. Chitre, and M. Moskowitz, "Artificial neural network based classification of mammographie microcalcifications using image structure features," in Proc. SPIE Conf. Biomed. Image Processing Biomed. Visualization, 1993, pp. 820-831.
14. D. L. Donoho, "De-noising via soft-thresholding," Statistics, Stanford Univ., Stanford, CA, Tech. Rep., 1992.
15. D. L. Donoho and I. M. Johnstone, "Ideal spatial adaptation via wavelet shrinkage," Statistics, Stanford Univ., Stanford, CA, Tech. Rep., 1992.
16. J. L. Flanagan, "Technologies for multimedia communications," Proc. IEEE, vol. 84, pp. 590-603, Apr. 1994.
17. _, "Multimodality," in Survey of the State of the Art of Human Language Technology, R. Cole and J. Mariani, Eds. Cambridge, U.K.: Cambridge Univ. Press, pp. 277-300, to be published, available at http://www.cse.ogi.edu/CSLU/HLTsurvey.
18. _, "Synergistic modalities for human/machine communication," in Proc. IEEE Workshop Speech Recognition Understanding, Santa Barbara, CA, Dec. 1997, plenary talk.
19. J. L. Flanagan and I. Marsic, "Issues in measuring the benefits of multimodal interfaces," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Processing (ICASSP'96), Munich, Germany, Apr. 1997, pp. 163-166, invited article.
20. A. Frazer, "Computer diagnosis comes to mammography," Diagnostic Imaging, pp. 91-95, 1991.
21. R. Gordon and R. M. Ranyaggan, "Feature enhancement of film mammographies using fixed and adaptive neighborhoods," Appl. Opt., vol. 23, pp. 560-564, 1984.
22. L. Gupta and M. D. Srinath, "Contour sequence moments for the classification of closed planar shapes," Pattern Recognit., vol. 20, no. 3, pp. 267-272, 1987.
23. W. Hoeffken and M. Lanyi, Mammography. Philadelphia, PA: Saunders, 1977.
24. Human-Computer Interaction Lab., Res. Project Descriptions, Univ. Maryland, College Park, 1998, available at http://www.es.umd.edu/projects/hcil/Research/projects.html.
25. W. D. Keidel, "Information processing by sensory modalities in man," in Cybernetic Problems in Bionics. New York: Gordon and Breach, 1968, pp. 277-300.
26. A. Laine et al., "Wavelets for contrast enhancement of digital mammography," IEEE EMB Mag., pp. 536-550, Sept. 1995.
27. J. L. Lamarque, An Atlas of the Breast Clinical Radiodiagnosis. London, U.K.: Wolfe, 1981.
28. Q. Lin, E. Jan, and J. L. Flanagan, "Microphone arrays and speaker identification," IEEE Trans. Speech Audio Processing, vol. 2, pp. 622-629, Oct. 1994.
29. S. G. Mallat, "Multiresolution approximations and wavelet orthonormal bases of L2(Ä)," Trans. Amer. Math. Soc., vol. 315, pp. 69-87, 1989.
30. S. Mallat and Hwang, "Singularity detection and processing with waveless," IEEE Trans. Inform. Theory, vol. 38, Mar. 1992.
31. S. Mallat and S. Zhong, "Characterization of signals from multiscale edges," IEEE Trans. Pattern Anal. Machine Intell, vol. 14, pp. 710-732, June 1992.
32. I. Marsic and J. L. Flanagan, "DISCIPLE: A framework for multimodal collaboration in heterogeneous environments," in ACM Comput. Surv., to be published, available at http://www.caip.rutgers.edu/multimedia/groupwareJ.
33. A. Medl, I. Marsic, M. Andre, C. Kulikowski, and J. Flanagan, "Multimodal man-machine interface for mission planning," in Proc. AAAI Spring Symp. Intell. Environ., Stanford, CA, Mar. 1998, pp. 41-57.
34. P. McKevitt, Ed., Integration of Natural Language and Vision Processing: Computational Models and Systems. Dordrecht, The Netherlands: Kluwer, 1995.
35. Navy Res. Lab., Virtual Reality Lab. Web Page, 1997, available at http://www.ait.nrl.navy.mil/vrlab.
36. The Object Management Group, "The common object request broker: Architecture and specification (Revision 2.0)," Object Manage. Group, Inc., Framingham, MA, Tech. Rep., July 1995.
37. S. Oviatt, "User-centered modeling for spoken language and multimodal interfaces," IEEE Multimedia Mag., vol. 3, pp. 26-35, 1996.
38. S. Oviatt, A. DeAngeli, and K. Kühn, "Integration and synchronization of input modes during multimodal human-computer interaction," in Proc. Conf. Human Factors Comput. Syst. (CHI'97), Atlanta, GA, pp. 415-4.
39. W. Qian, L. P. Clarke, M. Kallergi, R. A. Clark, "Time-structured nonlinear filters in digital mammography," IEEE Trans. Med. Imag., vol. 13, pp. 25-36, Jan. 1994.
40. W. B. Richardson, "Wavelet packets applied to mammograms," in Proc. SPIE Symp. Electron. Imaging Sei. Technol., San Jose, CA, Jan. 31-Feb. 5, 1993.
41. L. Rosenblum, J. Durbin, R. Doyle, and D. Täte, "Situational awareness using the VR responsive workbench," IEEE Comput. Graph. Applicat., vol. 17, pp. 13-15, Apr. 1997.
42. B. Shneiderman, Designing the User Interface: Strategies for Effective Human-Computer Interaction, 2nd ed. Reading, MA: Addison-Wesley, 1992.
43. _, "Beyond intelligent machines: Just do it!" IEEE Software, vol. 10, no. 1, pp. 100-103, Jan. 1993.
44. _, "Dynamic queries for visual information seeking," IEEE Trans. Software Eng., vol. 11, pp. 70-77, June 1994.
45. F. Shtern, "Digital mammography and related technologies: A perspective from the national cancer institute," Radiology, vol. 183, pp. 629-630, 1992.
46. R. Sharma, V. I. Pavlovic, and T. S. Huang, "Toward multimodal human-computer interface," Proc. IEEE, vol. 86, pp. 853-869, May 1998.
47. R. Sharma, T. S. Huang, and V. I. Pavlovic, "Multimodal framework for interacting with virtual environments," in Human Interaction With Complex Systems: Conceptual Principles and Design Practice, C. E. Ntuen and E. H. Park, Eds. Dordrecht, The Netherlands: Kluwer, 1996, pp. 53-71.
48. P. G. Tachoes et al., "Enhancement of chest and breast radiography by automatic spatial filtering," IEEE Trans. Med. Imag., vol. MI-10, pp. 330-335, Feb. 1991.
49. M. T. Vo, R. Houghton, J. Yang, U. Bub, U. Meier, A. Waibel, and P. Duchnowski, "Multimodal learning interfaces," in Proc. ARPA Spoken Language Technol. Workshop, Austin, TX, Jan. 1995.
50. P. H. Winston, Artificial Intelligence. Reading, MA: Addison-Wesley, 1992.
51. A. Waibel, M. T. Vo, P. Duchnowski, and S. Manke, "Multimodal interfaces," Artif. Intell. Rev., vol. 10, nos. 3/4, 1995.
52. M. Wingfield, Applied Visualization Techniques for Virtual Environments. Bedford, MA: MITRE, 1997.
53. D. Comaniciu, P. Meer, D. Foran, and A. Medl, "Bimodal system for interactive indexing and retrieval of pathology images," in Proc. 4th IEEE Workshop Applicat. Comput. Vision (WACV'98), Princeton, NJ, pp. 76-81 and 268-269.