Saliency-driven real-time video-to-tactile translation

IEEE Transactions on Haptics

Volumn 7, Issue 3, 2014, Pages 394-404

  Kim, Myongchan ^a   Lee, Sungkil ^a   Choi, Seungmoon ^b  

^a Pohang University of Science and Technology (POSTECH)   (South Korea)

^b Sungkyunkwan University   (South Korea)

Author keywords

4D film; Authoring; Multimedia; Tactile effect; Visual saliency

Indexed keywords

AUTHORING; MULTIMEDIA; TACTILE EFFECT; VISUAL SALIENCY;

ALGORITHM; HUMAN; IMAGE PROCESSING; PHOTOSTIMULATION; PHYSIOLOGY; PROCEDURES; TOUCH;

ALGORITHMS; HUMANS; IMAGE PROCESSING, COMPUTER-ASSISTED; PHOTIC STIMULATION; TOUCH;

EID: 84907517821     PISSN: 19391412     EISSN: None     Source Type: Journal    
DOI: 10.1109/TOH.2013.58     Document Type: Article

References (38)

1. S. Choi and K.J. Kuchenbecker, "Vibrotactile Display: Perception, Technology, and Applications," Proc. IEEE, vol. 101, no. 9, pp. 2093-2104, Sept. 2013.
2. H.M. Hamed and R.A. Hema, "The Application of the 4-D Movie Theater System in Egyptian Museums for Enhancing Cultural Tourism," J. Tourism, vol. 10, no. 1, pp. 37-53, 2009.
3. E. Oh, M. Lee, and S. Lee, "How 4D Effects Cause Different Types of Presence Experience ?" Proc. Int'l Conf. Virtual Reality Continuum and Its Applications in Industry, pp. 375-378, 2011.
4. C. Collins, "Tactile Television-Mechanical and Electrical Image Projection," IEEE Trans. Man-Machine Systems, vol. 11, no. 1, pp. 65-71, Mar. 1970.
5. Y. Kim, J. Cha, J. Ryu, and I. Oakley, "A Tactile Glove Design and Authoring System for Immersive Multimedia," IEEE Multimedia, vol. 17, no. 3, pp. 34-45, July 2010.
6. P. Lemmens, F. Crompvoets, D. Brokken, J. van den Eerenbeemd, and G. de Vries, "A Body-Conforming Tactile Jacket to Enrich Movie Viewing," Proc. World Haptics '09 - Third Joint EuroHaptics Conf. and Symp. Haptic Interfaces for Virtual Environment and Teleoperator Systems, pp. 7-12, 2009.
7. M. Karam, F.A. Russo, and D.I. Fels, "Designing the Model Human Cochlea: An Ambient Crossmodal Audio-Tactile Display," IEEE Trans. Haptics, vol. 2, no. 3, pp. 160-169, July-Sept. 2009.
8. Y. Visell, J.R. Cooperstock, B.L. Giordano, K. Franinovic, A. Law, S. McAdams, K. Jathal, and F. Fontana, "A Vibrotactile Device for Display of Virtual Ground Materials in Walking," Proc. Eurohaptics '08, pp. 420-426, 2008.
9. I. Hwang, H. Lee, and S. Choi, "Real-Time Dual-Band Haptic Music Player for Mobile Devices," IEEE Trans. Haptics, vol. 6, no. 3, pp. 340-351, July-Sept. 2013.
10. J. Lee and S. Choi, "Real-Time Perception-Level Translation from Audio Signals to Vibrotactile Effects," Proc. ACM SIGCHI Conf. Human Factors in Computing Systems, pp. 2567-2576, 2013.
11. F. Danieau, A. Lécuyer, P. Guillotel, J. Fleureau, N. Mollet, and M. Christie, "Enhancing Audiovisual Experience with Haptic Feedback: A Survey on HAV," IEEE Trans. Haptics, vol. 6, no. 2, pp. 193-205, Apr.-June 2013.
12. B.-C. Lee, J. Lee, J. Cha, C. Seo, and J. Ryu, "Immersive Live Sports Experience with Vibrotactile Sensation," Proc. IFIP TC13 Int'l Conf. Human-Computer Interaction, pp. 1042-1045, 2005.
13. S. ur Réhman, J. Sun, L. Liu, and H. Li, "Turn Your Mobile into the Ball: Rendering Live Football Game Using Vibration," IEEE Trans. Multimedia, vol. 10, no. 6, pp. 1022-1033, Oct. 2008.
14. T. Yamaguchi, A. Akabane, J. Murayama, and M. Sato, "Automatic Generation of Haptic Effect into Published 2D Graphics," Proc. EuroHaptics Conf., 2006.
15. C. Spence and J. Driver, Crossmodal Space and Crossmodal Attention. Oxford Univ. Press, 2004.
16. H.Z. Tan, R. Gray, J.J. Young, and R. Traylor, "A Haptic Back Display for Attentional and Directional Cueing," Haptics-e, vol. 3, no. 1, pp. 1-20, 2003.
17. A. Sklar and N. Sarter, "Good Vibrations: Tactile Feedback in Support of Attention Allocation and Human-Automation Coordination in Event-Driven Domains," Human Factors, vol. 41, no. 4, pp. 543-552, 1999.
18. N. Sarter, "The Need for Multisensory Interfaces in Support of Effective Attention Allocation in Highly Dynamic Event-Driven Domains: The Case of Cockpit Automation," Int'l J. Aviation Psychology, vol. 10, no. 3, pp. 231-245, 2000.
19. C. Ho, H. Tan, and C. Spence, "Using Spatial Vibrotactile Cues to Direct Visual Attention in Driving Scenes," Transportation Research Part F: Traffic Psychology and Behaviour, vol. 8, no. 6, pp. 397-412, 2005.
20. C.E. Connor, H.E. Egeth, and S. Yantis, "Visual Attention: Bottom-Up versus Top-Down," Current Biology, vol. 14, no. 19, pp. R850-R852, 2004.
21. D. Parkhurst, K. Law, and E. Niebur, "Modeling the Role of Salience in the Allocation of Overt Visual Attention," Vision Research, vol. 42, no. 1, pp. 107-123, 2002.
22. D. Hoffman and M. Singh, "Salience of Visual Parts," Cognition, vol. 63, no. 1, pp. 29-78, 1997.
23. A.M. Treisman and G. Gelade, "A Feature-Integration Theory of Attention," Cognitive Psychology, vol. 12, pp. 97-136, 1980.
24. R. Desimone and J. Duncan, "Neural Mechanisms of Selective Visual Attention," Ann. Rev. of Neuroscience, vol. 18, no. 1, pp. 193-222, 1995.
25. M. Kim, S. Lee, and S. Choi, "Saliency-Driven Tactile Effect Authoring for Real-Time Visuotactile Feedback," Proc. Int'l Conf. Haptics: Perception, Devices, Mobility, and Comm. - Volume Part I (Eurohaptics '12), pp. 258-269, 2012.
26. J.P. Gottliebet al., "The Representation of Visual Salience in Monkey Parietal Cortex," Nature, vol. 391, no. 6666, pp. 481-484, 1998.
27. Z. Li, "A Saliency Map in Primary Visual Cortex," Trends in Cognitive Sciences, vol. 6, no. 1, pp. 9-16, 2002.
28. S.-i. Amari, "Dynamics of Pattern Formation in Lateral-Inhibition Type Neural Fields," Biological Cybernetics, vol. 27, no. 2, pp. 77-87, 1977.
29. J. Allman, F. Miezin, and E. McGuinness, "Stimulus Specific Responses from Beyond the Classical Receptive Field: Neurophysiological Mechanisms for Local-Global Comparisons in Visual Neurons," Ann. Rev. Neuroscience, vol. 8, no. 1, pp. 407-430, 1985.
30. L. Itti, C. Koch, and E. Niebur, "A Model of Saliency-Based Visual Attention for Rapid Scene Analysis," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 11, pp. 1254-1259, Nov. 1998.
31. K. Nakayamaet al., "Serial and Parallel Processing of Visual Feature Conjunctions," Nature, vol. 320, no. 6059, pp. 264-265, 1986.
32. C.I.E, "Recommendations on Uniform Colour Spaces, Colour Difference Equations, Psychometric Colour Terms," supplement No.2 to CIE publication No.15 (E.-1.3.1) 1971/(TC-1.3.), 1978.
33. C. Liu, P.C. Yuen, and G. Qiu, "Object Motion Detection Using Information Theoretic Spatio-Temporal Saliency," Pattern Recognition, vol. 42, no. 11, pp. 2897-2906, 2009.
34. R.C. Gonzalez and R.E. Woods, Digital Image Processing. Prentice Hall Press, 2002.
35. S. Lee, G.J. Kim, and S. Choi, "Real-Time Tracking of Visually Attended Objects in Virtual Environments and Its Application to LOD," IEEE Trans. Visualization and Computer Graphics, vol. 15, no. 1, pp. 6-19, Jan./Feb. 2009.
36. S. Weinstein, "Intensive and Extensive Aspects of Tactile Sensitivity as a Function of Body Part, Sex, and Laterality," Proc. Int'l Symp. Skin Senses, pp. 195-222, 1968.
37. J. Ryu and S. Choi, "posVibEditor: Graphical Authoring Tool of Vibrotactile Patterns," Proc. IEEE Int'l Workshop Haptic, Audio and Visual Environments and Games, pp. 120-125, 2008.
38. J. Ryu, J. Jung, G. Park, and S. Choi, "Psychophysical Model for Vibrotactile Rendering in Mobile," Presence: Teleoperators and Virtual Environments, vol. 19, no. 4, pp. 364-387, 2010.