Sensory substitution using tactile pin arrays: Human factors, technology and applications

Signal Processing

Volumn 86, Issue 12, 2006, Pages 3674-3695

  Wall, Steven A ^a   Brewster, Stephen ^a  

^a UNIVERSITY OF GLASGOW   (United Kingdom)

Author keywords

Blind; Haptic; Multimodal; Tactile

Indexed keywords

BLIND; GRAPHICAL INFORMATIONS; TACTILE;

DATA PROCESSING; HAPTIC INTERFACES; MATRIX ALGEBRA; ROBOTICS; SENSORY AIDS; VIRTUAL REALITY;

TACTILE READING AIDS;

EID: 33749430954     PISSN: 01651684     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.sigpro.2006.02.048     Document Type: Article

References (100)

1. Massie T., and Salisbury K. The phantom haptic interface: a device for probing virtual objects (1994), ASME Annual Winter Meeting, New York pp. 295-300
2. Srinivasan M.A., and Basdogan C. Haptics in virtual environments: taxonomy, research status, and challenges. Comput. Graph. 21 4 (1997) 393-404
3. M. Govindaraj, A. Garg, A. Raheja, G. Huang, D. Metaxas, Haptic simulation of fabric hand, Eurohaptics, Dublin, Ireland, 6-9 July 2003, pp. 253-260.
4. Dillon P., Moody W., Bartlett R., Scully P., Morgan R., and James C. Sensing the fabric: to simulate sensation through sensory evaluation and in response to standard acceptable properties of specific materials when viewed as a digital image. In: Brewster S., and Murray-Smith R. (Eds). Haptic Human-Computer Interaction (2001), Springer, Heidelberg 205-217
5. Baillie S., Crossan A., Reid S., and Brewster S. Preliminary development and evaluation of a bovine rectal palpation stimulator for training veterinary students. Cattle Practice. J. Br. Cattle Vet. Assoc. 11 2 (2003) 101-106
6. I.R. Summers, C.M. Chanter, A.L. Southall, A.C. Brady, Results from a tactile array on the fingertip, Eurohaptics, Birmingham, UK, 1-4 July 2001, pp. 26-28.
7. Craig J.C. Tactile pattern perception and its perturbations, J. Acoust. Soc. Am. 77 1 (1985) 238-246
8. Craig J.C., and Sherrick C.E. Dynamic tactile displays. In: Schiff W., and Foulke E. (Eds). Tactual Perception: A Sourcebook (1982), Cambridge University Press, Cambridge 209-233
9. K.A. Kaczmarek, P. Bach-y-rita, Tactile displays, in: W. Barfield, T.A. Furness, (Eds.), Virtual environments and advanced interface design, Oxford University Press, Inc., New York, 1995.
10. Burdea G.C. Force and Touch Feedback for Virtual Reality (1996), Wiley, New York, NY
11. C. Sjostrom, Using haptics in computer interfaces for blind people, CHI 2001, Seattle, WA, 31 March-5 April 2001, pp. 245-246.
12. J.P. Fritz, K. Barner, Design of a haptic graphing system, 19th RESNA Conference, Salt Lake City, UT, 1996.
13. Wies E., Gardner J., O'Modhrain S., and Bulatov V. Web-based touch display for accessible science education. In: Brewster S., and Murray-Smith R. (Eds). Haptic hci (2001), Springer, Lecture Notes in Computer Science, Berlin 52-60
14. W. Yu, S. Brewster, Comparing two haptic interfaces for multimodal graph rendering, IEEE VR2002, 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, Orlando, FL, 2002.
15. Yu W., Reid D., and Brewster S.A. Multimodal virtual reality versus printed medium in visualization for blind people (2002), Proceedings of ACM ASSETS, Edinburgh, Scotland pp. 57-64
16. Yu W., and Brewster S.A. Evaluation of multimodal graphs for blind people. J. Universal Access Inform. Soc. 2 2 (2003) 105-124
17. Foulke E., and Braille R. In: Schiff W., and Foulke E. (Eds). Tactual Perception: A Sourcebook (1982), Cambridge University Press, Cambridge 168-208
18. Gaver W. The sonicfinder: an interface that uses auditory icons. Hum-Comput. Interact. 4 (1989) 67-94
19. H. Petrie, V. Johnson, P. McNally, S. Morley, A.M. O'Neill, D. Majoe, Inexpensive tactile interaction for blind computer users: two application domains, IEE Colloquium on Developments in Tactile Displays, London, UK, 21 January 1997, pp. 2/1-2/3.
20. Challis B.P., and Edwards A.D.N. Design principles for tactile interaction. In: Brewster S., and Murray-Smith R. (Eds). Haptic hci 200 (2001), Springer, Berlin, Heidelberg 17-24
21. Wells L.R., and Landau S. Merging of tactile sensory input and audio data by means of the talking tactile tablet (2003), Eurohaptics 2003, Dublin, Ireland pp. 414-418
22. S. Brewster, L.M. Brown, Tactons: structured tactile messages for non-visual information display, 5th Australian User Interface Conference (AUIC2004), Dunedin, New Zealand, 2004.
23. Brown L.M., Brewster S., and Purchase H. A first investigation into the effectiveness of tactons (2005), World Haptics 2005, Pisa, Italy
24. Linjama J., and Kaaresoja T. Novel, minimalist haptic gesture interaction for mobile devices (2004), third Nordic Conference on Human-computer Interaction, Tampere, Finland pp. 457-458
25. Klatzky R.L., and Lederman S.J. Tactile roughness perception with a rigid link interposed between skin and surface. Percept. Psychophys. 61 4 (1999) 591-607
26. Loomis J.M. Distal attribution and presence. Presence: Teleoperator. Virtual Environ. 1 1 (1992) 113-119
27. Minsky M., Ouh-Young M., Steele O., Brooks Jr. F., and Behensky M. Feeling and seeing: issues in force display. Comput. Graph. 24 2 (1990) 235-243
28. Massie T.H. Initial Haptic Exploration With the Phantom: Virtual Touch Through Point Interaction (1996), Massachussets Institute of Technology, Cambridge, MA
29. Fritz J.P., and Barner K.E. Stochastic models for haptic texture (1996), SPIE International Symposium on Intelligent Systems and Advanced Manufacturing pp. 34-44
30. C. Basdogan, C. Ho, M.A. Srinivasan, A ray-based haptic rendering technique for displaying shape and texture of 3D objects in virtual environments, Proceedings of the ASME Dynamic Systems and Control Division, 1997, pp. 77-84.
31. S.A. Wall, W.S. Harwin, Modelling of surface identifying characteristics using Fourier series, Proceedings of ASME International Mechanical Engineering Congress: Dynamic Systems and Control Division (Haptic Interfaces for Virtual Environments and Teleoperator Systems), Nashville, TN, 1999, pp. 65-71.
32. S. Choi, H.Z. Tan, An analysis of perceptual instability during haptic texture rendering, 10th International Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (IEEE VR 2002), 2002, pp. 129-136.
33. Kaczmarek K.A., Webster J.G., Bach-y-rita P., and Tompkins W.J. Electrotactile and vibrotactile displays for sensory substitution systems. IEEE Trans. Biomed. Eng. 38 (1991) 1-15
34. L.A. Jones, M. Berris, Material discrimination and thermal perception, 11th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, Los Angeles, CA, 22-23 March 2003, pp. 171-178.
35. D.G. Caldwell, C. Gosney, Enhanced tactile feedback (tele-taction) using a multi-functional sensory system, IEEE International Conference on Robotics and Automation, 2-6 May 1993, pp. 955-960.
36. Summers I.R., and Chanter C.M. A broadband tactile array on the fingertip. J. Acoust. Soc. Am. 112 5 (2002) 2118-2126
37. J. Pasquero, V. Hayward, Stress: A practical tactile display with one millimeter spatial resolution and 700 Hz refresh rate, Eurohaptics, Dublin, Ireland, 6-9 July 2003, pp. 94-110.
38. D.G. Caldwell, N. Tsagarakis, C. Giesler, An integrated tactile/shear feedback array for stimulation of finger mechanoreceptor, IEEE International Conference on Robotics and Automation, Detroit, MI, May 1999, pp. 287-292.
39. Loomis J.M., and Lederman S.J. Tactual perception. In: Boff K., Kaufman L., and Thomas J. (Eds). Handbook of Perception and Human Performance (1986), Wiley, New York 31-41
40. Johnson K.O., and Hsiao S.S. Neural mechanisms of tactual form and texture perception. Annu. Rev. Neurosci. 15 (1992) 227-250
41. Johansson R.S., Landstrom U., and Lundstrom R. Responses of mechanoreceptive afferent units in the glabrous skin of the human hand to sinusoidal skin displacements. Brain Res. 244 (1982) 17-25
42. D.A. Kontarinis, R.D. Howe, Display of High Frequency Tactile Information to Teleoperators, 1993, pp. 40-50.
43. Johnson K.O. Neural basis of haptic perception. In: Pashler H., and Yantis S. (Eds). Stevens Handbook of Experimental Psychology (2002), Wiley, New York 537-583
44. Verillo R.T. Effect of contactor area on vibrotactile threshold. J. Acoust. Soc. Am. 35 (1963) 1962-1966
45. Rogers C.H. Choice of stimulator frequency for tactile arrays. IEEE Trans. Man Machine Systems MMS- 11 (1970) 5-11
46. Bolanowski S.J., Gescheider G.A., Verillo R.T., and Checkosky C.M. Four channels mediate the mechanical aspects of touch. J. Acoust. Soc. Am. 84 5 (1988) 1680-1694
47. Sherrick C.E., Cholewiak R.W., and Collins A.A. The localization of low- and high-frequency vibrotactile stimuli. J. Acoust. Soc. Am. 88 1 (1990) 169-179
48. Phillips J.R., and Johnson K.O. Neural mechanisms of scanned and stationary touch. J. Acoust. Soc. Am. 77 1 (1985) 220-224
49. Johansson R.S., and Vallbo A.B. Detection of tactile stimuli. thresholds of afferent units related to psychophysical thresholds in the human hand. J. Physiol. 297 (1979) 405-422
50. Lofvenberg J., and Johansson R.S. Regional differences in sensitivity to vibration in the glabrous skin of the human hand. Brain Res. 301 (1984) 65-72
51. Geldard F.A. The Human Senses (1972), Wiley, NY
52. Werner G., and Mountcastle V.B. Quantitative relations between mechanical stimuli to the skin and neural responses evoked by them. In: Kenshalo D. (Ed). The Skin Senses (1968), Charles C. Thomas, Springfield, IL 112-138
53. Tan H.Z. Information transmission with a multi-finger tactual display, Electrical Engineering and Computing science (1996), Massachussetts, Institute of Technology: Cambridge, MA
54. Goff G.D. Differential discrimination of frequency of cutaneous mechanical vibration. J. Exp. Psychol. 74 (1967) 294-299
55. Geldard F.A. Adventures in tactile literacy. Am. Psychol. 12 (1957) 115-124
56. Meijer P.B.L. An experimental system for auditory image representations. IEEE Trans. Biomed. Eng. 39 2 (1992) 112-121
57. Hill J.W. Limited field of view in reading letter shapes with the fingers. In: Geldard F.A. (Ed). Cutaneous Communication Systems and Devices (1974), Psychonomic Society, Austin, TX 95-105
58. Bliss J.C., Katcher M.H., Rogers C.H., and Shephard R.P. Optical-to-tactile image conversion for the blind. IEEE Trans. Man Machine Systems MMS- 11 (1970) 58-64
59. Loomis J.M., Klatzky R.L., and Lederman S.J. Similarity of tactual and visual picture recognition with limited field of view. Perception 20 (1991) 167-177
60. Loomis J.M. Tactile letter recognition under different modes of stimulus presentation. Percept. Psychophys. 16 (1974) 401-418
61. Bliss J.C., and Linvill J.G. A direct translation reading aid: reading alphabetic shapes tactually (1966), International Conference on Sensory Devices for the Blind pp. 389-407
62. Kops C.E., and Gardner E.P. Discrimination of simulated texture patterns on the human hand, Percept. Psychophys. 76 2 (1996) 1145-1165
63. Boyd L.H., Boyd W.L., and Vanderheiden G.C. The Graphical User Interface Crisis: Danger and Opportunity (1990), University of Wisconsin: Trace R&D Center, UK
64. Krueger L.E. David katz's der aufbau der tastwelt (the world of touch): a synopsis. Percept. Psychophys. 7 6 (1970) 337-341
65. Gibson J.J. Observations on active touch. Psychol. Rev. 69 (1962) 477-491
66. Lederman S.J., and Taylor M.M. Fingertip force surface geometry and the perception of roughness by active touch. Percept. Psychophys. 12 5 (1972) 401-408
67. Jansson G. Haptic perception of outline 2D shape: the contributions of information via the skin, the joints and the muscles. In: Bril B., Ledebt A., Dietrich G., and Roby-Brami A. (Eds). Advances in Perception-Action Coupling (1998), Editions EDK, Paris 25-30
68. Ramstein C. Combining haptic and Braille technologies: design issues and pilot study, ASSETS'96, Vancouver (1996), British Columbia, Canada pp. 37-44
69. T. Maucher, J. Schemmel, K. Meier, The Heidelberg tactile Vision substitution system, International Sensory Aids Conference, Exeter, UK, 23-26 May 2000.
70. Gapenne O., Lenay C., Stewart J., Beriot H., and Meidine D. Prosthetic device and 2D form perception: the role of increasing degrees of parallelism (2001), Proceedings of the Conference on Assistive Technology for Vision and Hearing Impairment (CVHI'2001), Castelvecchio Pascoli, Italy
71. N. Sribunruangrit, C. Marque, C. Lenay, O. Gapenne, C. Vanhoutte, J. Stewart, Application of parallelism concept to access graphic information with precision for blind people, EMBEC'02 2nd European Medical and Biological Engineering Conference, Vienna, Austria, 4-8 December 2002.
72. A. Ali Ammar, O. Gapenne, C. Lenay, J. Stewart, Effect of bimodality on the perception of 2D forms by means of a specific assistive technology for blind persons, CVHI'2002-EURO-ASSIST-VHI-2 Conference on Assistive Technologies for Vision and Hearing Impairment, Granada, Spain, 6-9 August 2002, pp. 45-52.
73. P. Kammermeier, G. Schmidt, Application-specific evaluation of tactile array displays for the human fingertip, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Lausanne, Switzerland, 30 September-4 October 2002, pp. 2937-2942.
74. G. Jansson, P. Pedersen, Obtaining geographical information from a virtual map with a haptic mouse, XXII International Cartographic Conference (ICC2005), A Coruna, Spain, 11-16 July 2005.
75. I.R. Summers, Single-channel information transfer through the skin: limitations and possibilities, International Sensory Aids Conference, Exeter, UK, 23-26 May 2000.
76. P. Bach-y-rita, J.G. Webster, W.J. Tompkins, and T. Crabb, Sensory substitution for space gloves and space robots. Space Telerobotics Workshop, Jet Propulsion Laboratory, Pasadena, CA, 20-22 January 1987, pp. 51-57.
77. K.B. Shimoga, A survey of perceptual feedback issues in dexterous telemanipulation. Part ii. Finger touch feedback, IEEE Virtual Reality Annual International Symposium, New York, September 1993, pp. 271-279.
78. N.J.M. Patrick, Design, Construction and Testing of a Fingertip Tactile Display for Interaction with Virtual and Remote Environments, M.Sc. Thesis, Department of Mechanical Engineering, Massachussets Institute of Technol. Cambridge, MA, 1990.
79. Edin B.B., Howe R., Westling G., and Cutkosky M. A physiological method for relaying frictional information to a human teleoperator. IEEE Trans. System Man Cybernet. 23 2 (1993)
80. Lederman S.J., and Klatzky R.L. Sensing and displaying spatially distributed fingertip forces in haptic interfaces for teleoperator and virtual environment systems. Presence: Teleoperators and Virtual Environments 8 1 (1999) 86-103
81. G. Jansson, L. Monaci, Improving haptic displays: providing differentiated information at the contact areas is more important than increasing the number of areas, World Haptics 2005, Pisa, Italy, 18-20 March 2005.
82. Cohn M.B., Lam M., and Fearing R.S. Tactile feedback for teleoperation. Telemanipulator Technology SPIE 1833 (1992) 240-252
83. Hasser C.J., and Weisenberger J.M. Preliminary evaluation of a shape-memory alloy tactile feedback display, Symposium on Haptic Interfaces for Virtual Environments and Teleoperator Systems (1993), ASME Winter Annual Meeting, New Orleans, LA pp. 73-80
84. R.D. Howe, The shape of things to come: pin-based tactile shape displays, Proceedings of the 4th International Conference Eurohaptics 2004, Munich, Germany, 5-7 June 2004, pp. 2-11.
85. D.A. Kontarinis, J.S. Son, W. Peine, R.D. Howe, A tactile shape sensing and display system for teleoperated manipulation, IEEE International Conference on Robotics and Automation, 1995, pp. 641-646.
86. R.D. Howe, A force-reflecting teleoperated hand system for the study of tactile sensing in precision manipulation, IEEE International Conference on Robotics and Automation, Nice, France, May 1992, pp. 1321-1326.
87. C. Richard, M.R. Cutkosky, Contact force perception with an ungrounded haptic interface, ASME IMECE 6th Annual Symposium on Haptic Interfaces, Dallas, TX, 15-21 November 1997.
88. Lee J.C., Dietz P., Leigh D., Yerazunis W., and Hudson S.E. Haptic pen: a tactile feedback stylus for touch screens (2004), Proceedings of UIST 2004, Santa Fe, NM, USA pp. 291-294
89. Poupyrev I., and Maruyama S. Tactile interfaces for small touch screens (2003), Proceedings of UIST 2003, Vancouver, Canada pp. 217-220
90. I. Poupyrev, S. Maruyama, J. Rekimoto, Ambient touch: designing tactile interfaces for handheld devices, UIST, Paris, France, 27-30 October 2002.
91. Fukumoto M., and Toshaki S. Activeclick: tacile feedback for touch panels (2001), Extended Abstracts of CHI 2001, Seattle, WA, USA pp. 121-122
92. Akamatsu M., MacKenzie I.S., and Hasbrouc T.A. A comparison of tactile auditory, and visual feedback in a pointing task using a mouse-type device. Ergonomics 38 (1995) 816-827
93. Soukoreff R.W., and MacKenzie I.S. Towards a standard for pointing device evaluation: Perspectives on 27 years of fitts' law research in hci. Int. J. Human-Comput. Stud. 61 (2004) 751-789
94. Akamatsu M., and MacKenzie I.S. Movement characteristics using a mouse with tactile and force feedback. Int. J. Human-Comput. Stud. 45 (1996) 483-493
95. Vitense H., Jacko J.A., and Emery V.K. Multimodal feedback: an assessment of performance and mental workload. Ergonomics 46 1-3 (2003) 68-87
96. A. Cockburn, S. Brewster, Multimodal feedback for the acquisition of small targets, Ergonomics 48 (9) (2005) 1129-1150.
97. Brewster, and King A.J. An investigation into the use of tactons to present progress information (2005), Proceedings of IFIP Interact 2005, Rome, Italy
98. Riedel B., and Burton A.M. Perception of gradient in haptic graphs: a comparison of virtual and physical stimuli (2001), Eurohaptics, Birmingham, UK pp. 90-92
99. Gentaz E., and Hatwell Y. Role of gravitational cues in the haptic perception of orientation. Percept. Psychophys. 58 8 (1996) 1278-1292
100. Gentaz E., and Hatwell Y. The haptic oblique effect in the perception of rod orientation by blind adults. Percept. Psychophys. 60 1 (1998) 157-167