Multisensory technology for flavor augmentation: A mini review

Frontiers in Psychology

Volumn 9, Issue JAN, 2018, Pages

  Velasco, Carlos ^a   Obrist, Marianna ^b   Petit, Olivia ^c   Spence, Charles ^d  

^a BI NORWEGIAN BUSINESS SCHOOL   (Norway)

^b UNIVERSITY OF SUSSEX   (United Kingdom)

^c OMNES Education   (France)

^d UNIVERSITY OF OXFORD   (United Kingdom)

Author keywords

Audition; Augmentation; Experience; Food; Haptics; Multisensory; Touch; Vision

Indexed keywords

EID: 85041600190     PISSN: None     EISSN: 16641078     Source Type: Journal    
DOI: 10.3389/fpsyg.2018.00026     Document Type: Short Survey

References (67)

1. Biggs, L., Juravle, G., and Spence, C. (2016). Haptic exploration of plateware alters the perceived texture and taste of food. Food Qual. Prefer. 50, 129-134. doi: 10.1016/j.foodqual.2016.02.007
2. Bruijnes, M., Huisman, G., and Heylen, D. (2016). "Tasty tech: human-food interaction and multimodal interfaces," in Proceedings of the 1st Workshop on Multi-Sensorial Approaches to Human-Food Interaction (MHFI '16), eds A. Nijholt, C. Velasco, G. Huisman, and K. Karunanayaka (New York, NY: ACM). doi: 10.1145/3007577.3007581
3. Choi, J. H. J., Foth, M., and Hearn, G. (eds) (2014). Eat, Cook, Grow: Mixing Human-Computer Interactions with Human-Food Interactions. Cambridge, MA: MIT Press.
4. Comber, R., Choi, J. H. J., Hoonhout, J., and O'Hara, K. (2014). Designing for human-food interaction: an introduction to the special issue on 'food and interaction design'. Int. J. Hum. Comput. Stud. 72, 181-184. doi: 10.1016/j.ijhcs.2013.09.001
5. Crisinel, A.-S., Cosser, S., King, S., Jones, R., Petrie, J., and Spence, C. (2012). A bittersweet symphony: systematically modulating the taste of food by changing the sonic properties of the soundtrack playing in the background. Food Qual. Prefer. 24, 201-204. doi: 10.1016/j.foodqual.2011.08.009
6. Elder, R. S., and Mohr, G. S. (2016). The crunch effect: food sound salience as a consumption monitoring cue. Food Qual. Prefer. 51, 39-46. doi: 10.1016/j.foodqual.2016.02.015
7. Endo, H., Ino, S., and Fujisaki, W. (2016). The effect of a crunchy pseudo-chewing sound on perceived texture of softened foods. Physiol. Behav. 167, 324-331. doi: 10.1016/j.physbeh.2016.10.001
8. Hashimoto, K., and Nakamoto, T. (2016). "Olfactory display using surface acoustic wave device and micropumps for wearable applications," in Proceedings of the 2016 IEEE Virtual Reality (VR), Greenville, SC, 179-180. doi: 10.1109/VR.2016.7504712
9. Hashimoto, Y., Nagaya, N., Kojima, M., Miyajima, S., Ohtaki, J., Yamamoto, A., et al. (2006). "Straw-like user interface: virtual experience of the sensation of drinking using a straw," in Proceedings of the 2006 ACM SIGCHI International Conference on Advances in Computer Entertainment Technology (ACE '06), (New York, NY: ACM). doi: 10.1145/1178823.1178882
10. Haverkamp, M. (2013). Synesthetic Design: Handbook for a Multi-Sensory Approach. Basel: Birkhäuser Verlag.
11. Hirose, M., Iwazaki, K., Nojiri, K., Takeda, M., Sugiura, Y., and Inami, M. (2015). "Gravitamine spice: a system that changes the perception of eating through virtual weight sensation," in Proceedings of the 6th Augmented Human International Conference (AH '15), (New York, NY: ACM), 33-40. doi: 10.1145/2735711.2735795
12. Huisman, G., Bruijnes, M., and Heylen, D. K. J. (2016). "A moving feast: Effects of color, shape and animation on taste associations and taste perceptions," in Proceedings of the 13th International Conference on Advances in Computer Entertainment Technology (ACE 2016), (New York, NY: ACM), 12. doi: 10.1145/3001773.3001776
13. Ikeno, S., Okazaki, R., Hachisu, T., Sato, M., and Kajimoto, H. (2013). "Audio-haptic rendering of water being poured from sake bottle," in Proceeding of the 10th International Conference on Advances in Computer Entertainment - Volume 8253 (ACE 2013), eds D. Reidsma, H. Katayose, and A. Nijholt (New York, NY: Springer-Verlag), 548-551. doi: 10.1007/978-3-319-03161-3_50
14. Ikeno, S., Watanabe, R., Okazaki, R., Hachisu, T., Sato, M., and Kajimoto, H. (2015). "Change in the amount poured as a result of vibration when pouring a liquid," in Haptic Interaction. Lecture Notes in Electrical Engineering, Vol. 277, eds H. Kajimoto, H. Ando, and K. U. Kyung (Tokyo: Springer), 7-11.
15. Iwata, H., Yano, H., Uemura, T., and Moriya, T. (2004). "Food simulator: a haptic interface for biting," in Proceedings of the IEEE Virtual Reality (Washington, DC: IEEE Computer Society), 51-57.
16. Kadomura, A., Nakamori, R., Tsukada, K., and Siio, I. (2011). "EaTheremin," in Proceedings of the SIGGRAPH Asia 2011 Emerging Technologies (SA '11), (New York, NY: ACM). doi: 10.1145/2073370.2073376
17. Kadomura, A., Tsukada, K., and Siio, I. (2013). "EducaTableware: computer-augmented tableware to enhance the eating experiences," in Proceedings of the CHI '13 Extended Abstracts on Human Factors in Computing Systems (CHI EA '13), (New York, NY: ACM), 3071-3074. doi: 10.1145/2468356.2479613
18. Kakutani, Y., Narumi, T., Kobayakawa, T., Kawai, T., Kusakabe, Y., Kunieda, S., et al. (2017). Taste of breath: the temporal order of taste and smell synchronized with breathing as a determinant for taste and olfactory integration. Sci. Rep. 7:8922. doi: 10.1038/s41598-017-07285-7
19. Kantono, K., Hamid, N., Shepherd, D., Yoo, M. J., Grazioli, G., and Carr, B. T. (2016). Listening to music can influence hedonic and sensory perceptions of gelati. Appetite 100, 244-255. doi: 10.1016/j.appet.2016.02.143
20. Kita, Y., and Rekimoto, J. (2013). "Spot-light: multimodal projection mapping on food," in HCI International 2013-Posters' Extended Abstracts. HCI 2013. Communications in Computer and Information Science, Vol. 374, ed. C. Stephanidis (Berlin: Springer), 652-655.
21. Koizumi, N., Tanaka, H., Uema, Y., and Inami, M. (2011). "Chewing jockey: augmented food texture by using sound based on the cross-modal effect," in Proceedings of the 8th International Conference on Advances in Computer Entertainment Technology (ACE '11), eds T. Romão, N. Correia, M. Inami, H. Kato, R. Prada, T. Terada, et al. (New York, NY: ACM), 4. doi: 10.1145/2071423.2071449
22. Krishna, A., and Morrin, M. (2008). Does touch affect taste? The perceptual transfer of product container haptic cues. J. Consum. Res. 34, 807-818.
23. Lingle, R. (2017). AR and VR in packaging: Beyond the buzz. Available at: http://www.packagingdigest.com/packaging-design/arvr-packaging-beyond-the-buzz1707
24. Michail, N. (2017). From Marketing to Taste: How Virtual Reality will Change the Food Industry. Available at: www.foodnavigator.com/Market-Trends/From-marketing-to-taste-How-virtual-reality-will-change-the-food-industry
25. Michel, C., Velasco, C., and Spence, C. (2015). Cutlery matters: heavy cutlery enhances diners' enjoyment of the food served in a realistic dining environment. Flavour 4, 1-8. doi: 10.1186/s13411-015-0036-y
26. Nambu, A., Narumi, T., Nishimura, K., Tanikawa, T., and Hirose, M. (2010). "Visual-olfactory display using olfactory sensory map," in Proceedings of the 2010 IEEE Virtual Reality (VR), Waltham, MA, 39-42. doi: 10.1109/VR.2010.5444817
27. Narumi, N. (2016). "Multi-sensorial virtual reality and augmented human food interaction," in Proceedings of the 1st Workshop on Multi-sensorial Approaches to Human-Food Interaction (MHFI '16), eds A. Nijholt, C. Velasco, G. Huisman, and K. Karunanayaka (New York, NY: ACM), 6. doi: 10.1145/3007577.3007587
28. Narumi, T., Nishizaka, S., Kajinami, T., Tanikawa, T., and Hirose, M. (2011). "Meta Cookie+: an illusion-based gustatory display," in Proceedings of the 2011 International Conference on Virtual and Mixed Reality: New Trends - Volume Part I, ed. R. Shumaker (Berlin: Springer-Verlag), 260-269. doi: 10.1007/978-3-642-22021-0_29
29. Nijholt, A., Velasco, C., Karunanayaka, K., and Huisman, G. (2016). "1st international workshop on multi-sensorial approaches to human-food interaction (workshop summary)," in Proceedings of the 18th ACM International Conference on Multimodal Interaction (ICMI 2016), (Tokyo: ACM), 601-603. doi: 10.1145/2993148.3007633
30. Nishizawa, M., Jiang, W., and Okajima, K. (2016). "Projective-AR system for customizing the appearance and taste of food," in Proceedings of the 2016 Workshop on Multimodal Virtual and Augmented Reality (MVAR '16), (New York, NY: ACM), 6. doi: 10.1145/3001959.3001966
31. Obrist, C., Velasco, C., Vi, C. T., Ranasinghe, N., Israr, A., Cheok, A. D., et al. (2016). Sensing the future of HCI: Touch, taste, & smell user interfaces. Interactions 23, 40-49. doi: 10.1145/2973568
32. Obrist, M., Gatti, E., Maggioni, E., Vi, C. T., and Velasco, C. (2017). Multisensory experiences in HCI. IEEE MultiMedia 24, 9-13. doi: 10.1109/MMUL.2017.33
33. Okajima, K., and Spence, C. (2011). Effects of visual food texture on taste perception. i-Perception 2:966. doi: 10.1068/ic966
34. Okajima, K., Ueda, J., and Spence, C. (2013). Effects of visual texture on food perception. J. Vis. 13, 1078-1078. doi: 10.1167/13.9.1078
35. Petit, O., Cheok, A. D., and Oullier, O. (2016). Can food porn make us slim? How brains of consumers react to food in digital environments. Integr. Food Nutr. Metab. 3, 251-255. doi: 10.15761/IFNM.1000138
36. Petit, O., Cheok, A. D., Spence, C., Velasco, C., and Karunanayaka, K. T. (2015). "Sensory marketing in light of new technologies," in Proceedings of the 12th International Conference on Advances in Computer Entertainment Technology (ACE '15), (New York, NY: ACM), 4. doi: 10.1145/2832932.2837006
37. Piqueras-Fiszman, B., and Spence, C. (2015). Sensory expectations based on product-extrinsic food cues: an interdisciplinary review of the empirical evidence and theoretical accounts. Food Qual. Prefer. 40, 165-179. doi: 10.1016/j.foodqual.2014.09.013
38. Prescott, J. (2015). Multisensory processes in flavour perception and their influence on food choice. Curr. Opin. Food Sci. 3, 47-52. doi: 10.1016/j.cofs.2015.02.007
39. Ranasinghe, N., Lee, K.-Y., and Do, E. Y. L. (2014). "FunRasa: an interactive drinking platform," in Proceedings of the 8th International Conference on Tangible, Embedded and Embodied Interaction (TEI '14), (New York, NY: ACM), 133-136.
40. Sawada, R., Sato, W., Toichi, M., and Fushiki, T. (2017). Fat content modulates rapid detection of food: a visual search study using fast food and Japanese diet. Front. Psychol. 8:1033. doi: 10.3389/fpsyg.2017.01033
41. Schifferstein, H. N., and Desmet, P. M. (2008). Tools facilitating multi-sensory product design. Des. J. 11, 137-158. doi: 10.2752/175630608X329226
42. Schroeder, P. A., Lohmann, J., Butz, M. V., and Plewnia, C. (2016). Behavioral bias for food reflected in hand movements: a preliminary study with healthy subjects. Cyberpsychol. Behav. Soc. Netw. 19, 120-126. doi: 10.1089/cyber.2015.0311
43. Shermer, D. Z., and Levitan, C. A. (2014). Red hot: the crossmodal effect of color intensity on perceived piquancy. Multisens. Res. 27, 207-223. doi: 10.1163/22134808-00002457
44. Slocombe, B. G., Carmichael, D. A., and Simner, J. (2016). Cross-modal tactile-taste interactions in food evaluations. Neuropsychologia 88, 58-64. doi: 10.1016/j.neuropsychologia.2015.07.011
45. Spence, C. (2015a). Multisensory flavor perception. Cell 161, 24-35. doi: 10.1016/j.cell.2015.03.007
46. Spence, C. (2015b). Eating with our ears: assessing the importance of the sounds of consumption on our perception and enjoyment of multisensory flavour experiences. Flavour 4:3. doi: 10.1186/2044-7248-4-3
47. Spence, C. (2017). Gastrophysics: The New Science of Eating. London: Viking Penguin.
48. Spence, C., Obrist, M., Velasco, C., and Ranasinghe, N. (2017). Digitizing the chemical senses: possibilities & pitfalls. Int. J. Hum. Comput. Stud. 107, 62-74. doi: 10.1016/j.ijhcs.2017.06.003
49. Spence, C., Okajima, K., Cheok, A. D., Petit, O., and Michel, C. (2016). Eating with our eyes: from visual hunger to digital satiation. Brain Cogn. 110, 53-63. doi: 10.1016/j.bandc.2015.08.006
50. Spence, C., and Piqueras-Fiszman, B. (2013). Technology at the dining table. Flavour 2:16. doi: 10.1186/2044-7248-2-16
51. Spence, C., Velasco, C., and Knoeferle, K. (2014). A large sample study on the influence of the multisensory environment on the wine drinking experience. Flavour 3:1-12. doi: 10.1186/2044-7248-3-8
52. Stewart, J. (2017). Digital Installation Transforms Restaurant Into Immersive Dining Experience. Available at: http://mymodernmet.com/teamlab-sagaya-interactive-restaurants/
53. Stroebele, N., and De Castro, J. M. (2004). Effect of ambience on food intake and food choice. Nutrition 20, 821-838. doi: 10.1016/j.nut.2004.05.012
54. Tsutsui, Y., Hirota, K., Nojima, T., and Ikei, Y. (2016). "High-resolution tactile display for lips," in Human Interface and the Management of Information: Applications and Services. HIMI 2016. Lecture Notes in Computer Science, Vol. 9735, ed. S. Yamamoto (Berlin: Springer), 357-366.
55. van Rompay, T. J., Finger, F., Saakes, D., and Fenko, A. (2017). "See me, feel me": effects of 3D-printed surface patterns on beverage evaluation. Food Qual. Prefer. 62, 332-339. doi: 10.1016/j.foodqual.2016.12.002
56. Velasco, C., Carvalho, F. R., Petit, O., and Nijholt, A. (2016a). "A multisensory approach for the design of food and drink enhancing sonic systems," in Proceedings of the 1st Workshop on Multi-sensorial Approaches to Human-Food Interaction (MHFI '16), eds A. Nijholt, C. Velasco, G. Huisman, and K. Karunanayaka (New York, NY: ACM), 7. doi: 10.1145/3007577.3007578
57. Velasco, C., Wan, C., Knoeferle, K., Zhou, X., Salgado-Montejo, A., and Spence, C. (2015). Searching for flavor labels in food products: the influence of color-flavor congruence and association strength. Front. Psychol. 6:301. doi: 10.3389/fpsyg.2015.00301
58. Velasco, C., Woods, A. T., Petit, O., Cheok, A. D., and Spence, C. (2016b). Crossmodal correspondences between taste and shape, and their implications for product packaging: A review. Food Qual. Prefer. 52, 17-26. doi: 10.1016/j.foodqual.2016.03.005
59. Verhagen, J. V., and Engelen, L. (2006). The neurocognitive bases of human multimodal food perception: sensory integration. Neurosci. Biobehav. Rev. 30, 613-650. doi: 10.1016/j.neubiorev.2005.11.003
60. Vi, C. T., Gatti, E., Ablart, D., Velasco, C., and Obrist, M. (2017). Not just see, but feel, smell, and taste the art: a case study on the creation and evaluation of multisensory art experiences in the museum. Int. J. Hum. Comput. Stud. 108, 1-14. doi: 10.1016/j.ijhcs.2017.06.004
61. Wei, J., Wang, X., Peiris, R. L., Choi, Y., Martinez, X. R., Tache, R., et al. (2011). "CoDine: an interactive multi-sensory system for remote dining," in Proceedings of the 13th International Conference on Ubiquitous Computing (UbiComp '11), (New York, NY: ACM), 21-30. doi: 10.1145/2030112.2030116
62. Wiederhold, B. K., Riva, G., and Gutiérrez-Maldonado, J. (2016). Virtual reality in the assessment and treatment of weight-related disorders. Cyberpsychol. Behav. Soc. Netw. 19, 67-73. doi: 10.1089/cyber.2016.0012
63. Woods, A. T., Poliakoff, E., Lloyd, D. M., Kuenzel, J., Hodson, R., Gonda, H., et al. (2011). Effect of background noise on food perception. Food Qual. Prefer. 22, 42-47. doi: 10.1016/j.foodqual.2010.07.003
64. Yan, K. S., and Dando, R. (2015). A crossmodal role for audition in taste perception. J. Exp. Psychol. 41, 590-596. doi: 10.1037/xhp0000044
65. Yap, S. (2016). Rise of the Machines: How Technology is Shaking up the Dining World. Available at: www.lifestyleasia.com/481919/rise-machines-technology-shaking-dining-world/
66. Youssef, J., Youssef, L., Juravle, G., and Spence, C. (2017). Plateware and slurping influence regular consumers' sensory discriminative and hedonic responses to a hot soup. Int. J. Gastron. Food Sci. 9, 100-104. doi: 10.1016/j.ijgfs.2017.06.005
67. Zampini, M., and Spence, C. (2004). The role of auditory cues in modulating the perceived crispness and staleness of potato chips. J. Sens. Stud. 19, 347-363. doi: 10.1111/j.1745-459x.2004.080403.x