Evaluating Mobile Applications in Virtual Environments: A Survey

International Journal of Mobile Human Computer Interaction

Volumn 5, Issue 4, 2013, Pages 1-19

  Delikostidis, Ioannis ^a   Fechner, Thore ^a   Fritze, Holger ^a   Abdelmouty, Ahmed Mahmoud ^a   Kray, Christian ^a  

^a UNIVERSITY OF MÜNSTER   (Germany)

Author keywords

Evaluation Methodologies; Mobile Applications; Survey; User Studies; Virtual Environments

Indexed keywords

EID: 85001777604     PISSN: 1942390X     EISSN: 19423918     Source Type: Journal    
DOI: 10.4018/ijmhci.2013100101     Document Type: Article

References (86)

1. Akamatsu M., MacKenzie I. S., Hasbrouq T. (1995). A comparison of tactile, auditory, and visual feedback in a pointing task using a mouse-type device. Ergonomics, 38, 816–827. 10.1080/00140139508925152 7729406
2. Alexandrova, I. V., Teneva, P. T., de la Rosa, S., Kloos, U., B#ülthoff, H. H., & Mohler, B. J. (2010). Egocentric distance judgments in a large screen display immersive virtual environment. In Proceedings of the 7th Symposium on Applied Perception in Graphics and Visualization (pp. 57–60). New York, NY: ACM.
3. Baharuddin R., Singh D., Razali R. (2013). Research Journal of Applied Sciences. Engineering and Technology, 5(6), 2225–2231.
4. Ballendat, T., Marquardt, N., & Greenberg, S. (2010). Proxemic interaction: designing for a proximity and orientation-aware environment. In Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces (pp. 121–130). New York, NY: ACM.
5. Barton J. J., Vijayaraghavan V. (2003). UBIWISE, a simulator for ubiquitous computing systems design (Tech. Report). HEWLETT-PACKARD LABS.
6. Benini L., Bonfigli M. E., Calori L., Farella E., Ricc#ò B. (2002). Palmtop computers for managing interaction with immersive virtual heritage. Proceedings of, EUROMEDIA 2002, 183–189.
7. Benko, H., & Wilson, A. D. (2010). Multi-point interactions with immersive omnidirectional visualizations in a dome. In Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces (pp. 19–28). New York, NY: ACM.
8. Benyon D., Smyth M., O#’Neill S., McCall R., Carroll F. (2006). The place probe: Exploring a sense of place in real and virtual environments. Presence: Teleoper. Virtual Environ., 15(6), 668–687. 10.1162/pres.15.6.668
9. Bergstrand, F., & Landgren, J. (2011). Visual reporting in time-critical work: Exploring video use in emergency response. In Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services (pp. 415–424). New York, NY: ACM.
10. Biocca F., Kim T., Levy M. (1995). The vision of virtual reality. In BioccaF.,LevyM.,(Eds.), Communication in the age of virtual reality (pp. 3–14). Hillsdale, NJ: Lawrence Erlbaum Press.
11. Bouzit M., Burdea G., Popescu G., Boian R. (2002). The Rutgers master II-new design force-feedback glove. IEEE/ASME Transactions on Mechatronics, 7(2), 256–263. 10.1109/TMECH.2002.1011262
12. Bowman D. A., Hodges L. F. (1999). Formalizing the design, evaluation, and application of interaction techniques for immersive virtual environments. Journal of Visual Languages and Computing, 10, 37–53. 10.1006/jvlc.1998.0111
13. Bowman D. A., McMahan R. P. (2007). Virtual reality: How much immersion is enough? Computer, 40(7), 36–43. 10.1109/MC.2007.257
14. Brenton, H., Gillies, M., Ballin, D., & Chatting, D. (2005). The uncanny valley: Does it exist? In Proceedings of the 19th British HCI Group Annual Conference: Workshop on Human-Animated Character Interaction.
15. Brooks F. P. (1999). What#’s real about virtual reality? IEEE Computer Graphics and Applications, 19(6), 16–27. 10.1109/38.799723
16. Brunelli, D., Farella, E., & Bonfigli, M. E. (2002). Untethered interaction for immersive virtual environments through handheld devices. In Proceedings of the EUROGRAPHICS Italian Chapter Conference (EG_IT 2002).
17. Chervest, K., Mitchell, K., & Davies, N. (2002). The role of adaptive hypermedia in a context-aware tourist GUIDE. Communications of the ACM - The Adaptive Web, 45(5), 47–51.
18. Chim J., Lau R. W. H., Leong H. V., Si A. (2003). CyberWalk: A web-based distributed virtual walkthrough environment. IEEE Transactions on Multimedia, 5(4), 503–515.
19. Csikszentmihalyi M. (1975). Beyond boredom and anxiety: Experiencing flow in work and play. San Francisco, CA: Jossey-Bass.
20. De Ara#ùjo, B. R., Casiez, G., & Jorge, J. A. (2012). Mockup builder: Direct 3D modeling on and above the surface in a continuous interaction space. In Proceedings of the 2012 Graphics Interace Conference (pp. 173–180). Toronto, Canada: Canadian Information Processing Society.
21. Diewald, S., Roalter, L., M#öller, A., & Kranz, M. (2011). Towards a holistic approach for mobile application development in intelligent environments. In Proceedings of the 10th International Conference on Mobile and Ubiquitous Multimedia (pp. 73–80). New York, NY: ACM.
22. Edward L., Lourdeaux D., Barth#ès J.-P., Lenne D., Burkhardt J.-M. (2008). Modelling autonomous virtual agent behaviours in a virtual environment for risk. The International Journal of Virtual Reality, 7(3), 13–22.
23. Elmqvist, N., Tudoreanu, M. E., & Tsigas, P. (2008). Evaluating motion constraints for 3D wayfinding in immersive and desktop virtual environments. In Proceedings of Human Factors in Computing Systems (CHI #’08) (pp. 1769–1778). New York, NY: ACM.
24. Frenz H., Lappe M., Kolesnik M., B#ührmann T. (2007). Estimation of travel distance from visual motion in virtual environments. [TAP]. ACM Transactions on Applied Perception, 4(3).
25. Garau, M., Slater, M., Vinayagamoorthy, V., Brogni, A., Steed, A., & Sasse, M. A. (2003). The impact of avatar realism and eye gaze control on perceived quality of communication in a shared immersive virtual environment. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 529–536). New York, NY: ACM.
26. Guo Y. M., Poole M. S. (2009). Antecedents of flow in online shopping: A test of alternative models. Information Systems Journal, 19(4), 369–390. 10.1111/j.1365–2575.2007.00292.x
27. Guti#érrez M., Vexo F., Thalmann D. (2004). The mobile animator: Interactive character animation in collaborative virtual environments. In Proceedings of the IEEE Virtual Reality, 2004, 125–132.
28. Hamhoum, F., & Kray, C. (2011). Scalable navigation support for crowds: personalized guidance via augmented signage. In Proceedings of the 10th International Conference on Spatial Information Theory (pp. 40–56). Berlin, Heidelberg: Springer-Verlag.
29. Heeter C. (1992). Being there: The subjective experience of presence. Presence (Cambridge, Mass.), 1(2), 262–271.
30. Hoang R. V., Sgambati M. R., Brown T. J., Coming D. S., Harris F. C. (2010). VFire: Immersive wildfire simulation and visualization. Computers & Graphics, 34(6), 655–664. 10.1016/j.cag.2010.09.014
31. HoloDesk. (2012). HoloDesk. Retrieved November 15, 2012, from http://research.microsoft.com/en-us/ projects/holodesk/
32. H#ühn, A. E., Khan, V. J., Lucero, A., & Ketelaar, P. (2012). On the use of virtual environments for the evaluation of location-based applications. In Proceedings of the 2012 ACM Annual Conference on Human Factors in Computing Systems (pp. 2569–2578). New York, NY: ACM.
33. Interrante, V., Ries, B., & Anderson, L. (2006). Distance perception in immersive virtual environments, revisited. In Proceedings of the Virtual Reality Conference, IEEE, 0, 3–10.
34. Jaeho R., Naoki H., Makoto S., Soeda M., Ryozo O. (2007). Application of human-scale immersive VR system for environmental design assessment# : A proposal for an architectural design evaluation tool (Architectural/Urban Planning and Design). Journal of Asian Architecture and Building Engineering, 6(1), 57–64. 10.3130/jaabe.6.57
35. Jennett C., Cox A. L., Cairns P., Dhoparee S., Epps A., Tijs T., Walton A. (2008). Measuring and defining the experience of immersion in games. International Journal of Human-Computer Studies, 66(9), 641–661. 10.1016/j.ijhcs.2008.04.004
36. Kalff, C., K#ühner, D., Senk, M., Dalton, R., & Hoelscher, C. (2010). Turning the shelves: Empirical findings and space syntax analyses of two virtual supermarket variations. In Proceedings of the Spatial Cognition 2010. Springer Verlag.
37. Khan V.-J., Markopoulos P., Eggen B., Metaxas G. (2010). Evaluation of a pervasive awareness system designed for busy parents. Pervasive and Mobile Computing, 6(5), 537–558. 10.1016/j.pmcj.2010.03.003
38. Kjeldskov J., Paay J. (2012). A longitudinal review of mobile HCI research methods. In Proceedings of Mobile HCI 2012. San Francisco, CA: ACM Press.
39. Kn#ödel, S., Hachet, M., & Guitton, P. (2007). Sketch-based Route Planning with Mobile Devices in immersive Virtual Environments. 2#èmes journ#ées de l#’AFRV - 2007.
40. Kopf J., Chen B., Szeliski R., Cohen M. (2010). Street slide: Browsing street level imagery. ACM Transactions on Graphics, 29.
41. Lee, J. Y., Kim, M. S., Seo, D. W., Lee, C.-W., Kim, J. S., & Lee, S. M. (2011). Dual interactions between multi-display and smartphone for collaborative design and sharing. In Proceedings of the Virtual Reality Conference (VR), 2011 IEEE (pp. 221–222). IEEE.
42. Leeb R., Keinrath C., Friedman D., Guger C., Scherer R., Neuper C., et al(2006). Walking by thinking: the brainwaves are crucial, not the muscles! Teleoperators and Virtual Environments, 15(5), 500–514. 10.1162/pres.15.5.500
43. Leichtenstern, K., Andr#é, E., & Rehm, M. (2010). Using the hybrid simulation for early user evaluations of pervasive interactions. In Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries (pp. 315–324). New York, NY: ACM.
44. Li C., Longley P. (2006). A test environment for location-based services applications. Transactions in GIS, 10(1), 43–61. 10.1111/j.1467–9671.2006.00243.x
45. Lucero, A., Boberg, M., & Uusitalo, S. (2009). Image space: capturing, sharing and contextualizing personal pictures in a simple and playful way. In Proceedings of the International Conference on Advances in Computer Enterntainment Technology (pp. 215–222). New York, NY: ACM.
46. Marchal, M., Lecuyer, A., Cirio, G., Bonnet, L., & Emily, M. (2010). Walking up and down in immersive virtual worlds: Novel interactive techniques based on visual feedback (pp. 19–26).
47. Masud, R., & Fickas, S. (2011). Virtual environments for testing location-based applications. In Proceedings of the IUI Workshop on Location Awareness for Mixed and Dual Reality (LAMDa#’11).
48. McMahan A. (2003). Immersion, engagement, and presence a method for analyzing 3-d video games. In WolfM.,PerronB.,(Eds.), The video game theory reader (pp. 67–86).
49. Miguel, M. M., Ogawa, T., Kiyokawa, K., & Takemura, H. (2007). A PDA-based see-through interface within an immersive environment. In Proceedings of the International Conference on Artificial Reality and Telexistence (pp. 113–118).
50. Mulloni, A., Seichter, H., D#ünser, A., Baudisch, P., & Schmalstieg, D. (2012). 360 degree panoramic overviews for location-based services. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 2565–2568). New York, NY: ACM.
51. Patel H., Stefani O., Sharples S., Hoffmann H., Karaseitanidis I., Amditis A. (2006). Human centred design of 3-D interaction devices to control virtual environments. Interaction with Virtual Environments, 64(3), 207–220.
52. P#érez, S. A., G#ómez, J. A. G., Alca#ñiz, M., & Fardoun, H. M. (2012). Evaluation of the sense of presence and immersion in virtual rehabilitation: Influence of user movement feedback. In Proceedings of the 13th International Conference on Interacci#ón Persona-Ordenador. New York, NY: ACM.
53. Polys N. F., Bowman D. A., North C. (2011). The role of depth and Gestalt cues in information-rich virtual environments. International Journal of Human-Computer Studies, 69(1–2), 30–51. 10.1016/j.ijhcs.2010.05.007
54. Riecke Bernhard, Cunningham Douglas, Bulthoff, & Heinrich. (2007). Spatial updating in virtual reality: The sufficiency of visual information. Psychological Research. Psychologische Forschung, 71(3), 298–313. 10.1007/s00426–006–0085-z 17024431
55. Riva G., Waterworth J., Waterworth E., Mantovani F. (2011). From intention to action: The role of presence. New Ideas in Psychology, 29(1), 24–37. 10.1016/j.newideapsych.2009.11.002
56. Rolland, J. P., & Fuchs, H. (2000). Optical versus video see-through head-mounted displays. In Medical visualization. Presence: Teleoperators and virtual environments (pp. 287–309).
57. Rothbaum B. O., Hodges L. F., Kooper R., Opdyke D., Williford J. S., North M. (1995). Virtual reality graded exposure in the treatment of acrophobia: A case report. Behavior Therapy, 26(3), 547–554. 10.1016/S0005–7894(05)80100–5
58. Ruddle R. A., Lessels S. (2009). The benefits of using a walking interface to navigate virtual environments. [TOCHI]. ACM Transactions on Computer-Human Interaction, 16(1), 1–18. 10.1145/1502800.1502805 22563232
59. S#ärkel#ä H., Takatalo J., May P., Laakso M., Nyman G. (2009). The movement patterns and the experiential components of virtual environments. International Journal of Human-Computer Studies, 67(9), 787–799. 10.1016/j.ijhcs.2009.05.003
60. Schellenbach, M., Kr#üger, A., L#övd#én, M., & Lindenberger, U. (2007). A laboratory evaluation framework for pedestrian navigation devices. In Proceedings of the 4th International Conference on Mobile Technology, Applications, and Systems and the 1st International Symposium on Computer Human Interaction in Mobile Technology (Mobility #’07) (pp. 495–502). New York, NY: ACM.
61. Schellenbach M., L#övd#én M., Verrel J., Kr#üger A., Lindenberger U. (2009). Adult age differences in familiarization to treadmill walking within virtual environments. Gait & Posture. 20031413
62. Schnall S., Hedge C., Weaver R. (2012). The immersive virtual environment of the digital fulldome: Considerations of relevant psychological processes. International Journal of Human-Computer Studies, 70(8), 561–575. 10.1016/j.ijhcs.2012.04.001
63. Schubert T. W. (2004). The sense of presence in virtual environments: A three-component scale measuring spatial presence, involvement, and realness. Zeitschrift f#ür Medienpsychologie, 15, 69–71. 10.1026//1617–6383.15.2.69
64. Second Life. (2012). Retrieved from http://secondlife.com
65. Seffah A., Benn J., Habieb Mammar H. (2008). A low-cost test environment for usability studies of head-mounted virtual reality systems. Journal of Usability Studies, 3(2), 60–73.
66. Shoemaker, G., Tang, A., & Booth, K. S. (2007). Shadow reaching: A new perspective on interaction for large displays. In Proceedings of the 20th Annual ACM Symposium on User Interface Software and Technology (pp. 53–56). New York, NY: ACM.
67. Simon, A. (2005). First-person experience and usability of co-located interaction in a projection-based virtual environment. In Proceedings of the ACM symposium on Virtual reality software and technology (VRST #’05) (pp. 23–30). New York, NY: ACM Press.
68. Singh P., Ha H. N., Kuang Z., Olivier P., Kray C., Blythe P., James P. (2006). Immersive video as a rapid prototyping and evaluation tool for mobile and ambient applications. In NieminenM.,R#öykkeeM.,(Eds.), Mobile HCI 2006 (pp. 264–264). ACM.
69. Sj#ölinder M., H#ö#ök K., Nilsson L.-G., Andersson G. (2005). Age differences and the acquisition of spatial knowledge in a three-dimensional environment: Evaluating the use of an overview map as a navigation aid. International Journal of Human-Computer Studies, 63(6), 537–564. 10.1016/j.ijhcs.2005.04.024
70. Slater M., Wilbur S. (1997). A framework for immersive virtual environments (FIVE)# : Speculations on the role of presence in virtual environments. Presence (Cambridge, Mass.), 6(6), 603–606.
71. Snowdon, C., & Kray, C. (2009). Exploring the use of landmarks for mobile navigation support in natural environments. In Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services. New York, NY: ACM.
72. Streetside. (2012). Retrieved from http://www.microsoft.com/maps/streetside.aspx
73. Sutcliffe A., Gault B., Shin J.-E. (2005). Presence, memory and interaction in virtual environments. International Journal of Human-Computer Studies, 62(3), 307–327. 10.1016/j.ijhcs.2004.11.010
74. Taylor, M. T., Chandak, A., Antani, L., & Manocha, D. (2009). RESound: Interactive sound rendering for dynamic virtual environments. In Proceedings of the 17th ACM international conference on Multimedia (pp. 271–280). New York, NY: ACM.
75. Usoh, M., Arthur, K., Whitton, M. C., Bastos, R., Steed, A., Slater, M., & Brooks, F. P. (1999). Walking > walking-in-place > flying, in virtual environments. In Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH #’99) (pp. 359–364). New York, NY: ACM Press/Addison-Wesley Publishing Co.
76. Van Schaik P., Martin S., Vallance M. (2012). Measuring flow experience in an immersive virtual environment for collaborative learning. Journal of Computer Assisted Learning, 28(4), 350–365. 10.1111/j.1365–2729.2011.00455.x
77. Van Veen H. A. H. C., Distler H. K., Braun S. J., B#ülthoff H. H. (1998). Navigating through a virtual city: Using virtual reality technology to study human action and perception. Future Generation Computer Systems, 14(3–4), 231–242. 10.1016/S0167–739X(98)00027–2
78. Vidal M., Amorim M.-A., Berthoz A. (2004). Navigating in a virtual three-dimensional maze: How do egocentric and allocentric reference frames interact? Brain Research. Cognitive Brain Research, 19(3), 244–258. 10.1016/j.cogbrainres.2003.12.006 15062862
79. View, S. (2012). Retrieved from http://maps.google.com/help/maps/streetview/
80. Waern, A., Montola, M., & Stenros, J. (2009). The three-sixty illusion: Designing for immersion in pervasive games. In Proceedings of the 27th International Conference on Human Factors in Computing Systems (CHI #’09) (pp. 1549–1558). New York, NY: ACM.
81. Wald J. (2004). Efficacy of virtual reality exposure therapy for driving phobia: A multiple baseline across-subjects design. Behavior Therapy, 35(3), 621–635. 10.1016/S0005–7894(04)80035–2
82. Wallet, G., Sauz#éon, H., Rodrigues, J., & N#’Kaoua, B. (2008). Use of virtual reality for spatial knowledge transfer: Effects of passive/active exploration mode in simple and complex routes for three different recall tasks. In Proceedings of the 2008 ACM Symposium on Virtual Reality Software and Technology (pp. 175–178). New York, NY: ACM.
83. Weihong-Guo A., Blythe P., Olivier P., Singh P., Ha H. N. (2008). Using immersive video to evaluate future traveller information systems. Intelligent Transport Systems, IET, 2(1), 38–46. 10.1049/iet-its:20070032
84. Willemsen, P., & Gooch, A. A. (2002). Perceived egocentric distances in real, image-based, and traditional virtual environments. In Proceedings of the IEEE Virtual Reality Conference 2002 (VR #’02) (pp. 275–276). Washington, DC: IEEE Computer Society.
85. Witmer B. G., Singer M. J. (1998). Measuring presence in virtual environments: A presence questionnaire. Presence (Cambridge, Mass.), 7(3), 225–240. 10.1162/105474698565686
86. Ziemer C. J., Plumert J. M., Cremer J. F., Kearney J. K. (2009). Estimating distance in real and virtual environments: Does order make a difference? Attention, Perception & Psychophysics, 71(5), 1095–1106. 10.3758/APP.71.5.1096 19525540