Visualization-based analysis of gameplay data - A review of literature

Entertainment Computing

Volumn 4, Issue 3, 2013, Pages 143-155

  Wallner, G ^a   Kriglstein, S ^b  

^a UNIVERSITY OF APPLIED ARTS VIENNA   (Austria)

^b VIENNA UNIVERSITY OF TECHNOLOGY   (Austria)

Author keywords

Evaluation; Gameplay analysis; Games; Visualization

Indexed keywords

FLOW VISUALIZATION; HUMAN COMPUTER INTERACTION; INTERACTIVE COMPUTER GRAPHICS; TESTING; VIDEO RECORDING; VISUALIZATION;

DATA COLLECTION; DESIGN DECISIONS; EVALUATION; GAMEPLAY ANALYSIS; GAMES; PLAYER BEHAVIOR; QUALITATIVE METHOD; VISUALIZATION TECHNIQUE;

DATA VISUALIZATION;

EID: 84875590999     PISSN: 18759521     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.entcom.2013.02.002     Document Type: Review

References (130)

1. P.N. Dixit, G.M. Youngblood, Understanding playtest data through visual data mining in interactive 3D environments, in: Proc. CGAMES 2008.
2. Drachen A., Canossa A. Analyzing spatial user behavior in computer games using geographic information systems. Proc. MindTrek 2009 2009, 182-189. ACM Press.
3. Drachen A., Canossa A. Towards gameplay analysis via gameplay metrics. Proc. MindTrek 2009 2009, 202-209. ACM Press.
4. Lennart E. Nacke, A. Drachen, K. Kuikkaniemi, J. Niesenhaus, H.J. Korhonen, W.M. van den Hoogen, K. Poels, W.A. IJsselsteijn, Y.A.W. de Kort, Playability and player experience research (panel abstracts), in: A. Barry, K. Helen, K. Tanya (Eds.), Breaking New Ground: Innovation in Games, Play, Practice and Theory: Proc. DiGRA 2009 Conference, Brunel University, 2009.
5. Andersen E., Liu Y.-E., Apter E., Boucher-Genesse F., Popović Z. Gameplay analysis through state projection. Proc. FDG 2010 2010, 1-8. ACM Press.
6. Drachen A., Canossa A., Yannakakis G.N. Player modeling using self-organization in Tomb Raider: Underworld. Proc. CIG 2009 2009, 1-8. IEEE Press.
7. Thawonmas R., Kurashige M., Chen K.-T. Detection of landmarks for clustering of online-game players. International Journal of Virtual Reality 2007, 6:11-16.
8. Medler B. Generations of game analytics, achievements and high scores, Eludamos. Journal for Computer Game Culture 2009, 3.
9. Medler B., John M., Lane J. Data Cracker: developing a visual game analytic tool for analyzing online gameplay. Proc. CHI 2011 2011, 2365-2374. ACM Press.
10. Thawonmas R., Hirano M., Kurashige M. Cellular automata and Hilditch thinning for extraction of user paths in online games. Proc. NetGames 2006 2006, ACM Press.
11. Desurvire H., Caplan M., Toth J.A. Using heuristics to evaluate the playability of games. Ext. Abstracts CHI 2004 2004, 1509-1512. ACM Press.
12. Desurvire H., Wiberg C. Game usability heuristics (play) for evaluating and designing better games: the next iteration. Proc. OCSC 2009 2009, 557-566. Springer.
13. Desurvire H., Wiberg C. User experience design for inexperienced gamers: gap game approachability principles. Human-Computer Interaction Series 2010, 131-147. Springer. R. Bernhaupt (Ed.).
14. Febretti A., Garzotto F. Usability, playability, and long-term engagement in computer games. Ext. Abstracts CHI 2009 2009, 4063-4068. ACM Press.
15. Garzotto F. Investigating the educational effectiveness of multiplayer online games for children. Proc. IDC 2007 2007, 29-36. ACM Press.
16. Isbister K., Schaffer N. Game Usability: Advancing the Player Experience 2008, Morgan Kaufmann.
17. Pinelle D., Wong N., Stach T. Heuristic evaluation for games: usability principles for video game design. Proc. CHI 2008 2008, 1453-1462. ACM Press.
18. Sweetser P., Wyeth P. Gameflow: a model for evaluating player enjoyment in games. Computer Entertainment 2005, 3:1-24.
19. Tychsen A., Canossa A. Defining personas in games using metrics. Proc. Future Play 2008 2008, 73-80. ACM Press.
20. R. Crossley, Gabe Newell on Valve, 2011. Available at: (accessed November 2012). http://www.develop-online.net/features/1192/Gabe-Newell-on-Valve.
21. Kim J.H., Gunn D.V., Schuh E., Phillips B., Pagulayan R.J., Wixon D. Tracking real-time user experience (TRUE): a comprehensive instrumentation solution for complex systems. Proc. CHI 2008 2008, 443-452. ACM Press.
22. J. Lynn, Data metrics and user experience testing, in: CHI 2012 Workshop on Game User Research, 2012.
23. A. Canossa, Y.-G. Cheong, Between intention and improvisation: limits of gameplay metrics analysis and phenomenological debugging, in: Proc. DIGRA 2011.
24. Chen K.-T., Hong L.-W. User identification based on game-play activity patterns. Proc. NetGames 2007 2007, 7-12. ACM Press.
25. P. DeRosa, Tracking player feedback to improve game design, Gamasutra (2007). Available at: (accessed July 2012). http://www.gamasutra.com/view/feature/1546/.
26. L. Scarlatos, T. Scarlatos, Visualizations for the assessment of learning in computer games, in: Proc. CEWIT 2010.
27. Milam D., El Nasr M.S. Design patterns to guide player movement in 3D games. Proc. SIGGRAPH Sandbox 2010 2010, 37-42. ACM Press.
28. Hoobler N., Humphreys G., Agrawala M. Visualizing competitive behaviors in multi-user virtual environments. Proc. VIS 2004 2004, 163-170. IEEE Computer Society.
29. Moura D., el Nasr M.S., Shaw C.D. Visualizing and understanding players' behavior in video games: discovering patterns and supporting aggregation and comparison. Proc. SIGGRAPH Sandbox 2011 2011, 11-15. ACM Press.
30. J. Dankoff, Game Telemetry with Playtest DNA on Assassin's Creed, 2012. Available at: (accessed November 2012). http://engineroom.ubi.com/game-telemetry-with-playtest-dna-on-assassins-creed-part-3/.
31. D. Nozhnin, Predicting churn: data-mining your game, Gamasutra (2012). Available at: (accessed November 2012). http://www.gamasutra.com/view/feature/170472/predicting_churn_datamining_your_.php.
32. J.L. Miller, J. Crowcroft, Avatar movement in World of Warcraft battlegrounds, in: Proc. NetGames 2009, pp. 1:1-1:6.
33. Weber B.G., John M., Mateas M., Jhala A. Using data mining to model player experience. FDG Workshop on Evaluating Player Experience in Games 2011, ACM Press.
34. Hullett K., Nagappan N., Schuh E., Hopson J. Data analytics for game development (NIER track). Proc. ICSE 2011 2011, 940-943. ACM Press.
35. Weber B.G., John M., Mateas M., Jhala A. Modeling player retention in Madden NFL 11. Proc. IAAI 2011 2011, AAAI Press.
36. B. Hillier, Interview with Randy Pitchford, 2012. Available at: (accessed November, 2012. http://www.vg247.com/2012/08/23/borderlands-2-interview-randy-pitchford-and-the-500-game/.
37. G. Zoeller, Development telemetry in video games projects, in: Game Developer Conference 2010.
38. D. Kennerly, Better game design through data miningn, Gamasutra (2003). Available at: (accessed November 2012). http://www.gamasutra.com/view/feature/2816/better_game_design_through_data_.php.
39. Mitterhofer S., Kruegel C., Kirda E., Platzer C. Server-side BOT detection in massively multiplayer online games. IEEE Security and Privacy 2009, 7:29-36.
40. Bowman B., Elmqvist N., Jankun-Kelly T. Toward visualization for games: theory, design space, and patterns. IEEE Transactions on Visualization and Computer Graphics 2012, 99.
41. Valve Corporation, Half-life 2: Episode Two Stats. Available at: (accessed June 2012). http://www.steampowered.com/status/ep2/ep2_stats.php.
42. I. Activision Publishing, Call of Duty Elite. Available at: (accessed November 2012). http://www.callofduty.com/elite.
43. J.L. Plass, M. Biles, J. Frye, T.-T. Huang, Games & The future of Learning, 2012. Available at: (accessed November 2012). http://www.nyu.edu/about/news-publications/publications/connect-information-technology/2012/04/30/video-games-and-the-future-of-learning.html.
44. S.B. Linek, G. Öttl, D. Albert, Non-invasive data tracking in educational games: combination of logfiles and natural language processing, in: Proc. INTED 2010.
45. D. Michael, S. Chen, Proof of learning: Assessment in serious games, Gamasutra (2005). Available at: (accessed November 2012). http://www.gamasutra.com/view/feature/2433/proof_of_learning_assessment_in_.php.
46. del Blanco A., Torrente J., Marchiori E.J., Marti{dotless}́nez-Ortiz I., Moreno-Ger P., Fernández-Manjón B. Easing assessment of game-based learning with and LAMS. Proc. MTDL 2010 2010, 25-30. ACM Press.
47. Chaffin A., Doran K., Hicks D., Barnes T. Experimental evaluation of teaching recursion in a video game. Proc. Sandbox 2009 2009, 79-86. ACM Press.
48. Ke F. A case study of computer gaming for math: engaged learning from gameplay?. Computer and Education 2008, 51:1609-1620.
49. A. Serrano-Laguna, J. Torrente, P. Moreno-Ger, B. Fernández-Manjón, Tracing a little for big improvements: application of learning analytics and videogames for student assessment, in: Proc. VS-Games 2012.
50. Habgood M.J., Ainsworth S.E. Motivating children to learn effectively: exploring the value of intrinsic integration in educational games. Journal of the Learning Sciences 2011, 20:169-206.
51. Liu Y.-E., Andersen E., Snider R., Cooper S., Popović Z. Feature-based projections for effective playtrace analysis. Proc. FDG 2011 2011, 69-76. ACM Press.
52. Wallner G., Kriglstein S. Design and evaluation of the educational game DOGeometry - a case study. Proc. ACE 2011 2011, 14:1-14:8. ACM Press.
53. Gilleade K.M., Dix A. Using frustration in the design of adaptive videogames. Proc. ACE 2004 2004, 228-232. ACM Press.
54. Hunicke R., Chapman V. AI for dynamic difficulty adjustment in games. Proc. AIIDE 2004 2004, AAAI Press.
55. Jennings-Teats M., Smith G., Wardrip-Fruin N. Polymorph: dynamic difficulty adjustment through level generation. Proc. PCGames 2010 2010, 11:1-11:4. ACM Press.
56. Shaker N., Yannakakis G., Togelius J. Towards automatic personalized content generation for platform games. Proc. AIIDE 2010 2010, AAAI Press.
57. J. Togelius, R.D. Nardi, S.M. Lucas, Towards automatic personalised content creation in racing games, in: Proc. CIG 2007, pp. 252-259.
58. S. Miller, Auto-dynamic Difficulty, 2004. Available at: (accessed November 2012). http://dukenukem.typepad.com/game_matters/2004/01/autoadjusting_g.html.
59. D. Kazemi, Metrics and Dynamic difficulty in Ritual's Sin Episodes (Part 1), 2008. Available at: (accessed November 2012). http://orbusgameworks.com/blog/article/70/metrics-anddynamic-difficulty-in-rituals-sin-episodes-part-1.
60. M. Booth, The AI systems of Left 4 Dead, in: Artificial Intelligence and Interactive Digital Entertainment Conference, Stanford, USA.
61. Tastan B., Sukthankar G.R. Learning policies for first person shooter games using inverse reinforcement learning. Proc. AIIDE 2011 2011, 85-90. AAAI Press.
62. Bauckhage C., Gorman B., Thurau C., Humphrys M. Learning human behavior from analyzing activities in virtual environments. MMI Interaktiv-Human 2007, 1:3-17.
63. Thurau C., Bauckhage C., Sagerer G. Learning human-like movement behavior for computer games. Proc. SAB 2004 2004, 315-323. MIT Press.
64. Reeder J., Sukthankar G., Georgiopoulos M., Anagnostopoulos G. Intelligent trading agents for massively multi-player game economies. Proc. AIIDE 2008 2008, 102-107. AAAI Press.
65. Ha E., Rowe J.P., Mott B.W., Lester J.C. Goal recognition with markov logic networks for player-adaptive games. Proc. AIIDE 2011 2011, TAAAI Press.
66. Mochibot. Available at: (accessed June 2012). http://https://www.mochibot.com.
67. Nonoba. Available at: (accessed June 2012). http://nonoba.com/developers/statistics.
68. K. Börner, G.J. Lee, S. Penumarthy, R.J. Jones, Visualizing the VLearn3D 2002 conference in space and time, in: Visualization and Data Analysis, vol. 5295, SPIE-IS& T, 2004, pp. 24-32.
69. Chen C., Börner K. From spatial proximity to semantic coherence: a quantitative approach to the study of group dynamics in collaborative virtual environments, Presence: Teleoper. Virtual Environment 2005, 14:81-103.
70. Stellmach S., Nacke L., Dachselt R. 3D attentional maps: aggregated gaze visualizations in three-dimensional virtual environments. Proc. AVI 2010 2010, 345-348. ACM Press.
71. Zanbaka C.A., Lok B.C., Babu S.V., Ulinski A.C., Hodges L.F. Comparison of path visualizations and cognitive measures relative to travel technique in a virtual environment. IEEE Transactions on Visualization and Computer Graphics 2005, 11:694-705.
72. Haworth R., Bostani S.S.T., Sedig K. Visualizing decision trees in games to support children's analytic reasoning: any negative effects on gameplay?. International Journal of Computer Games Technology 2010, 1-11.
73. C. Macklin, J. Wargaski, M. Edwards, K.Y. Li, Dataplay: mapping game mechanics to traditional data visualization, in: A. Barry, K. Helen, K. Tanya (Eds.), Breaking New Ground: Innovation in Games, Play, Practice and Theory: Proc. DiGRA 2009 Conference, Brunel University, 2009.
74. V. Zammitto, Visualization techniques in video games, in: Proc. EVA 2008, pp. 267-276.
75. Medler B., Magerko B. Analytics of play: using information visualization and gameplay practices for visualizing video game data. Parsons Journal for Information Mapping 2011, 3.
76. C. Pruett, Hot failure: tuning gameplay with simple player metrics, Gamasutra (2010). Available at: (accessed November 2012). http://www.gamasutra.com/view/feature/6155/hot_failure_tuning_gameplay_with_.php.
77. D. Schoenblum, Zero to millions: building an XLSP for Gears of War 2, in: Game Developer Conference, 2010.
78. J. Ludwig, Flogging: data collection on the high seas, in: Game Developer Conference, 2007.
79. Dixit P.N., Youngblood G.M. Understanding information observation in interactive 3D environments. Proc. SIGGRAPH Sandbox 2008 2008, 163-170. ACM Press.
80. Playtomic. Available at: (accessed June 2012). http://playtomic.com/.
81. Medler B. Player dossiers: analyzing gameplay data as a reward. The International Journal of Computer Game Research 2011, 11.
82. GiantBomb. Available at: (accessed June 2012). http://www.giantbomb.com.
83. PsychoStats, Heatmaps. Available at: (accessed July 2012). http://www.psychostats.com/doc/Heatmaps.
84. A. Belicza, Sc2gears. Available at: (accessed June 2012). https://sites.google.com/site/sc2gears/.
85. Cheong Y.-G., Jhala A., Bae B.-C., Young R.M. Automatically generating summary visualizations from game logs. Proc. AIIDE 2008 2008, The AAAI Press.
86. N. Halper, M. Masuch, Action summary for computer games: Extracting action for spectator modes and summaries, in: Proc. ADCOG 2003, City University of Hong Kong, 2003, pp. 124-132.
87. R. Games, Rockstar Games Social Club. Available at: (accessed June 2012). http://socialclub.rockstargames.com.
88. Thawonmas R., Iizuka K. Visualization of online-game players based on their action behaviors. International Journal of Computer Games Technology 2008, 2008:1-9.
89. Iosup A. Cameo: continuous analytics for massively multiplayer online games on cloud resources. Proc. Euro-Par 2009 2010, 289-299. Springer.
90. S. Finnegan, R. Holte, G. Xiao, M. Trommelen, Machine learning for semi-automated gameplay analysis, in: Game Developers Conference 2005.
91. Penumarthy S., Börner K. Analysis and visualization of social diffusion patterns in three-dimensional virtual worlds. Avatars at Work and Play: Collaboration and Interaction in Shared Virtual Environments (Computer Supported Cooperative Work) 2006, 39-61. Springer. R. Schroeder, A.-S. Axelsson (Eds.).
92. Joslin S., Brown R., Drennan P. The gameplay visualization manifesto: a framework for logging and visualization of online gameplay data. Computer Entertainment 2007, 5.
93. H. Marselas, Profiling, data analysis, scalability, and magic numbers: meeting the minimum requirements for Age of Empires II: The Age of Kings, Gamasutra (2000). Available at: (accessed July 2012). http://www.gamasutra.com/view/feature/3137/profiling_data_analysis_.php.
94. Williams D., Yee N., Caplan S.E. Who plays, how much, and why? Debunking the stereotypical gamer profile. Journal of Computer-Mediated Communication 2008, 13:993-1018.
95. J. Timmer, Science gleans 60TB of behavior data from Everquest 2 logs, 2009. Available at: (accessed July 2012). http://arstechnica.com/gaming/2009/02/aaas-60tb-of-behavioral-data-the-everquest-2-server-logs/.
96. P. Mirza-Babaei, G. McAllister, Biometric storyboards to improve understanding of the players gameplay experience, in: Proc. Videogame Cultures and the Future of Interactive Entertainment, 2011.
97. Mirza-Babaei P., Nacke L., Fitzpatrick G., White G., McAllister G., Collins N. Biometric storyboards: visualising game user research data. Proc. CHI 2012 Extended Abstracts 2012, 2315-2320. ACM Press.
98. Ashton M., Verbrugge C. Measuring cooperative gameplay pacing in World of Warcraft. Proc. FDG 2011 2011, 77-83. ACM Press.
99. Coulton P., Bamford W., Cheverst K., Rashid O. 3D space-time visualization of player behavior in pervasive location-based games. International Journal of Computer Games Technology 2008, 1-5.
100. Lewis C., Wardrip-Fruin N. Mining game statistics from web services: a World of Warcraft armory case study. Proc. FDG 2010 2010, 100-107. ACM Press.
101. T. Zimmermann, B. Phillips, N. Nagappan, C. Harrison, Data-driven games user research, in: CHI 2012 Workshop on Game User Research, 2012.
102. C. Thompson, Halo 3: how Microsoft Labs invented a new science of play, Wired (2007). Available at: (accessed July 2012). http://www.wired.com/gaming/virtualworlds/magazine/15-09/ff_halo.
103. Gagné A.R., El-Nasr M.S., Shaw C.D. A deeper look at the use of telemetry for analysis of player behavior in RTS games. Proc. ICEC 2011 2011, 247-257. Springer.
104. Miller J.L., Crowcroft J. Group movement in World of Warcraft battlegrounds. Int. J. Adv. Media Commun. 2010, 4:387-404.
105. Canossa A., Drachen A., Sørensen J.R.M. Arrrgghh!!! - blending quantitative and qualitative methods to detect player frustration. Proc. FDG 2011 2011, 61-68. ACM Press.
106. Thawonmas R., Kurashige M., Iizuka K., Kantardzic M.M. Clustering of online game users based on their trails using self-organizing map. LNCS 2006, 366-369. Springer.
107. Wallner G., Kriglstein S. A spatiotemporal visualization approach for the analysis of gameplay data. Proc. CHI 2012 2012, 1115-1124. ACM Press.
108. Inselberg A. Parallel Coordinates: Visual Multidimensional Geometry and Its Applications 2009, Springer.
109. Andrienko G., Andrienko N. A general framework for using aggregation in visual exploration of movement data. The Cartographic Journal 2010, 22-40.
110. Hilditch C. Linear skeletons from {square cup}boards. Machine Intelligence 1969, 4:403-420.
111. Kohonen T. Self-organizing Maps 2001, Springer.
112. MacQueen J.B. Some methods for classification and analysis of multivariate observations. Proc. of the Fifth Berkeley Symposium on Mathematical Statistics and Probability 1967, vol. 1:281-297. University of California Press.
113. Kruskal J.B., Wish M. Multidimensional Scaling 1978, Sage Publications.
114. Tollis I.G., Di Battista G., Eades P., Tamassia R. Graph Drawing: Algorithms for the Visualization of Graphs 1998, Prentice Hall.
115. Ohsawa Y., Benson N.E., Yachida M. Keygraph: automatic indexing by co-occurrence graph based on building construction metaphor. Proc. ADL 1998 1998, 12-18. IEEE Computer Society.
116. Lameman B.A., El-Nasr M.S., Drachen A., Foster W., Moura D., Aghabeigi B. User studies - a strategy towards a successful industry - academic relationship. Proc. Futureplay 2010 2010, 134-142. ACM Press.
117. Moskowitz D., Shodhan S., Twardos M. Spore API: accessing a unique database of player creativity. SIGGRAPH 2009 2009, ACM Press, p. 62:1.
118. Electronic Arts Inc., Spore API. Available at: (accessed July 2012). http://www.spore.com/comm/developer/.
119. D. Herbst, BF3 Stats API. Available at: (accessed July 2012). http://www.wowwiki.com/World_of_Warcraft_API.
120. Blizzard Entertainment, World of Warcraft API. Available at: (accessed July 2012). http://bf3stats.com/api.
121. Electronic Arts, Privacy Policy. Available at: (accessed November 2012). http://www.ea.com/privacy-policy.
122. Rockstar Games, Rockstar Games Online Privacy Statement. Available at: (accessed November 2012). http://www.rockstargames.com/privacy.
123. Böhme R., Köpsell S. Trained to accept? A field experiment on consent dialogs. Proc. CHI 2010 2010, 2403-2406. ACM Press.
124. Chittaro L., Ranon R., Ieronutti L. VU-Flow: a visualization tool for analyzing navigation in virtual environments. IEEE Transactions on Visualization and Computer Graphics 2006, 12:1475-1485.
125. Nacke L., Lindley C., Stellmach S. Log who's playing: psychophysiological game analysis made easy through event logging. LNCS 2008, 150-157. Springer. P. Markopoulos, B. de Ruyter, W. IJsselsteijn, D. Rowland (Eds.).
126. Chittaro L., Ieronutti L. A visual tool for tracing users' behavior in virtual environments. Proc. AVI 2004 2004, 40-47. ACM Press.
127. Pagulayan R.J., Keeker K., Wixon D., Romero R.L., Fuller T. The Human-Computer Interaction Handbook 2003, L. Erlbaum Associates Inc., pp. 883-906.
128. Mandryk R.L., Atkins M.S., Inkpen K.M. A continuous and objective evaluation of emotional experience with interactive play environments. Proc. CHI 2006 2006, 1027-1036. ACM Press.
129. Nacke L., Drachen A., Goebel S. Methods for evaluating gameplay experience in a serious gaming context. International Journal of Computer Science in Sport 2010, 9.
130. Keim D.A. Information visualization and visual data mining. IEEE Transactions on Visualization and Computer Graphics 2002, 8:1-8.