Formalizing agreement analysis for elicitation studies: New measures, significance test, and toolkit

Conference on Human Factors in Computing Systems - Proceedings

Volumn 2015-April, Issue , 2015, Pages 1325-1334

  Vatavu, Radu Daniel ^a   Wobbrock, Jacob O ^b  

^a University Stefancel Mare of Suceava   (Romania)

^b UNIVERSITY OF WASHINGTON   (United States)

Author keywords

Agreement rate; Coagreement; Disagreement; Guessability study; Methodology; Statistical test; User defined gestures

Indexed keywords

DISTRIBUTION FUNCTIONS; HUMAN COMPUTER INTERACTION; HUMAN ENGINEERING; PROBABILITY DISTRIBUTIONS; STATISTICAL TESTS; STATISTICS;

COAGREEMENT; DISAGREEMENT; GUESSABILITY; METHODOLOGY; USER-DEFINED GESTURES;

TESTING;

EID: 84951118264     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2702123.2702223     Document Type: Conference Paper

References (39)

1. Bailly, G., Pietrzak, T., Deber, J., and Wigdor, D. J. Métamorphe: Augmenting hotkey usage with actuated keys. In Proc. of CHI '13, ACM (2013), 563-572.
2. Bergvall-Kareborn, B., and Stahlbrost, A. Participatory design: One step back or two steps forward? In Proc. of PDC '08, Indiana University (2008), 102-111.
3. Buchanan, S., Floyd, B., Holderness, W., and LaViola, J. J. Towards user-defined multi-touch gestures for 3D objects. In Proc. of ITS '13, ACM (2013), 231-240.
4. Chong, M. K., and Gellersen, H. W. How groups of users associate wireless devices. In Proc. of CHI '13, ACM (2013), 1559-1568.
5. Cochran, W. G. The comparison of percentages in matched samples. Biometrika 37, 3/4 (1950), 256-266.
6. Cohen, J. A coefficient of agreement for nominal scales. 37-46.
7. Cohen, J. A power primer. Psychological Bulletin 112, 1 (1992), 155-159.
8. Findlater, L., Lee, B., and Wobbrock, J. Beyond QWERTY: Augmenting touch screen keyboards with multi-touch gestures for non-alphanumeric input. In Proc. of CHI '12, ACM (2012), 2679-2682.
9. James, R. C. The Mathematics Dictionary, 5th Ed. Chapman & Hall, New York, 1992.
10. Kaptein, M., and Robertson, J. Rethinking statistical analysis methods for CHI. In Proc. of CHI '12, ACM (2012), 1105-1114.
11. Kaptein, M. C., Nass, C., and Markopoulos, P. Powerful and consistent analysis of likert-type ratingscales. In Proc. of CHI '10, ACM (2010), 2391-2394.
12. Kendall, M. G., and Babington Smith, B. The problem of m rankings. Annals of Math. Stats. 10, 3 (1939), 275-287.
13. Kensing, F., and Blomberg, J. Participatory design: Issues and concerns. Computer Supported Cooperative Work 7, 3-4 (Jan. 1998), 167-185.
14. Kray, C., Nesbitt, D., Dawson, J., and Rohs, M. User-defined gestures for connecting mobile phones, public displays, and tabletops. In Proc. of MobileHCI '10, ACM (2010), 239-248.
15. Kurdyukova, E., Redlin, M., and Andŕe, E. Studying user-defined ipad gestures for interaction in multi-display environment. In Proc. of UI '12, ACM (2012), 93-96.
16. Liang, H.-N., Williams, C., Semegen, M., Stuerzlinger, W., and Irani, P. User-defined surface+motion gestures for 3D manipulation of objects at a distance through a mobile device. In Proc. of APCHI '12, ACM (2012), 299-308.
17. Martens, J.-B. Interactive statistics with illmo. ACM Trans. Interact. Intell. Syst. 4, 1 (Apr. 2014), 4:1-4:28.
18. Mauney, D., Howarth, J., Wirtanen, A., and Capra, M. Cultural similarities and differences in user-defined gestures for touchscreen user interfaces. In Proc. of CHI EA '10, ACM (2010), 4015-4020.
19. McNemar, Q. Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika 12, 2 (June 1947), 153-157.
20. Morris, M. R. Web on the wall: Insights from a multimodal interaction elicitation study. In Proc. of ITS '12, ACM (2012), 95-104.
21. Morris, M. R., Danielescu, A., Drucker, S., Fisher, D., Lee, B., schraefel, m. c., and Wobbrock, J. O. Reducing legacy bias in gesture elicitation studies. Interactions 21, 3 (May 2014), 40-45.
22. Morris, M. R., Wobbrock, J. O., and Wilson, A. D. Understanding users' preferences for surface gestures. In Proc. of GI '10, Canadian Inf. Proc.Soc. (2010), 261-268.
23. Obaid, M., Häring, M., Kistler, F., Bühling, R., and Andŕe, E. User-defined body gestures for navigational control of a humanoid robot. In Proc. of ICSR'12, Springer-Verlag (2012), 367-377.
24. Piumsomboon, T., Clark, A., Billinghurst, M., and Cockburn, A. User-defined gestures for augmented reality. In Proc. of CHI EA '13, ACM (2013), 955-960.
25. Piumsomboon, T., Clark, A., Billinghurst, M., and Cockburn, A. User-defined gestures for augmented reality. In Proc. of INTERACT '13, Springer (2013), 282-299.
26. Pyryeskin, D., Hancock, M., and Hoey, J. Comparing elicited gestures to designer-created gestures for selection above a multitouch surface. In Proc. of ITS '12, ACM (2012), 1-10.
27. Ruiz, J., Li, Y., and Lank, E. User-defined motion gestures for mobile interaction. In Proc. of CHI '11, ACM (2011), 197-206.
28. Sedgewick, R. Permutation generation methods. ACM Computing Surveys 9, 2 (June 1977), 137-164.
29. Seyed, T., Burns, C., Costa Sousa, M., Maurer, F., and Tang, A. Eliciting usable gestures for multi-display environments. In Proc. of ITS '12, ACM (2012), 41-50.
30. Sheskin, D. J. Handbook of Parametric and Nonparametric Statistical Procedures: Third Edition. CRC Press, 2004.
31. Valdes, C., Eastman, D., Grote, C., Thatte, S., Shaer, O., Mazalek, A., Ullmer, B., and Konkel, M. K. Exploring the design space of gestural interaction with active tokens through user-defined gestures. In Proc. of CHI '14, ACM (2014), 4107-4116.
32. Vatavu, R.-D. User-defined gestures for free-hand TV control. In Proc. of EuroiTV '12, ACM (2012), 45-48.
33. Vatavu, R.-D. A comparative study of user-defined handheld vs. freehand gestures for home entertainment environments. Journal of Ambient Intelligence and Smart Environments 5, 2 (2013), 187-211.
34. Vatavu, R.-D., and Zaiti, I.-A. Leap gestures for TV: Insights from an elicitation study. In Proc. of TVX '14, ACM (2014), 131-138.
35. Weigel, M., Mehta, V., and Steimle, J. More than touch: Understanding how people use skin as an input surface for mobile computing. In Proc. of CHI '14, ACM (2014), 179-188.
36. Weisstein, E. W. CRC Concise Encyclopedia of Mathematics, 2nd Ed. Chapman & Hall, New York, 2003.
37. Wobbrock, J. O., Aung, H. H., Rothrock, B., and Myers, B. A. Maximizing the guessability of symbolic input. In Proc. of CHI EA '05, ACM (2005), 1869-1872.
38. Wobbrock, J. O., Findlater, L., Gergle, D., and Higgins, J. J. The aligned rank transform for nonparametric factorial analyses using only anova procedures. In Proc. of CHI '11, ACM (2011), 143-146.
39. Wobbrock, J. O., Morris, M. R., and Wilson, A. D. User-defined gestures for surface computing. In Proc. of CHI '09, ACM (2009), 1083-1092.