Is once enough? on the extent and content of replications in human-computer interaction

Conference on Human Factors in Computing Systems - Proceedings

Volumn , Issue , 2014, Pages 3523-3532

  Hornbæk, Kasper ^a   Sander, Søren S ^a   Bargas Avila, Javier ^b   Simonsen, Jakob Grue ^a  

^a UNIVERSITY OF COPENHAGEN   (Denmark)

^b GOOGLE   (Switzerland)

Author keywords

Replications

Indexed keywords

HUMAN ENGINEERING;

EFFECT SIZE; REPLICATIONS;

HUMAN COMPUTER INTERACTION;

EID: 84900453198     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2556288.2557004     Document Type: Conference Paper

References (53)

1. Appert, C., Chapuis, O., and Pietriga, E. High-precision magnification lenses. In Proc. CHI (2010), 273-282
2. Begley, C. G. and Ellis, L. M. Drug development: Raise standards for preclinical cancer research. Nature 483, 7391 (2012), 531-533
3. Casiez, G. and Vogel, D. The effect of spring stiffness and control gain with an elastic rate control pointing device. In Proc. CHI, (2008), 1709-1718
4. Chapuis, O., Labrune, J.-B., and Pietriga, E. DynaSpot: Speed-dependent area cursor. In Proc. CHI (2009), 1391-1400
5. Chen, N. Y., Guimbretière, F., and Löckenhoff, C. E. Relative role of merging and two-handed operation on command selection speed. International Journal of Human-Computer Studies 66, 10 (2008), 729-740
6. D'Mello, S. K., Graesser, A., and King, B. Toward Spoken Human-Computer Tutorial Dialogues. Human-Computer Interaction 25, 4 (2010), 289-323
7. Egan, D. E., Remde, J. R., Gomez, L. M., Landauer, T. K., Eberhardt, J., and Lochbaum, C. C. Formative design evaluation of superbook. ACM Transactions on Information Systems (TOIS) 7, 1 (1989), 30-57
8. Evanschitzky, H., Baumgarth, C., Hubbard, R., and Armstrong, J. S. Replication research's disturbing trend. Journal of Business Research 60, 4 (2007), 411-415
9. Findlater, L. and McGrenere, J. Impact of screen size on performance, awareness, and user satisfaction with adaptive graphical user interfaces. In Proc. CHI (2008), 1247-1256
10. Forlines, C. and Balakrishnan, R. Evaluating tactile feedback and direct vs. indirect stylus input in pointing and crossing selection tasks. In Proc. CHI, (2008), 1563-1572
11. Greenberg, S. and Buxton, B. Usability evaluation considered harmful (some of the time). In Proc. CHI (2008), 111-120
12. Grimes, D., Tan, D. S., Hudson, S. E., Shenoy, P., and Rao, R. P. Feasibility and pragmatics of classifying working memory load with an electroencephalograph. In Proc. CHI, (2008), 835-844
13. Hartmann, J., De Angeli, A., and Sutcliffe, A. Framing the user experience: Information biases on website quality judgement. In Proc. CHI (2008), 855-864
14. Heer, J. and Bostock, M. Crowdsourcing graphical perception: Using mechanical turk to assess visualization design. In Proc. CHI (2010), 203-212
15. Hendrick, C. Replications, strict replications, and conceptual replications: Are they important? Journal of Social Behavior & Personality, (1990)
16. Hoggan, E., Brewster, S. A., and Johnston, J. Investigating the effectiveness of tactile feedback for mobile touchscreens. In Proc. CHI (2008), 1573-1582
17. Hornbæk, K. Some Whys and Hows of Experiments in Human-Computer Interaction. Foundations and Trends in Human-Computer Interaction 5, 4 (2013), 299-373
18. Huizenga, J. R. Cold fusion: The scientific fiasco of the century (1992)
19. Hunter, J. E. and Schmidt, F. L. Methods of metaanalysis. Sage, 2004
20. Ioannidis, J. P. Why most published research findings are false. PLoS medicine 2, 8 (2005), e124
21. Jasny, B. R., Chin, G., Chong, L., and Vignieri, S. Again, and again, and again. Science 334, 6060 (2011), 1225-1225
22. John, B. Avoiding "It's JUST a Replication", RepliCHI 2013 Workshop
23. Jones, K. S., Derby, P. L., and Schmidlin, E. A. An investigation of the prevalence of replication research in human factors. Human Factors, 52, 5 (2010), 586-595
24. Kaplan, A. The Conduct of Inquiry: Methodology for behavioral science. Chandler, San Francisco, CA, 1964
25. Karlson, A. K. and Bederson, B. B. One-handed touchscreen input for legacy applications. In Proc. CHI (2008), 1399-1408
26. Kelly, C. W., Chase, L. J., and Tucker, R. K. Replication in experimental communication research: An analysis. Human Communication Research 5, 4 (1979), 338-342
27. Lallemand, C., Koenig, V., and Gronier, G. Replicating an International Survey on User Experience: Challenges, Successes and Limitations. RepliCHI 2013 Workshop
28. Landis, J. R. and Koch, G. G. The measurement of observer agreement for categorical data. Biometrics, (1977), 159-174
29. Lee, E.-J. Flattery may get computers somewhere, sometimes: The moderating role of output modality, computer gender, and user gender. International Journal of Human-Computer Studies 66, 11 (2008), 789-800
30. Lee, E.-J. I like you, but I won't listen to you: Effects of rationality on affective and behavioral responses to computers that flatter. International Journal of Human-Computer Studies 67, 8 (2009), 628-638
31. Makel, M. C., Plucker, J. A., and Hegarty, B. Replications in Psychology Research How Often Do They Really Occur? Perspectives on Psychological Science 7, 6 (2012), 537-542
32. Marshall, C. and Rossman, G. B. Designing qualitative research, Sage (1999)
33. Martin, B. and Isokoski, P. EdgeWrite with integrated corner sequence help. In Proc. CHI (2008), 583-592
34. McGrath, J. E. Methodology matters: Doing research in the behavioral and social sciences, in Human-computer interaction (1995), Morgan Kaufmann, 152-169
35. Moffatt, K. and McGrenere, J. Steadied-bubbles: Combining techniques to address pen-based pointing errors for younger and older adults. In Proc. CHI (2010), 1125-1134
36. Morris, D., Ringel Morris, M., and Venolia, G. SearchBar: A search-centric web history for task resumption and information re-finding. In Proc. CHI (2008), 1207-1216
37. Moscovich, T. and Hughes, J. F. Indirect mappings of multi-touch input using one and two hands. In Proc. CHI (2008), 1275-1284
38. Nacenta, M. A., Mandryk, R. L., and Gutwin, C. Targeting across displayless space. In Proc. CHI (2008), 777-786
39. Neuliep, J. W. and Crandall, R. Everyone was wrong: There are lots of replications out there. Journal of Social Behavior & Personality, (1993)
40. Newman, W. A preliminary analysis of the products of HCI research, using pro forma abstracts. In Proc. CHI (1994), 278-284
41. Olwal, A., Feiner, S., and Heyman, S. Rubbing and tapping for precise and rapid selection on touch-screen displays. In Proc. CHI (2008), 295-304
42. Pietriga, E. and Appert, C. Sigma lenses: Focus-context transitions combining space, time and translucence. In Proc. CHI (2008), 1343-1352
43. Reaves, M. L., Sinha, S., Rabinowitz, J. D., Kruglyak, L., and Redfield, R. J. Absence of detectable arsenate in DNA from arsenate-grown GFAJ-1 cells. Science 337, 6093 (2012), 470-473
44. Rosenthal, R. and Rosnow, R. Essentials of behavioural research. McGraw-Hill (1991)
45. Rosenthal, R. Replication in behavioral research. Journal of Social Behavior & Personality, 5, 1990, 1-30
46. Sterling, T. D. Publication decisions and their possible effects on inferences drawn from tests of significance- or vice versa. Journal of the American statistical association 54, 285 (1959), 30-34
47. Takayama, L. and Nass, C. Driver safety and information from afar: An experimental driving simulator study of wireless vs. in-car information services. International Journal of Human-Computer Studies 66, 3 (2008), 173-184
48. Tsang, E. W. and Kwan, K.-M. Replication and theory development in organizational science. Academy of Management Review 24, 4 (1999), 759-780
49. Tuddenham, P., Kirk, D., and Izadi, S. Graspables revisited. In Proc. CHI (2010), 2223-2232
50. Vogel, D. and Baudisch, P. Shift: A technique for operating pen-based interfaces using touch. In Proc. CHI (2007), 657-666
51. Wilson, M. L., Mackay, W., Chi, E., Bernstein, M., Russell, D., and Thimbleby, H. RepliCHI-CHI should be replicating and validating results more. In Extended Abstracts of CHI (2011), 463-466
52. Yatani, K., Partridge, K., Bern, M., and Newman, M. W. Escape: A target selection technique using visually-cued gestures. In Proc. CHI (2008), 285-294
53. Zhai, S., Conversy, S., Beaudouin-Lafon, M., and Guiard, Y. Human on-line response to target expansion. In Proc. CHI (2003), 177-184.