Evaluation of procedures for adjusting problem-discovery rates estimated from small samples

International Journal of Human-Computer Interaction

Volumn 13, Issue 4, 2001, Pages 445-479

  Lewis, James R ^a  

^a IBM   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

FISHER INFORMATION MATRIX; INTELLIGENT SYSTEMS; MONTE CARLO METHODS;

GOOD-TURING; HIGHLY ACCURATE; NUMBER OF METHODS; SAMPLE SIZES; SMALL SAMPLE SIZE; SMALL SAMPLES; USABILITY PROBLEMS; USABILITY STUDIES;

SAMPLING;

EID: 18244386731     PISSN: 10447318     EISSN: None     Source Type: Journal    
DOI: 10.1207/S15327590IJHC1304_06     Document Type: Article

References (21)

1. Chapanis, A. (1988). Some generalizations about generalization. Human Factors, 30, 253-267.
2. Cliff, N. (1987). Analyzing multivariate data. San Diego, CA: Harcourt Brace.
3. Connell, I. W., & Hammond, N. V. (1999). Comparing usability evaluation principles with heuristics: Problem instances vs. problem types. In M. A. Sasse & C. Johnson (Eds.), Proceedings of INTERACT '99 - Human-Computer Interaction (Vol. 1, pp. 621-629). Edinburgh, Scotland: International Federation for Information Processing.
4. Draper, N. R., & Smith, H. (1966). Applied regression analysis. New York: Wiley.
5. Jelinek, F. (1997). Statistical methods for speech recognition. Cambridge, MA: MIT Press.
6. Lewis, J. R. (1982). Testing small-system customer set-up. Proceedings of the Human Factors Society 26th Annual Meeting (pp. 718-720). Santa Monica, CA: Human Factors Society.
7. Lewis, J. R. (1994). Sample sizes for usability studies: Additional considerations. Human Factors, 36, 368-378.
8. Lewis, J. R. (2000a). Evaluation of problem discovery rate adjustment procedures for sample sizes from two to ten (Tech. Rep. No. 29.3362). Raleigh, NC: IBM. Available from the author.
9. Lewis, J. R. (2000b). Overestimation of p in problem discovery usability studies: How serious is the problem? (Tech. Rep. No. 29.3358). Raleigh, NC: IBM. Available from the author.
10. Lewis, J. R. (2000c). Reducing the overestimation of p in problem discovery usability studies: Normalization, regression, and a combination normalization/Good-Turing approach (Tech. Rep. No. 29.3361). Raleigh, NC: IBM. Available from the author.
11. Lewis, J. R. (2000d). Using discounting methods to reduce overestimation of p in problem discovery usability studies (Tech. Rep. No. 29.3359). Raleigh, NC: IBM. Available from the author.
12. Lewis, J. R. (2000e). Validation of Monte Carlo estimation of problem discovery likelihood (Tech. Rep. No. 29.3357). Raleigh, NC: IBM. Available from the author.
13. Lewis, J. R., Henry, S. C, & Mack, R. L. (1990). Integrated office scenario benchmarks: A case study. In Human Computer Interaction - INTERACT '90 (pp. 337-343). Cambridge, England: Elsevier, International Federation for Information Processing.
14. Manning, C. D., & Schutze, H. (1999). Foundations of statistical natural language processing. Cambridge, MA: MIT Press.
15. Nielsen, J. (1992). Finding usability problems through heuristic evaluation. In Conference Proceedings on Human Factors in Computing Systems - CHI '92 (pp. 373-380). Monterey, CA: ACM.
16. Nielsen, J., & Landauer, T. K. (1993). A mathematical model of the finding of usability problems. In Conference Proceedings on Human Factors in Computing Systems - CHI '93 (pp. 206-213). New York: ACM.
17. Nielsen, J., & Molich, R. (1990). Heuristic evaluation of user interfaces. In Conference Proceedings on Human Factors in Computing Systems - CHI '90 (pp. 249-256). New York: ACM.
18. Norman, D. A. (1983). Design rules based on analyses of human error. Communications of the ACM, 4, 254-258.
19. Pedhazur, E. J. (1982). Multiple regression in behavioral research: Explanation and prediction. Fort Worth, TX: Harcourt Brace.
20. Virzi, R. A. (1990). Streamlining the design process: Running fewer subjects. In Proceedings of the Human Factors Society 34th Annual Meeting (pp. 291-294). Santa Monica, CA: Human Factors Society.
21. Virzi, R. A. (1992). Refining the test phase of usability evaluation: How many subjects is enough? Human Factors, 34, 43-451.