Sensory difference tests as measurement instruments: A review of recent advances

Journal of Sensory Studies

Volumn 29, Issue 2, 2014, Pages 89-102

  Ennis, John M ^a   Rousseau, Benoît ^b   Ennis, Daniel M ^a  

^a Institute for Perception   (United States)

^b The Institute for Perception   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

STANDARDS;

BUSINESS RISKS; CLASSICAL TECHNIQUES; MEASUREMENT INSTRUMENTS; RECENT RESEARCHES; SENSORY TECHNIQUES;

ACCEPTANCE TESTS;

EID: 84897391366     PISSN: 08878250     EISSN: 1745459X     Source Type: Journal    
DOI: 10.1111/joss.12086     Document Type: Review

References (136)

1. Agresti, A. and Coull, B.A. 1998. Approximate is better than "exact" for interval estimation of binomial proportions. Am. Stat. 52, 119-126.
2. Alfaro-Rodriguez, H., Angulo, O. and O'Mahony, M. 2007. Be your own placebo: A double paired preference test approach for establishing expected frequencies. Food Qual. Prefer. 18, 353-361.
3. Amerine, M., Pangborn, R. and Roessler, E. 1965. Principles of Sensory Evaluation of Food, Academic Press, New York, NY.
4. Angulo, O., Lee, H.-S. and O'Mahony, M. 2007. Sensory difference tests: Overdispersion and warm-up. Food Qual. Prefer. 18, 190-195.
5. Bayarri, S., Carbonell, I., Izquierdo, L. and Tárrega, A. 2008. Replicated triangle and duo-trio tests: Discrimination capacity of assessors evaluated by Bayes' rule. Food Qual. Prefer. 19, 519-523.
6. Bi, J. 2002. Variance of d′ for the same-different method. Behav. Res. Methods 34, 37-45.
7. Bi, J. 2003. Difficulties and a way out: A Bayesian approach for sensory difference and preference tests. J. Sensory Studies 18, 1-18.
8. Bi, J. 2006. Sensory Discrimination Tests and Measurements: Statistical Principles, Procedures and Tables, Blackwell Publishing, Ames, Iowa.
9. Bi, J. 2010. Comments on D.M. Ennis' presentation on equivalence testing. Food Qual. Prefer. 21, 259-260.
10. Bi, J. 2011a. Similarity tests using forced-choice methods in terms of Thurstonian discriminal distance, d′. J. Sensory Studies 26, 151-157.
11. Bi, J. 2011b. Bayesian approach to sensory preference, difference and equivalence tests. J. Sensory Studies 26, 383-399.
12. Bi, J. and Ennis, D. 1999. Beta-Binomial tables for replicated difference and preference tests. J. Sensory Studies 14, 347-368.
13. Bi, J. and Ennis, D. 2001. Exact beta-binomial tables for small experiments. J. Sensory Studies 16, 319-325.
14. Bi, J. and Kuesten, C. 2012. Intraclass correlation coefficient (ICC): A framework for monitoring and assessing performance of trained sensory panels and panelists. J. Sensory Studies 27, 352-364.
15. Bi, J. and O'Mahony, M. 2013. Variance of d′ for the Tetrad Test and comparisons with other forced-choice methods. J. Sensory Studies 28, 91-101.
16. Bi, J., Ennis, D. and O'Mahony, M. 1997. How to estimate and use the variance of d′ from difference tests. J. Sensory Studies 12, 87-104.
17. Bi, J., Lee, H. and O'Mahony, M. 2010. D′ and variance of d′ for four-alternative forced choice (4-AFC). J. Sensory Studies 25, 740-750.
18. Bockenhölt, U. 1992. Thurstonian models for partial ranking data. Br. J. Math. Stat. Psychol. 43, 31-49.
19. Bradley, R. 1963. Some relationships among sensory difference tests. Biometrics 19, 385-397.
20. Braun, V., Rogeaux, M., Schneid, N., O'Mahony, M. and Rousseau, B. 2004. Corroborating the 2-AFC and 2-AC Thurstonian models using both a model system and sparkling water. Food Qual. Prefer. 15, 501-507.
21. Brockhoff, P. and Christensen, R. 2010. Thurstonian models for sensory discrimination tests as generalized linear models. Food Qual. Prefer. 21, 330-338.
22. Brockhoff, P. and Schlich, P. 1998. Handling replications in discrimination tests. Food Qual. Prefer. 9, 303-312.
23. Brockhoff, P.B. 2003. The statistical power of replications in difference tests. Food Qual. Prefer. 14, 405-417.
24. Brown, J. 1974. Recognition assessed by rating and ranking. Br. J. Psychol. 65, 13-22.
25. Brown, L.D., Cai, T.T. and Dasgupta, A. 2002. Confidence intervals for a binomial proportion and asymptotic expansions. Ann. Stat. 30, 160-201.
26. Byer, A. and Abrams, D. 1953. A comparison of the triangular and two-sample taste test methods. Food Technol. 7, 185-187.
27. Carbonell, L., Sendra, J., Bayarri, S., Izquierdo, L. and Tárrega, A. 2008. Thurstonian scales obtained by transformation of beta distributions. Food Qual. Prefer. 19, 407-411.
28. Castura, J. 2010. Equivalence testing: A brief review. Food Qual. Prefer. 21, 257-258.
29. Chae, J.E., Lee, Y.-M. and Lee, H.-S. 2010. Affective same-different discrimination tests for assessing consumer discriminability between milks with subtle differences. Food Qual. Prefer. 21, 427-438.
30. Chew, V. 1977. Statistical Hypothesis Testing: An Academic Exercise in Futility. Proceedings of Florida State Horticultural Society, Vol. 90, p. 214, Lake Alfred, FL.
31. Christensen, R., Lee, H., Brockhoff, P., Haubo, R., Christensen, B. and Bruun, P. 2012a. Estimation of the Thurstonian model for the 2-AC protocol. Food Qual. Prefer. 24, 119-128.
32. Christensen, R., Lee, H., Brockhoff, P., Haubo, R., Christensen, B. and Bruun, P. 2012b. Estimation of the Thurstonian model for the 2-AC protocol. Food Qual. Prefer. 24, 119-128.
33. Christensen, R.H.B. 2011. Statistical methodology for sensory discrimination tests and its implementation in sensR. http://cran.r-project.org (accessed December 12, 2012).
34. Christensen, R.H.B. and Brockhoff, P.B. 2009. Estimation and inference in the same-different test. Food Qual. Prefer. 20, 514-524.
35. Christensen, R.H.B., Cleaver, G. and Brockhoff, P.B. 2011. Statistical and Thurstonian models for the A-not A protocol with and without sureness. Food Qual. Prefer. 22, 542-549.
36. David, H. and Trivedi, M. 1962. Pair, Triangle and Duo-Trio Tests (Technical Report 55), Virginia Polytechnic Institute, Department of Statistics, Blacksburg, VA.
37. Delwiche, J. and O'Mahony, M. 1996. Flavour discrimination - An extension of Thurstonian paradoxes to the tetrad method. Food Qual. Prefer. 7, 1-5.
38. Dessirier, J. and O'Mahony, M. 1998. Comparison of d′ values for the 2-AFC (paired comparison) and 3-AFC discrimination methods: Thurstonian models, sequential sensitivity analysis and power. Food Qual. Prefer. 10, 1-8.
39. Dorfman, D. and Alf, E. 1969. Maximum likelihood estimation of parameters of signal detection theory and determination of confidence intervals - Rating method data. J. Math. Psychol. 6, 487-496.
40. Duineveld, K. and Meyners, M. 2008. Hierarchical Bayesian analysis of true discrimination rates in replicated triangle tests. Food Qual. Prefer. 19, 292-305.
41. Elliot, P. 1964. Tables of d′. In Signal Detection and Recognition by Human Observers: Contemporary Readings (J.A. Swets, ed.) pp. 651-684, John Wiley & Sons, New York, NY.
42. Ennis, D. 1990. Relative power of difference testing methods in sensory evaluation. Food Technol. 44, 114, 116-117.
43. Ennis, D. 1993. The power of sensory discrimination methods. J. Sensory Studies 8, 353-370.
44. Ennis, D. 2006. Sources and influence of perceptual variance: Comment on Dzhafarov's regular minimality principle. J. Math. Psychol. 50, 66-73.
45. Ennis, D. and Bi, J. 1998. The beta-binomial model: accounting for inter-trial variation in replicated difference and preference tests. J. Sensory Studies 13, 389-412.
46. Ennis, D. and Ennis, J. 2010. Equivalence hypothesis testing. Food Qual. Prefer. 21, 253-256.
47. Ennis, D. and Ennis, J. 2013. A Thurstonian ranking model with rank-induced dependencies. J. Classif. 30, 124-147.
48. Ennis, D. and Mullen, K. 1986a. Theoretical aspects of sensory discrimination. Chem. Senses 11, 513-522.
49. Ennis, D. and Mullen, K. 1986b. A multivariate model for discrimination methods. J. Math. Psychol. 30, 206-219.
50. Ennis, D. and Mullen, K. 1992. Probabilistic psychophysics with noisy stimuli. Math. Soc. Sci. 23, 221-234.
51. Ennis, D., Mullen, K. and Frijters, J. 1988. Variants of the method of triads: Unidimensional Thurstonian models. Br. J. Math. Stat. Psychol. 41, 25-26.
52. Ennis, D.M. and Ennis, J.M. 2012. Accounting for no difference/preference responses or ties in choice experiments. Food Qual. Prefer. 23, 13-17.
53. Ennis, D.M., Rousseau, B. and Anderson, J.L. 2003. IFPrograms user manual. http://www.ifpress.com (accessed December 12, 2012).
54. Ennis, J. 2012. Guiding the switch from Triangle testing to Tetrad testing. J. Sensory Studies 27, 223-231.
55. Ennis, J. and Jesionka, V. 2011. The power of sensory discrimination testing methods revisited. J. Sensory Studies 26, 371-382.
56. Ennis, J., Ennis, D., Yip, D. and O'Mahony, M. 1998. Thurstonian models for variants of the method of tetrads. Br. J. Math. Stat. Psychol. 51, 205-215.
57. Ennis, J.M. 2013. A Thurstonian analysis of the two-out-of-five test. J. Sensory Studies 28, 297-310.
58. Ennis, J.M. and Christensen, R. 2014. Precision of measurement in Tetrad testing. Food Qual. Prefer. 32(A), 98-106.
59. Fechner, G. 1860. Elemente der Psychophysik (Breitkopf and Härtel, Leipzig, 1860), H. Adler, Tran., Erster Theil, 65. Holt, Rinehart and Winston, New York, NY.
60. Frijters, J. 1979. The paradox of discriminatory nondiscriminators resolved. Chem. Senses 4, 355-358.
61. Frijters, J. 1980. Three-stimulus procedures in olfactory psychophysics: An experimental comparison of Thurstone-Ura and three-alternative-forced-choice models of signal detection theory. Percept. Psychophys. 28, 390-397.
62. Frijters, J., Blauw, Y. and Vermaat, S. 1982. Incidental training in the triangular method. Chem. Senses 7, 63-69.
63. Garcia, K., Ennis, J. and Prinyawiwatkul, W. 2012. A large-scale experimental comparison of the Tetrad and Triangle tests in children. J. Sensory Studies 27, 217-222.
64. Garcia, K., Ennis, J.M. and Prinyawiwatkul, W. 2013. Reconsidering the specified Tetrad test. J. Sensory Studies 28, 445-449.
65. Gourevitch, V. and Galanter, E. 1967. A significant test for one parameter isosensitivity functions. Psychometrika 32, 25-33.
66. Green, D. and Swets, J. 1966. Signal Detection Theory and Psychophysics, Wiley, New York, NY.
67. Green, D.M. 1964. General prediction relating Yes-No and forced-choice results. J. Acoust. Soc. Am. 36, 1042.
68. Gridgeman, N. 1970. A re-examination of the two-stage triangle test for the perception of sensory differences. J. Food Sci. 35, 87-91.
69. Hautus, M. and Irwin, R. 1995. Two models for estimating the discriminability of foods and beverages. J. Sens. Stud. 10, 203-215.
70. Hautus, M., Shepherd, D. and Peng, M. 2011. Decision strategies for the two-alternative forced choice reminder paradigm. Atten. Percept. Psychophys. 73, 729-737.
71. Hautus, M.J., Mahony, M.O. and Lee, H. 2007. Decision strategies determined from the shape of the same-different ROC curve: What are the effects of incorrect assumptions? J. Sensory Studies 23, 743-764.
72. Irwin, R., Hautus, M. and Butcher, J. 1999. An area theorem for the same-different experiment. Percept. Psychophys. 61, 766-769.
73. Ishii, R., Kawaguchi, H., O'Mahony, M. and Rousseau, B. 2007. Relating consumer and trained panels' discriminative sensitivities using vanilla flavored ice cream as a medium. Food Qual. Prefer. 18, 89-96.
74. Ishii, R., O'Mahony, M. and Rousseau, B. 2014. Triangle and tetrad protocols: Small sensory differences, resampling and consumer relevance. Food Qual. Prefer. 31, 49-55.
75. Jesionka, V., Rousseau, B. and Ennis, J.M. 2014. Transitioning from proportion of distinguishers to a more meaningful measure of sensory difference. Food Qual. Prefer. 32, 77-82.
76. Kendall, S., Stuart, A. and Ord, J. 1987. Kendall's Advanced Theory of Statistics, 2nd Ed., Vol. 1, Oxford University Press, New York, NY.
77. Kim, H., Jeon, S., Kim, K. and O'Mahony, M. 2006a. Thurstonian models and variance I: Experimental confirmation of cognitive strategies for difference tests and effects of perceptual variance. J. Sensory Studies 21, 465-484.
78. Kim, H., Lee, H., O'Mahony, M. and Kim, K. 2008. Paired preference tests using placebo pairs and different response options for chips, orange juice, and cookies. J. Sensory Studies 23, 417-438.
79. Kim, H.-J., Kim, K., Jeon, S.Y., Kim, J.-M. and O'Mahony, M. 2006b. Thurstonian models and variance II: Experimental confirmation of the effects of variance on Thurstonian models of scaling. J. Sensory Studies 21, 485-504.
80. Kim, M. and Lee, H. 2012. Investigation of operationally more powerful duo-trio test protocols: Effects of different reference schemes. Food Qual. Prefer. 25, 183-191.
81. Kim, M., Chae, J., van Hout, D. and Lee, H. 2012. Discriminations of the A-Not A difference test improved when "A" was familiarized using a brand image. Food Qual. Prefer. 23, 3-12.
82. Kim, M.-A., Lee, Y.-M. and Lee, H.-S. 2010. Comparison of d′ estimates produced by three versions of a duo-trio test for discriminating tomato juices with varying salt concentrations: The effects of the number and position of the reference stimulus. Food Qual. Prefer. 21, 504-511.
83. Kunert, J. 2001. On repeated difference testing. Food Qual. Prefer. 12, 385-391.
84. Lau, S., O'Mahony, M. and Rousseau, B. 2004. Are three-sample tasks less sensitive than two-sample tasks? Memory effects in the testing of taste discrimination. Percept. Psychophys. 66, 464-474.
85. Lawless, H. and Heymann, H. 2010. Sensory Evaluation of Food: Principles and Practices, Springer, New York, NY.
86. Lee, H. and O'Mahony, M. 2007. The evolution of a model: A review of Thurstonian and conditional stimulus effects on difference testing. Food Qual. Prefer. 18, 369-383.
87. Lee, H., van Hout, D. and Hautus, M. 2007. Comparison of performance in the A-Not A, 2-AFC, and same-different tests for the flavor discrimination of margarines: The effect of cognitive decision strategies. Food Qual. Prefer. 18, 920-928.
88. Lee, H.-S. and Kim, K.-O. 2008. Difference test sensitivity: Comparison of three versions of the duo-trio method requiring different memory schemes and taste sequences. Food Qual. Prefer. 19, 97-102.
89. Lee, H.-S. and van Hout, D. 2009. Quantification of sensory and food quality: The R-index analysis. J. Food Sci. 74, R57-R64.
90. Lee, Y.-M., Chae, J.-E. and Lee, H.-S. 2009. Effects of order of tasting in sensory difference tests using apple juice stimuli: Development of a new model. J. Food Sci. 74, S268-S275.
91. Liggett, R.E. and Delwiche, J.F. 2005. The beta-binomial model: Variability in overdispersion across methods and over time. J. Sensory Studies 20, 48-61.
92. Macmillan, N.A. and Creelman, C.D. 2005. Detection Theory: A User's Guide. Cambridge University Press, New York.
93. MacRae, A.W. 1995. Confidence intervals for the Triangle test can give reassurance that products are similar. Food Qual. Prefer. 6, 61-67.
94. Masuoka, S., Hatjopoulos, D. and O'Mahony, M. 1995. Beer bitterness detection: Testing Thurstonian and Sequential Sensitivity Analysis models for triad and tetrad methods. J. Sensory Studies 10, 295-306.
95. Mata-Garcia, M., Angulo, O. and O'Mahony, M. 2007. On warm-up. J. Sensory Studies 22, 187-193.
96. McClure, S. and Lawless, H. 2010. Comparison of the triangle and a self-defined two alternative forced choice test. Food Qual. Prefer. 21, 547-552.
97. Meilgaard, M., Civille, G. and Carr, B. 2007. Sensory Evaluation Techniques, Taylor & Francis, Boca Raton, FL.
98. Meyners, M. 2007a. Proper and improper use and interpretation of beta-binomial models in the analysis of replicated difference and preference tests. Food Qual. Prefer. 18, 741-750.
99. Meyners, M. 2007b. Least equivalent allowable differences in equivalence testing. Food Qual. Prefer. 18, 541-547.
100. Meyners, M. 2012. Equivalence tests - A review. Food Qual. Prefer. 26, 231-245.
101. Morrison, D. 1978. A probability model for forced binary choices. Am. Stat. 32, 23-25.
102. Mullen, K. and Ennis, D. 1987. Mathematical formulation of multivariate Euclidean models for discrimination methods. Psychometrika 52, 235-249.
103. Mullen, K. and Ennis, D. 1991. A simple multivariate probabilistic model for preferential and triadic choices. Psychometrika 56, 69-75.
104. Mullen, K., Ennis, D., de Doncker, E. and Kapenga, J. 1988. Models for the duo-trio and triangular methods. Biometrics 44, 1169-1175.
105. Naes, T., Brockhoff, P. and Tomic, O. 2010. Statistics for Sensory and Consumer Science, John Wiley & Sons, Ltd, West Sussex, U.K.
106. Ogilvie, J. and Creelman, C. 1968. Maximum-likelihood estimation of receiver operating characteristic curve parameters. J. Math. Psychol. 5, 377-391.
107. O'Mahony, M. 1992. Understanding discrimination tests: A user-friendly treatment of response bias, rating and ranking R-index tests and their relationship to signal detection. J. Sensory Studies 7, 1-47.
108. O'Mahony, M. and Hautus, M. 2008. The signal detection theory ROC curve: Some applications in food sensory science. J. Sensory Studies 23, 186-204.
109. O'Mahony, M. and Rousseau, B. 2002. Discrimination testing: a few ideas, old and new. Food Qual. Prefer. 14, 157-164.
110. O'Mahony, M., Masuoka, S. and Ishii, R. 1994. A theoretical note on difference tests: models, paradoxes and cognitive strategies. J. Sensory Studies 9, 247-272.
111. Pawitan, Y. 2001. In All Likelihood Statistical Modelling and Inference Using Likelihood, Oxford University Press, Oxford, U.K.
112. Peryam, D.R. and Swartz, V.W. 1950. Measurement of sensory differences. Food Technol. (Chicago) IV, 390-395.
113. Petrov, V.V. 1976. Sums of independent random variables. Bull. Amer. Math. Soc. 83, 696-697.
114. Rousseau, B. 2007. Simultaneous estimations of multiple product similarities using a new discrimination protocol. J. Sensory Studies 22, 533-549.
115. Rousseau, B. and Ennis, D. 2001. A Thurstonian model for the dual pair (4IAX) discrimination method. Percept. Psychophys. 63, 1083-1090.
116. Rousseau, B. and Ennis, J.M. 2013. Importance of correct instructions in the Tetrad test. J. Sensory Studies 28, 264-269.
117. Rousseau, B. and O'Mahony, M. 1997. Sensory difference tests: Thurstonian and SSA predictions for vanilla flavored yogurts. J. Sensory Studies 12, 127-146.
118. Rousseau, B. and O'Mahony, M. 2001. Investigation of the dual-pair method as a possible alternative to the triangle and same-different tests. J. Sensory Studies 16, 161-178.
119. Rousseau, B., Meyer, A. and O'Mahony, M. 1998. Power and sensitivity of the same-different test: Comparison with triangle and duo-trio methods. J. Sensory Studies 13, 149-173.
120. Rousseau, B., Stroh, S. and O'Mahony, M. 2002. Investigating more powerful discrimination tests with consumers: effects of memory and response bias. Food Qual. Prefer. 13, 39-45.
121. Santosa, M., Hautus, M. and O'Mahony, M. 2011. ROC curve analysis to determine effects of repetition on the criteria for Same-Different and A Not-A tests. Food Qual. Prefer. 22, 66-77.
122. Sauvageot, F., Herbreteau, V., Berger, M. and Dacremont, C. 2012. A comparison between nine laboratories performing triangle tests. Food Qual. Prefer. 24, 1-7.
123. Schlich, P. 1993. Risk tables for discrimination tests. Food Qual. Prefer. 4, 141-151.
124. Stein, R.B., Gossen, E.R. and Jones, K.E. 2005. Neuronal variability: noise or part of the signal? Nat. Rev. Neurosci. 6, 389-397.
125. Stillman, J. 1993. Response selection, sensitivity, and taste-test performance. Percept. Psychophys. 54, 190-194.
126. Stillman, J. and Irwin, R. 1995. Extracting more information from the triangle task. J. Sensory Studies 10, 105-112.
127. Stone, H., Bleibaum, R. and Thomas, H.A. 2012. Sensory Evaluation Practices, 4th Ed., Academic Press, New York, NY.
128. Sung, Y.-E., Lee, H.-S., O'Mahony, M. and Kim, K.-O. 2011. Paired preference tests: Use of placebo stimuli with liking and buying preferences. J. Sensory Studies 26, 106-117.
129. Tedja, S., Nonaka, R., Ennis, D. and O'Mahony, M. 1994. Triadic discrimination testing: refinement of Thurstonian and Sequential Sensitivity Analysis approaches. Chem. Senses 19, 279-301.
130. Thurstone, L. 1927. A law of comparative judgment. Psychol. Rev. 34, 273-286.
131. Torgerson, W. 1958. Theory and Methods of Scaling, Wiley, New York, NY.
132. Ura, S. 1960. Pair, triangle and duo-trio test. Rep. Stat. Appl. Res., JUSE 7, 107-119.
133. van Hout, D., Hautus, M.J. and Lee, H.-S. 2011. Investigation of test performance over repeated sessions using signal detection theory: comparison of three nonattribute-specified difference tests 2-AFCR, A-NOT A and 2-AFC. J. Sensory Studies 26, 311-321.
134. White, J.A., Rubinstein, J.T. and Kay, A.R. 2000. Channel noise in neurons. Trends Neurosci. 23, 131-137.
135. Wichchukit, S. and O'Mahony, M. 2010. A transfer of technology from engineering: Use of ROC curves from signal detection theory to investigate information processing in the brain during sensory difference testing. J. Food Sci. 75, R183-R193.
136. Young, T.A., Pecore, S., Stoer, N., Hulting, F., Holschuh, N. and Case, F. 2008. Incorporating test and control product variability in degree of difference tests. Food Qual. Prefer. 19, 734-736.