Advances in Tetrad Testing

Sensometrics 2012Rennes, FranceAdvances in Tetrad TestingJohn M. EnnisThe Institute for Perceptionjohn.m.ennis@ifpress.comRune H.B. ChristensenTechnical University of Denmarkrhbc@imm.dtu.dkwww.ifpress.com

Discrimination Testing‣ Discrimination testing as important as ever: Compliance with health initiatives Cost reductions Changes to ingredients, processes, packaging, handling, etc. Quality control‣ Three challenges:1. Identify sensitive methods for unspecified testing2. Measurement:a) Quantify sensory differencesb) Understand precision in measurement3. Determine size of meaningful difference2

The Tetrad Test - Methodology‣ Four samples presented:“Group the stimuli into two groups oftwo samples based on similarity”‣ Six possible presentation orders:‣ Guessing probability = 1/3AABB, ABAB, ABBABBAA, BABA, BAAB3

The Tetrad Test - History‣ Mentioned by Lockhart (1951) and Gridgeman (1954)‣ Revisited by O’Mahony, Masuoka, & Ishii (1994)‣ First experiments: Masuoka, Hatjopolous, & O'Mahony (1995) Delwiche & O'Mahony (1996)‣ Psychometric function derived by Ennis et al. (1998)‣ Support for Tetrad testing in IFPrograms (2009)‣ Sample size tables published by Ennis & Jesionka (2011)‣ Operational power-based comparison with Triangle testby Ennis (2012)‣ Large-scale comparison with Triangle test by Garcia,Ennis, & Prinyawiwatkul (2012)‣ Support for Tetrad testing in sensR (2012)4

Experimental Results (1/3)‣ Masuoka, Hatjopoulos & O’Mahony (1995)‣ Beer samples varying in bitterness‣ 9 judges with 12 replications: N=108 per conditionProportion Correctd'0.730.630.530.430.331.51.00.50.0Triangle Tetrad3-AFCTriangle Tetrad3-AFC‣ d' values not significantly different5

Experimental Results (2/3)‣ Delwiche & O’Mahony (1996)‣ Chocolate pudding varying in sweetness‣ 13 judges with 12 replications: N = 156 per conditionProportion Correctd'0.930.830.730.630.530.430.332.52.01.51.00.50.0Triangle Tetrad3-AFCTriangle Tetrad3-AFC‣ d' values not significantly different6

Experimental Results (3/3)‣ Garcia, Prinyawiwatkul, Ennis (2012)‣ Apple juices varying in sweetness‣ 404 children: 1 Tetrad, 2 Triangle evaluationsProportion Correctd'0.631.50.531.00.430.50.330.0TriangleTetradTriangleTetrad‣ d' values not significantly different7

Percentage correctThurstonian Theory‣ Psychometric function (Ennis et al.,1998)100%3-AFC Tetrad Triangle90%80%70%60%50%40%30%0 1 2 3δ8

Δ(a)(b)(c)(d)(e)Triangle/Tetrad – Possible Cases (d = 1.5)WWW WSWW SS W WSW W• CorrectS• Wrong• Wrong• Wrong• Correct48.0%17.9%28.5%3.2%2.3%63.0%20.2%6.7%6.9%2.1%1.0%P c = 50.3% P c = 64.0%WWW WS W WS SSS WWS SSS WW WWSSS• Correct• Wrong• Wrong• Wrong• Wrong• Correct□(d)9(a)(b)(c)(e)(f)

Triangle/Tetrad – Sample Sizes‣ Suppose α = 0.05 and want 80% power‣ If δ = 1.5 Tetrad N = 20 Triangle N = 57‣ If δ = 1.0 Tetrad N = 65 Triangle N = 220‣ Tetrad sample sizes areroughly 1/3 Trianglesample sizes‣ See Ennis & Jesionka (2011)for more information250200150100500Tetrad Triangle1.5 1δ10

B ValuePrecision of Measurement (1/4)‣ Variance in estimate of δ (Bi, Ennis, & O’Mahony, 1997) Variance is B value divided by sample size35TetradTriangle3025201510500.5 1 1.5 2 2.5δ11

Precision of Measurement (2/4)‣ Tetrad test can be analyzed using GLM framework(Brockhoff and Christensen, 2010):P Cf □−1δ‣ Convenient access to statistical analysis12

Relative likelihoodPrecision of Measurement (3/4)‣ Relative likelihood (Christensen & Brockhoff, 2009)1.00 Function shape gives improved estimate of precision Example: N = 60, δ ~ 1TetradTriangle0.750.500.250.000.0 0.5 1.0 1.5 2.0 2.5δ13

Expected WidthPrecision of Measurement (4/4)‣ Expected widths of likelihood confidence intervals2.0 N = 60, 95% confidenceTetradTriangle1.51.00.50.00.0 0.5 1.0 1.5 2.0 2.5δ14

Comparative Examples (1/2)‣ Six pasta sauces for food service applications‣ Research to compare Triangle and Tetrad tests‣ Test sample sizes vary between 96 and 13273%Proportion correctTetradTriangle63%53%43%33%Mild Savory Pesto Alfredo Neopolitan Meat2.00d' valuesTetradTriangle1.501.000.500.00Mild Savory Pesto Alfredo Neopolitan Meat15

Final Points‣ Future topics: Equivalence Unequal variance Multivariate Tetrad model Comparison to 2-AFCR Decision rule investigation‣ Thanks to: Daniel Ennis & Benoit Rousseau, The Institute for Perception Pieter Punter, OP&P Product Research Per Brockhoff, Technical University of Denmark17

Sensometrics 2012Rennes, FranceAdvances in Tetrad TestingJohn M. EnnisThe Institute for Perceptionjohn.m.ennis@ifpress.comRune H.B. ChristensenTechnical University of Denmarkrhbc@imm.dtu.dkwww.ifpress.com