Individual differences in replicated multi-product 2-AFC data with ...

Individual differences in replicated multi-product 2-AFCdata with and without supplementary difference scoring:Comparing Thurstonian mixed regression models forbinary and ordinal data with linear mixed modelsChristine Borgen Linander 1,∗ , Rune Haubo Bojesen Christensen 1 ,Rebecca Evans 2 , Graham Cleaver 2 , Per Bruun Brockhoff 11 Department of Informatics and Mathematical Modeling, Section for Statistics, TechnicalUniversity of Denmark2 Unilever Research and Development Port Sunlight, UK*Contact author: chjo@imm.dtu.dk| Christine Borgen Linander | Sensometrics 2012 | July 13th 2012 | 1/30 |

IntroductionModelsWe consider 2 types of models:Naive aggregation model - ignoring the replicated structureMixed model - including Assessor as a random effect, modelling thereplicated structureWhere the naive model will be fitted with the binary response.And the mixed model will be fitted with the 3 different responses.| Christine Borgen Linander | Sensometrics 2012 | July 13th 2012 | 6/30 |

ResultsF test - mixed model - Quantitative responseThe PanelCheck way by the R-package lmerTestAssessor−by−Product interaction effectsF statistic0 10 20 30 400.05 < p−value0.01 < p−value < 0.050.001 < p−value < 0.01p−value < 0.001A B C D E C.5m D.5m E.5mAttributes| Christine Borgen Linander | Sensometrics 2012 | July 13th 2012 | 7/30 |

ResultsF test - mixed model - Quantitative responseAssessor main effectsF statistic0 10 20 30 40 500.05 < p−value0.01 < p−value < 0.050.001 < p−value < 0.01p−value < 0.001A B C D E C.5m D.5m E.5mAttributes| Christine Borgen Linander | Sensometrics 2012 | July 13th 2012 | 8/30 |

ResultsF test - mixed model - Quantitative responseProduct main effectsF statistic0 10 20 30 400.05 < p−value0.01 < p−value < 0.050.001 < p−value < 0.01p−value < 0.001A B C D E C.5m D.5m E.5mAttributes| Christine Borgen Linander | Sensometrics 2012 | July 13th 2012 | 9/30 |

ResultsLikelihood Ratio test - mixed model - Binary responseAssessor−by−Product interaction effectsLikelihood Ratio statistic0.0 0.4 0.8 1.20.05 < p−value0.01 < p−value < 0.050.001 < p−value < 0.01p−value < 0.001A B C D E C.5m D.5m E.5mAttributes| Christine Borgen Linander | Sensometrics 2012 | July 13th 2012 | 10/30 |

ResultsLikelihood Ratio test - mixed model - Binary responseAssessor main effectsLikelihood Ratio statistic0 10 20 30 400.05 < p−value0.01 < p−value < 0.050.001 < p−value < 0.01p−value < 0.001A B C D E C.5m D.5m E.5mAttributes| Christine Borgen Linander | Sensometrics 2012 | July 13th 2012 | 11/30 |

ResultsLikelihood Ratio test - mixed model - Binary responseProduct main effects − mixed modelLikelihood Ratio statistic0 50 100 2000.05 < p−value0.01 < p−value < 0.050.001 < p−value < 0.01p−value < 0.001A B C D E C.5m D.5m E.5mAttributes| Christine Borgen Linander | Sensometrics 2012 | July 13th 2012 | 12/30 |

ResultsLikelihood Ratio test - naive model - Binary responseProduct main effects − naive modelLikelihood Ratio statistic0 50 100 2000.05 < p−value0.01 < p−value < 0.050.001 < p−value < 0.01p−value < 0.001A B C D E C.5m D.5m E.5mAttributes| Christine Borgen Linander | Sensometrics 2012 | July 13th 2012 | 13/30 |

ResultsLikelihood Ratio test - comparison of Product - BinaryresponseComparing test statistics for the mixed and naive modelLikelihood Ratio statistic0 50 100 150 200Mixed modelNaive modelA B C D E C.5m D.5m E.5mAttributes| Christine Borgen Linander | Sensometrics 2012 | July 13th 2012 | 14/30 |

ResultsComparing the two models for Product - Binary responseNot a big difference in values between the methods. This fits nicelywith the Assessor-by-Product interaction being non-significantIn general you would expect the values from the mixed model to beless than the values from the naive modelFor some attributes this is not the case. These are the ones with thehighest values of the test of Assessor main effectLooking at the plot of the Assessor main effect you would expect thatthis was also the case for for the D.5m attribute.But looking at the plot of the Assessor-by-Product interaction this isthe attribute with the highest value| Christine Borgen Linander | Sensometrics 2012 | July 13th 2012 | 15/30 |

ResultsPost hoc - Product differences in terms of d-primeProduct estimates for the attribute E.5md−prime(95% confidence interval)−4 −3 −2 −1 0 1 2A B C D E F G H I J K LProducts| Christine Borgen Linander | Sensometrics 2012 | July 13th 2012 | 16/30 |

ResultsPost hoc - Product differences in terms of d-primeABCDd−prime−2 0 2 4d−prime−2 0 2d−prime−4 0 2d−prime−3 −1A D G JA D G JA D G JA D G JProductsProductsProductsProductsEC.5mD.5mE.5md−prime−300 0 300d−prime−5 −2 0 2d−prime−3 −1 1d−prime−4 −2 0 2A D G JA D G JA D G JA D G JProductsProductsProductsProducts| Christine Borgen Linander | Sensometrics 2012 | July 13th 2012 | 17/30 |

ResultsPost hoc - PCA of product d-prime valuesBi plotPrincipal component 2 (11.4%)−1 0 1 2 3 4LEHAJCKB BEA DE.5mD.5mGCC.5mDIF−6 −4 −2 0 2 4 6Principal component 1 (74.4%)| Christine Borgen Linander | Sensometrics 2012 | July 13th 2012 | 18/30 |

ResultsPost hoc - Assessor performance in terms of d-primeAssessor estimates for the attribute E.5md−prime(95% confidence interval)−2 −1 0 1 2| Christine Borgen Linander | Sensometrics 2012 | July 13th 2012 | 19/30 |

ResultsPost hoc - Assessor performance in terms of d-primeABCDd−prime−1.0 0.0 1.0d−prime−1.0 0.0 1.0d−prime−1.5 0.0 1.0d−prime−1 0 1 22 6 23 920 8 14 6 11 7 14 21 275 8 21 6AssessorsAssessorsAssessorsAssessorsEC.5mD.5mE.5md−prime−1.5 0.0 1.5d−prime−1 0 1 2d−prime−1.5 0.0 1.5d−prime−2 0 220 15 24 8 711 9 1 23 102 7 3 13 2020 24 21 5 7AssessorsAssessorsAssessorsAssessors| Christine Borgen Linander | Sensometrics 2012 | July 13th 2012 | 20/30 |

ResultsLikelihood Ratio test - mixed model - Ordinal responseUsing the R-package ordinalAssessor−by−Product interaction effectsLikelihood Ratio statistic0 5 10 15 200.05 < p−value0.01 < p−value < 0.050.001 < p−value < 0.01p−value < 0.001A B C D E C.5m D.5m E.5mAttributes| Christine Borgen Linander | Sensometrics 2012 | July 13th 2012 | 21/30 |

ResultsLikelihood Ratio test - mixed model - Ordinal responseAssessor main effectsLikelihood Ratio statistic0 10 20 30 40 500.05 < p−value0.01 < p−value < 0.050.001 < p−value < 0.01p−value < 0.001A B C D E C.5m D.5m E.5mAttributes| Christine Borgen Linander | Sensometrics 2012 | July 13th 2012 | 22/30 |

ResultsLikelihood Ratio test - mixed model - Ordinal responseProduct main effectsLikelihood Ratio statistic0 50 100 150 200 2500.05 < p−value0.01 < p−value < 0.050.001 < p−value < 0.01p−value < 0.001A B C D E C.5m D.5m E.5mAttributes| Christine Borgen Linander | Sensometrics 2012 | July 13th 2012 | 23/30 |

ResultsLikelihood Ratio test - comparison AssessorLikelihood Ratio test − Assessor main effectsLikelihood ratio statistic0 10 20 30 40 50BinaryOrdinal1 2 3 4 5 6 7 8Attributes| Christine Borgen Linander | Sensometrics 2012 | July 13th 2012 | 24/30 |

ResultsLikelihood Ratio test - comparison ProductLikelihood Ratio test − Product main effectsLikelihood ratio statistic0 50 100 150 200BinaryOrdinal1 2 3 4 5 6 7 8Attributes| Christine Borgen Linander | Sensometrics 2012 | July 13th 2012 | 25/30 |

ResultsLikelihood Ratio test - comparison Assessor-by-ProductLikelihood Ratio test − Assessor−by−Product interaction effectsLikelihood ratio statistic0 2 4 6 8 10 12BinaryOrdinal1 2 3 4 5 6 7 8Attributes| Christine Borgen Linander | Sensometrics 2012 | July 13th 2012 | 26/30 |

Final remarksFinal remarksWe can handle incomplete observationsWe can fit the Thurstonian mixed model with binary dataThe Thurstonian mixed model provides additional information (comparedto the naive aggregation model)d-prime interpretations of Product estimatesd-prime interpretations of Assessor estimatesThese preliminary results indicate that the ordinal information is moresensitive than the binary information| Christine Borgen Linander | Sensometrics 2012 | July 13th 2012 | 27/30 |

Future workFinal remarksAt some point in the future:Investigation of how much binary data is needed to find a significantAssessor-by-Product interactionCome up with a proper Thurstonian interpretation of the ordinalapproachGeneral modelling - for instance how to handle order effects| Christine Borgen Linander | Sensometrics 2012 | July 13th 2012 | 28/30 |

Final remarksThank you for your attention!| Christine Borgen Linander | Sensometrics 2012 | July 13th 2012 | 29/30 |

ReferencesFinal remarksPanelCheck: Open source software for sensory profile data,www.panelcheck.comChristensen, R. H. B. (2012). Ordinal - Regression Models for OrdinalData. R package version 2012.01-19http://www.cran.r-project.org/package=ordinal/Christensen, R. H. B. and P. B. Brockhoff (2012). Analysis ofreplicated categorical ratings data from sensory experiments. Workingpaper.Alexandra Kuznetsova, Per Bruun Brockhoff and Rune HauboBojesen Christensen (2012). lmerTest: Tests for random and fixedeffects for linear mixed effect models (lmer objects of lme4 package)..R package version 1.0.Brockhoff, P.B. and Christensen, R.H.B. (2010). Thurstonian modelsfor sensory discrimination tests as generalized linear models. FoodQuality and Preference 21(3), 330-338| Christine Borgen Linander | Sensometrics 2012 | July 13th 2012 | 30/30 |