Sweating the Small Stuff: Does data cleaning and testing ... - Frontiers

More documents

Recommendations

Info

Kasper and ÜnlüAssumptions of factor analytic approachesMaraun, M. D. (1996). Metaphor takenas math: indeterminacy in the factoranalysis model. Multivariate Behav.Res. 31, 517–538.Martin, M., Mullis, I., and Kennedy, A.(2007). PIRLS 2006 Technical Report.Boston, MA: TIMSS & PIRLS InternationalStudy Center.Mattson, S. (1997). How to generatenon-normal data for simulation ofstructural equation models. MultivariateBehav. Res. 32, 355–373.Maydeu-Olivares, A. (2005). “Linearitem response theory, nonlinear itemresponse theory and factor analysis:a unified framework,” in ContemporaryPsychometrics, eds A. Maydeu-Olivares and J. J. McArdle (Mahwah,NJ: Lawrence Erlbaum Associates),73–102.McDonald, R. (1985). Factor Analysisand Related Methods. Hillsdale, NJ:Lawrence Erlbaum Associates.Mulaik, S. (2009). Foundations of FactorAnalysis. Boca Raton, FL: Chapman& Hall/CRC.Mullis, I. V., Kennedy, A. M., Martin, M.O., and Sainsbury, M. (2006). PIRLS2006 Assessment Framework andSpecifications. Boston, MA: TIMSS &PIRLS International Study Center.Muthén, B. (1978). Contributions tofactor analysis of dichotomous variables.Psychometrika 43, 551–560.Muthén, B. (1983). Latent variablestructural equation modeling withcategorical data. J. Econom. 22,43–65.Muthén, B. (1984). A general structuralequation model with dichotomous,ordered categorical, and continuouslatent variable indicators.Psychometrika 49, 115–132.Muthén, B. (1989). Dichotomousfactor analysis of symptomdata. Sociol. Methods Res. 18,19–65.Muthén, B., and Christoffersson, A.(1981). Simultaneous factor analysisof dichotomous variables in severalgroups. Psychometrika 46, 407–419.Muthén, B., and Kaplan, D. (1992). Acomparison of some methodologiesfor the factor analysis of non-normalLikert variables: a note on the size ofthe model. Br. J. Math. Stat. Psychol.45, 19–30.OECD. (2005). PISA 2003 TechnicalReport. Paris: OECD Publishing.OECD. (2012). PISA 2009 TechnicalReport. Paris: OECD Publishing.Pearson, K. (1901). On lines and planesof closest fit to a system of points inspace. Philos. Mag. 2, 559–572.R Development Core Team. (2011). R:A Language and Environment for StatisticalComputing. Vienna: R Foundationfor Statistical Computing.Ramberg, J. S., Tadikamalla, P. R.,Dudewicz, E. J., and Mykytka, E.F. (1979). A probability distributionand its uses in fitting data.Technometrics 21, 201–214.Reinartz, W. J., Echambadi, R., andChin, W. W. (2002). Generatingnon-normal data for simulationof structural equation modelsusing Mattson’s method. MultivariateBehav. Res. 37, 227–244.Roznowski, M., Tucker, L., andHumphreys, L. (1991). Threeapproaches to determining thedimensionality of binary items.Appl. Psychol. Meas. 15, 109–127.Seier, E. (2002). Comparisonof tests for univariatenormality. Retrieved from:http://interstat.statjournals.net/YEAR/2002/articles/0201001.pdf[March 4, 2013].Shapiro, S., and Wilk, M. (1965).An analysis of variance test fornormality (complete samples). Biometrika52, 591–611.Skrondal, A., and Rabe-Hesketh, S.(2004). Generalized Latent VariableModeling: Multilevel, Longitudinal,and Structural Equation Models.Boca Raton, FL: Chapman &Hall/CRC.Spearman, C. (1904). “General intelligence,”objectively determined andmeasured. Am. J. Psychol. 15,201–292.Sturzbecher, D., Kasper, D., Bönninger,J., and Rüdel, M. (2008). Evaluationder theoretischen Fahrerlaubnisprüfung:Methodische Konzeptionund Ergebnisse des Revisionsprojekts[Evaluation of the Theoretical DrivingLicense Test: MethodologicalConcepts and Results of the RevisionProject]. Dresden: TÜV|DEKRAarge tp 21.Thurstone, L. L. (1931). Multiple factoranalysis. Psychol. Rev. 38, 406–427.Thurstone, L. L. (1965). Multiple FactorAnalysis. Chicago, IL: University ofChicago Press.van der Linden, W., and Hambleton,R. (eds). (1997). Handbook of ModernItem Response Theory. New York:Springer.Velicer, W., and Jackson, D. (1990).Component analysis versus commonfactor analysis: some issues inselecting an appropriate procedure.Multivariate Behav. Res. 25, 1–28.Wang, C. (2001). Effect of Number ofResponse Categories and Scale Distributionon Factor Analysis. Master’sthesis, National Taiwan University,Taipei.Weng, L., and Cheng, C. (2005). Parallelanalysis with unidimensionalbinary data. Educ. Psychol. Meas. 65,697–716.Widaman, K. F. (2007). “Commonfactors versus components: principalsand principles, errors andmisconceptions,” in Factor Analysisat 100: Historical Developmentsand Future Directions, eds R. Cudeckand R. C. MacCallum (Mahwah,NJ: Lawrence Erlbaum Associates),177–204.Wirth, R., and Edwards, M. (2007).Item factor analysis: currentapproaches and future directions.Psychol. Methods 12,58–79.Conflict of Interest Statement: Theauthors declare that the research wasconducted in the absence of any commercialor financial relationships thatcould be construed as a potential conflictof interest.Received: 10 June 2012; accepted: 17 February2013; published online: 27 March2013.Citation: Kasper D and Ünlü A(2013) On the relevance of assumptionsassociated with classical factor analyticapproaches. Front. Psychol. 4:109.doi:10.3389/fpsyg.2013.00109This article was submitted to Frontiersin Quantitative Psychology and Measurement,a specialty of Frontiers in Psychology.Copyright © 2013 Kasper and Ünlü. Thisis an open-access article distributed underthe terms of the Creative Commons AttributionLicense, which permits use, distributionand reproduction in other forums,provided the original authors and sourceare credited and subject to any copyrightnotices concerning any third-partygraphics etc.Frontiers in Psychology | Quantitative Psychology and Measurement March 2013 | Volume 4 | Article 109 | 141
ORIGINAL RESEARCH ARTICLEpublished: 06 March 2012doi: 10.3389/fpsyg.2012.00056How predictable are “spontaneous decisions” and“hidden intentions”? Comparing classification resultsbased on previous responses with multivariate patternanalysis of fMRI BOLD signalsMartin Lages* and Katarzyna JaworskaSchool of Psychology, University of Glasgow, Glasgow, UKEdited by:Jason W. Osborne, Old DominionUniversity, USAReviewed by:Eric-Jan Wagenmakers, University ofAmsterdam, NetherlandsJudit Antal, College Board, USA*Correspondence:Martin Lages, Department ofPsychology, University of Glasgow, 8Hillhead Street, Scotland, G12 8QB,Glasgow, UK.e-mail: martin.lages@glasgow.ac.ukIn two replication studies we examined response bias and dependencies in voluntary decisions.We trained a linear classifier to predict “spontaneous decisions” and in the secondstudy “hidden intentions” from responses in preceding trials and achieved comparable predictionaccuracies as reported for multivariate pattern classification based on voxel activitiesin frontopolar cortex. We discuss implications of our findings and suggest ways to improveclassification analyses of fMRI BOLD signals that may help to reduce effects of responsedependencies between trials.Keywords: prediction accuracy, classification, sequential dependencies, response bias, decision making, free will,frontal cortexThe prospect of decoding brain activity to predict spontaneousor free decisions captivates not only the neuroscientific community(Haggard, 2008) but increasingly inspires researchers in otherdisciplines (Mobbs et al., 2007; Heisenberg, 2009).The purpose of this paper is to draw attention to possibleconfounds and improved data analyses when decoding neural correlatesto predict behavior. We focus on a specific task and setof results but we believe that the problem of sequential dependenciesis pervasive and needs to be considered carefully whenapplying machine learning algorithms to predict behavior frombrain imaging data.In two replication studies we illustrate how individual responsebias and response dependencies between trials may affect theprediction accuracy of classification analyses. Although our behavioralresults are not sufficient to dismiss the original findings basedon fMRI BOLD signals they highlight potential shortcomings andsuggest alternative ways to analyze the data.In recent studies Soon et al. (2008) and Bode et al. (2011)used a self-paced free or voluntary decision task to study unconsciousdeterminants preceding “spontaneous” motor decisions. Inboth studies subjects spontaneously pressed a left or right responsebutton with their corresponding index finger in a series of trials.Brain activity was measured by fMRI BOLD signals and the patternof voxel activity before (and after) the decision was used topredict binary motor responses. Soon et al. (2008) applied a linearmultivariate pattern classifier and searchlight technique to localizepatterns of predictive voxel activities (Haxby et al., 2001) andachieved 60% prediction accuracy in a localized region of frontopolarcortex (FPC). Bode et al. (2011), using a high-resolutionscan of the prefrontal cortex, reported 57% prediction accuracyfor the same task. The authors conclude that patterns of voxelactivities in FPC constitute the neural correlate of unconsciousdeterminants preceding spontaneous decisions.In both studies the activation patterns in FPC occurred up to10–12 s before participants reported their conscious decisions. Ifvalidated this finding would dramatically extend the timeline ofpre-SMA/SMA (Lau et al., 2004; Leuthold et al., 2004) aswellas results on readiness potentials for voluntary acts (Libet et al.,1983; see however Trevena and Miller, 2010) with far-reachingimplications (Mobbs et al., 2007; Heisenberg, 2009).Soon et al. (2008) and Bode et al. (2011) made considerableattempts to control carry-over effects from one trial to the next andselected a subset of trials with balanced left and right responsesto eliminate response bias. We argue that despite these precautionsresponse dependencies in combination with response bias inthe raw data may have introduced spurious correlations betweenpatterns of voxel activities and decisions.Naïve participants have difficulties to generate randomsequences and sequential dependencies across trials are commonlyobserved in binary decisions (Lages and Treisman, 1998;Lages, 1999, 2002) as well as random response tasks (Bakan,1960; Treisman and Faulkner, 1987). In some tasks, these dependenciesare not just simple carry-over effects from one trial tothe next but reflect stimulus and response dependencies (Lagesand Treisman, 1998) as well as contextual information processing(Lages and Treisman, 2010; Treisman and Lages, 2010), oftenstraddling across several trials and long inter-trial-intervals (Lagesand Paul, 2006). These dependencies may indicate involvementof memory and possibly executive control (Luce, 1986), especiallyin self-ordered tasks where generation of a response requiresmonitoring previously executed responses (Christoff and Gabrieli,2000).In order to address the issue of sequential dependencies inconnection with response bias we conducted two behavioralreplication studies and performed several analyses on individualbehavioral data, the results of which are summarized below.www.frontiersin.org March 2012 | Volume 3 | Article 56 | 142
Page 2 and 3:
FRONTIERS COPYRIGHTSTATEMENT© Copy
Page 4 and 5:
Table of Contents05 Is Data Cleanin
Page 7 and 8:
OsborneAssumptions and data cleanin
Page 9 and 10:
ORIGINAL RESEARCH ARTICLEpublished:
Page 12 and 13:
Hoekstra et al.Why assumptions are
Page 14 and 15:
Page 16 and 17:
Page 18 and 19:
Page 20 and 21:
García-PérezStatistical conclusio
Page 22 and 23:
Page 24 and 25:
Page 26 and 27:
Page 28 and 29:
Page 30 and 31:
Sheng and ShengEffect of non-normal
Page 32 and 33:
Page 34 and 35:
Page 36 and 37:
Page 38 and 39:
Page 40 and 41:
Page 42 and 43:
REVIEW ARTICLEpublished: 12 April 2
Page 44 and 45:
Nimon et al.The assumption of relia
Page 46 and 47:
Page 48 and 49:
Page 50 and 51:
Page 52 and 53:
Page 54 and 55:
Page 56 and 57:
TressoldiPower replication unreliab
Page 58 and 59:
TressoldiPower replication unreliab
Page 60 and 61:
Page 62 and 63:
FinchModern methods for the detecti
Page 64 and 65:
Page 66 and 67:
Page 68 and 69:
Page 70 and 71:
Page 72 and 73:
MINI REVIEW ARTICLEpublished: 28 Au
Page 74 and 75:
NimonStatistical assumptionsand Del
Page 76 and 77:
NimonStatistical assumptionsFor exa
Page 78 and 79:
Kraha et al.Interpreting multiple r
Page 80 and 81:
Page 82 and 83:
Page 84 and 85:
Page 86 and 87:
Page 88 and 89:
Page 90 and 91:
Page 92 and 93: Kraha et al.Interpreting multiple r
Page 94 and 95: SmithsonComparing moderation of slo
Page 102 and 103: REVIEW ARTICLEpublished: 01 March 2
Page 104 and 105: Flora et al.Factor analysis assumpt
Page 124 and 125: Kasper and ÜnlüAssumptions of fac
Page 144 and 145: Lages and JaworskaHow predictable a
Page 152 and 153: Cummiskey et al.Testing assumptions
Page 158: Cummiskey et al.Testing assumptions
show all

Sweating the Small Stuff: Does data cleaning and testing ... - Frontiers

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?