Psychometrics, Dynamics, and Functional Data Analysis

Psychometrics, Dynamics, andFunctional Data Analysis“TheJim RamsayMcGill Universityviews

Testing as Input/Output Analysis• A test score is actually a derivative with respectto time.• Consequently a differential equation model fortesting data seems natural.• Dynamic testing data will be more and moreimportant.• We have some new tools for working withdynamic data.• So let’s s consider how to use time as a covariate.

Learning to Play Golf• We buy some clubs. We play a few games, eachbeing an 18 item test. It’s s harder than it looks.• We take a lesson. We play a few games. Ourscore improves to a better level.• We take another lesson, play some games, andthings improve again.• Key question: How quickly is a lesson reflectedin an improvement in score?

Brain Power• What counts is problem solving per unittime.• Power = energy expended per unit time.• Brain power = maximum difficulty ofproblem solvable per unit time, or• number of lighter problems solved per unittime.• That is, brain power = d brainergy/dtdt.

Brain Power and Time ScalesWe need the concept of brain power when weconsider intelligence on two time scales:1. Long term: : How much knowledge is availableover large time intervals, like a school year2. Short term: : How much new knowledge can beacquired over a short time interval, like asingle class.

Tests Measure Brain Power• Mental tests and psychological scales areone of the greatest technologicalachievements of the 20 th century.• Tests work so well because they are time-limited.• Test scores reflect brain power rather thanbrainergy.

Inputs to Brain Power• Information about the structure of theproblems.• A set of tools to solve them.• Training in the use of these tools.• All these require time.• Inputs to acquisition of brain power arefunctions of time.

A Differential Equation in Time• Links one or more time-derivatives,dx/dtdt,d 2 x/dt 2 ,…,, to the function x(t) itself.• Is a model for system dynamics: changeover time.• Can also include one or more input orcovariate functions.• x(t) is a long-term description.• dx/dtdt is a short-term term description.

A Simple ExampledEdt=− βE() t + α f()t• E(t) is brainergy, dE/dtdt is brainpower.• f(t) is an input function of time,such as education.• α and β are constants, β > 0. 0

• Most differential equations don’t t have explicitsolutions, but this one does.• Let E 0 be brainergy at time t = 0, and whichwill often be 0.t=−βt+ α∫00βtEt () e [ E e f ( udu ) ]•Let’s s see what happens when α=1,β varies,and f(t) is a step function.

10.8E(t)0.60.40.2β = 1β = 3β = 10f(t)00 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2Timet=−βt+ α∫00βtEt () e [ E e f ( udu ) ]

• The slope of E(t) when f(t) goes positive isβ.• β controls how fast the system respondsto the input f(t).• If the system is a problem solver, then βindicates how quickly the person learns tosolve a problem.• After about 4/β time units, full capacity isreached, and the system is ready for moreinput.

Fitting Differential Equations• We have noisy discrete-time time data, andwant to use them to estimate a differentialequation.• We want a solution E(t) to the equation tofit the data as well as possible.• We need lots of flexibility in choosing adifferential equation, and we can’t t assumethat there is an explicit solution to theequation.

Functional Data Analysis• A collection of methods for analyzingcurves or functions as data• A common theme is using derivatives invarious ways• See Ramsay and Silverman (1997)Functional Data Analysis. . Springer.• And Ramsay and Silverman (2002) AppliedFunctional Data Analysis. . Springer.

Two Functional Data AnalysisTechniques• L-splineSmoothing: : given noisy data anda differential equation, find a function E(t)that will smooth the data and at the sametime be nearly a solution to the differentialequation.• Principal Differential Analysis: : given afunction E(t), estimate a linear differentialequation for which E(t) is a solution.

Estimating a DIFE from noisy data• We’ve recently combined these two methodsinto a technique for estimating a differentialequation from noisy data.• In our simple example, this amounts toestimating parameters α and β.• But much more complex DIFE’s s can beestimated as well, including linear or nonlinear,and single or multiple variable systems.

An Oil Refinery• Here are some data from an oil refinery inCorpus Christi.• The input f(t) (reflux flow) is negativelycoupled to the output E(t) (tray 47 level).• The smooth curve is a solution to thedifferential equation that best representsthis relationship.

Tray 47 Level432100 20 40 60 80 100 120 140 160 180Reflux Flow0−0.1−0.2−0.3−0.4−0.50 20 40 60 80 100 120 140 160 180Perhaps this oil refinery is not too smart!

• Many situations will call for multipleoutputs: Performance with a putter, adriver, and an iron, for example. Or inalgebra and geometry.• And many situations will involve multipleinputs: Regular classes, tutoring sessions,labs and etc.• The technology used in these illustrationscan handle these situations, at least forlinear differential equations. Nonlinearequations don’t t pose any problem inprinciple.

Some Simulated Data• Imagine that the data are golf scores oversuccessive games, and that the input is aset of three equally-spaced lessons from agolf pro.• The following slides show three golfers.Which is a future Tiger Woods?

Rate constant = 532.5Performance21.510.500 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2EducationThese lessons are nicely timed.

Rate constant = 132.5Performance21.510.500 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2EducationThis person needs to find another sport!

Rate constant = 2032.5Performance21.510.500 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2EducationThis person should get lessons more often!

Is this Model Good Enough?• Specifying β to be constant is too simple.Allowing for fatigue, boredom, and otherthings requires a function β(t).• A first order equation can’t t allow forsudden transient effects like insight. Wemay need a differential equation involvinghigher derivatives.• We may need nonlinear equations as well.

A Nonlinear Differential EquationdEdtdEdtA( BE ( t))=− [ ( ) + ] + α f ( t)21 2β E1 t21 + C( BE2( t))A( BE ( t))=− [ ( ) + ] + α f ( t)22 1β E2 t21 + C( BE1( t))•The summed output from these two equationswill exhibit both the rapid learning and long-termretention required of human learners.•See H. R. Wilson (1999) Spikes, Decisions andActions, Oxford, for many more examples ofdifferential equation models in neuroscience.

Control Theory• Engineers who work with input/outputsystems have developed ways ofdesigning feedback loops to optimizeoutputs.• We’re working with a team of chemicalengineers at Queen’s s University.

Where Would the Data Come From?• Can we design customized learningsituations, like golf, and track how alearner makes progress as a function oftime and inputs?• Perhaps video and computer games arenearly what we need.• We already know that people will pay bigmoney to have these experiences.• Would corporations with deep pockets payfor this kind of testing?

Conclusions• Dynamic testing would generateperformance data over time that dependon one or more functional covariates.• New tools are available for these data thatfit them with a differential equation.• Dynamic psychometrics looks promising!