8. Multi-Factor Designs

8. Multi-Factor Designs
Chapter 8. Experimental Design II: Factorial Designs
1

Goals
• Identify, describe and create multifactor
(a.k.a. “factorial”) designs
• Identify and interpret main effects and
interaction effects
• Calculate N for a given factorial design
2

Complexity and Design
• As experimental designs increase in complexity:
• More information can be obtained.
• Care in design becomes ever more important.
• Designs with multiple factors and levels:
• Allow detection of interaction effects
• Allow detection of non-linear effects
• Involve more complexity around potential sequence
effects and equivalent groups problems
3

8.1 Describing
Multi-Factor Designs
4

Multi-Factor Designs
• Have more than one IV (or factor). a.k.a. “factorial
design”
• Described by a numbering system that gives the
number of levels of each IV
Examples: “2 × 2” or “3 × 4 × 2” design
• Also described by factorial matrices
5

Numbering System for
Factorial Designs
• Number of digits = number of IVs:
• “3 × 3” or “5 × 2” means two IVs.
• “2 × 2 × 2” or “3 × 4 × 2” means three IVs.
• Value of each digit = # of levels in each IV:
• 3 × 3 means two IVs, each with three levels.
• 3 × 4 × 2 means three IVs with 3, 4 and 2
levels, respectively
6

2 x 2 Factorial Design
Psychotherapy
Drug Therapy
Placebo Prozac
None Control Prozac
CBT CBT
7
Combined
Therapy

2 x 3 Factorial Design
Psychotherapy
Drug Therapy
Placebo Prozac
None Control Prozac
CBT CBT
EFT EFT
8
CBT +
Prozac
EFT +
Prozac

Levels vs. Conditions
• Level: One level of one IV.
A row or column in the Factorial Matrix.
Also, for 3+ IVs, one of the sub-matrices
• Condition: A particular combination of one
level of each IV.
One cell in the Factorial Matrix.
• In single-factor designs: level = condition
12

Placebo Level of Drug
Therapy IV
Drug Therapy
Placebo Prozac
Psycho-
None Control Prozac
therapy
CBT CBT Combo
13

Prozac Level of Drug
Therapy IV
Drug Therapy
Placebo Prozac
Psycho-
None Control Prozac
therapy
CBT CBT Combo
14

None Level of
Psychotherapy IV
Drug Therapy
Placebo Prozac
Psycho-
None Control Prozac
therapy
CBT CBT Combo
15

CBT Level of
Psychotherapy IV
Drug Therapy
Placebo Prozac
Psycho-
None Control Prozac
therapy
CBT CBT Combo
16

One-factor Designs
2-level
Multilevel
Study Time
2 Hours 5 Hours
2
Hours
Study Time
3
Hours
4
Hours
5
Hours
17

Discussion / Questions
• Why are the terms level and factor
interchangeable in a single-factor design?
• How many IVs are there in a 3×2×2 design?
How many levels of each IV? How many
total conditions?
18

8.2 Interpreting Data From
Multi-Factor Designs
19

Interpreting Data from
Factorial Designs
• Two types of effects can emerge in multi-factorial
designs:
• Main Effects: When one IV has an effect on its own.
That is, the mean for some pair of levels of the IV
differ significantly from one another.
• Interaction Effects: When the effect of one IV is
different for different levels of another IV.
• These are NOT mutually exclusive
20

A Simple 2x2 Design
Drug Therapy
Placebo Prozac
Psycho-
None Control Prozac
therapy
CBT CBT Combo
21

Main Effect of
Psychotherapy
Psycho-
None
therapy
CBT
Drug Therapy
Placebo Prozac
(Control+ Prozac )
/ 2
(CBT + Combo)
/ 2
We collapse across the levels of all other
IVs to evaluate a main effect
22

Main Effect of Drug Therapy
Psycho-
None
therapy
CBT
Drug Therapy
Placebo Prozac
(Control+
CBT )
/2
(Prozac +
Combo)
/2
We collapse across the levels of all other
IVs to evaluate a main effect
23

Numerical Example
Psychotherapy
Drug Therapy
Placebo Prozac
None 12 ± 2 18 ± 1
CBT 17 ± 1 23 ± 3
24

Main Effect of
Psychotherapy?
Drug Therapy
Placebo Prozac
Psycho-
None (12+18)/2 = 15
therapy
CBT (17+23)/2 (17+23)/2 = 20
25

Main Effect of Drug
Therapy?
Psycho-
None 12+17
2
therapy
CBT 14.5
Drug Therapy
Placebo Prozac
18+23
2
20.5
26

Numerical Example
Drug Therapy
Placebo Prozac µ ∆
Psycho-
None 12 ± 2 18 ± 1 15 -6
therapy
CBT 17 ± 1 23 ± 3 20 -6
µ 14.5 20.5
∆ -5 -5
27

Numerical Example
Drug Therapy
Placebo Prozac µ ∆
Psycho-
None 12 ± 2 18 ± 1 15 -6
therapy
CBT 17 ± 1 30 ± 3 20 -13
µ 14.5 20.5
∆ -5 -12
Evidence of
Interaction
28

Discussion / Questions
• In a 3x3x2 design, how many potential main
effects are there? How many IVs would
you collapse across to evaluate each main
effect?
29

•
•
•
•
Example Multi-Factorial
Multi-factorial experiments manipulate several IVs to see
if their effects interact
Example Question: Does gender interact with
psychotherapy in affecting depression?
•
•
Two IVs:
Experiment
Gender. 2 Levels = male; female
Psychotherapy. 2 levels: control (none); experimental
(therapy)
One DV: Depression (measure = BDI)
30

!"
#!"
$!"
%!"
&!"
'()*+(,"""""""""""""
-)("*./+0123"
4./+012"
!"#$%&#'()*)+'
,-)./$"'()*)+'
!"
#!"
$!"
%!"
&!"
'()*+(,"""""""""""""
-)("*./+0123"
4./+012"
!"#$%&#'()*)+'
,-)./$"'()*)+'
!"
#!"
$!"
%!"
&!"
'()*+(,"""""""""""""
-)("*./+0123"
4./+012"
!"#$%&#'()*)+'
,-)./$"'()*)+'
!"
#!"
$!"
%!"
&!"
'()*+(,"""""""""""""
-)("*./+0123"
4./+012"
!"#$%&#'()*)+'
,-)./$"'()*)+'
!"
#!"
$!"
%!"
&!"
'()*+(,"""""""""""""
-)("*./+0123"
4./+012"
!"#$%&#'()*)+'
,-)./$"'()*)+'
!"
#!"
$!"
%!"
&!"
'()*+(,"""""""""""""
-)("*./+0123"
4./+012"
!"#$%&#'()*)+'
,-)./$"'()*)+'
!"
#!"
$!"
%!"
&!"
'()*+(,"""""""""""""
-)("*./+0123"
4./+012"
!"#$%&#'()*)+'
,-)./$"'()*)+'
!"
#!"
$!"
%!"
&!"
'()*+(,"""""""""""""
-)("*./+0123"
4./+012"
!"#$%&#'()*)+'
,-)./$"'()*)+'
31

Another 2-Factor Design, 3 Levels Per Factor
Task
Difficulty
Easy
Average
Hard
Arousal
Low Med High
Low
Easy
Low
Average
Low
Hard
32
Med
Easy
Med
Average
Med
Hard
High
Easy
High
Average
High
Hard

Another 2-Factor Design, 3 Levels Per Factor
Task
Difficulty
Arousal
Low Med High µ ΔLM ΔMH ΔLH
Easy 40 40 40 40 0 0 0
Avrge 15 30 15 20 15 -15 0
Hard 8 5 2 5 -3 -3 -6
µ 21 25 19
ΔEA -25 -10 -25
ΔAH -7 -25 -13
ΔEH -32 -35 -38
33

!"#$%#&'()"*
Results: 3x3 Design
'!"
&!"
%!"
$!"
#!"
!"
()*" +,-./0" 1.23"
+#%,-'.*
34
4567"
89,:52,"
15:-"

3x3 Results: Main Effects, No Interaction
Task
Difficult
y
Arousal
Low Med High µ ΔL
M
ΔM
H
ΔLH
Easy 30 40 50 40 10 10 20
Avrge 15 25 35 25 10 10 20
Hard 6 16 26 16 10 10 20
µ 17 27 37
ΔEA -15 -15 -15
ΔAH -9 -9 -9
ΔEH -24 -24 -24
35

3x3 Results: 2 Main Effects,
!"#$%#&'()"*
(!"
'!"
&!"
%!"
$!"
#!"
!"
No Interaction
)*+" ,-./01" 2/34"
+#%,-'.*
36
5678"
9:-;63-"
26;."

Interpreting Data from
Factorial Designs
• If one IV has an effect--that is, there’s a significant
effect of going from one level of that IV to
another, while ignoring (“collapsing across”) all
other IVs--then that IV is said to produce a “main
effect”.
• If the effect of one IV differs depending on the
level of another IV, there’s an interaction.
37

!"
#!"
$!"
%!"
&!"
'()*+(,"""""""""""""
-)("*./+0123"
4./+012"
!"#$%&#'()*)+'
,-)./$"'()*)+'
!"
#!"
$!"
%!"
&!"
'!"
()*+,)-"""""""""""""
.*)"+/0,1234"
5/0,123"
!"#$%&#'()*)+'
,-)./$"'()*)+'
!"
#!"
$!"
%!"
&!"
'()*+(,"""""""""""""
-)("*./+0123"
4./+012"
!"#$%&#'()*)+'
,-)./$"'()*)+'
!"
#!"
$!"
%!"
&!"
'()*+(,"""""""""""""
-)("*./+0123"
4./+012"
!"#$%&#'()*)+'
,-)./$"'()*)+'
!"
#!"
$!"
%!"
&!"
'()*+(,"""""""""""""
-)("*./+0123"
4./+012"
!"#$%&#'()*)+'
,-)./$"'()*)+'
!"
#!"
$!"
%!"
&!"
'()*+(,"""""""""""""
-)("*./+0123"
4./+012"
!"#$%&#'()*)+'
,-)./$"'()*)+'
!"
#!"
$!"
%!"
&!"
'()*+(,"""""""""""""
-)("*./+0123"
4./+012"
!"#$%&#'()*)+'
,-)./$"'()*)+'
!"
#!"
$!"
%!"
&!"
'()*+(,"""""""""""""
-)("*./+0123"
4./+012"
!"#$%&#'()*)+'
,-)./$"'()*)+'
38

•
•
The Importance of
Interactions
Interpretation of interaction fx
overrides interpretation of main fx
Example: What’s most important in
these results:
Main effect of gender?
Main effect of therapy?
Interaction of the two?
39
!"#$%&#'()*)+'
&!"
%!"
$!"
#!"
!"
'()*+(,""""""""""""" 4./+012"
-)("*./+0123"
,-)./$"'()*)+'
If the gender factor is ignored, the therapy
seems to simply be effective for all people.
But this is not true. It is effective for
females only.

X-Way Interactions
• When there are 2 IVs, a 2-way interaction
is possible,with 3 IVs, may have a 3-way
interaction, etc.
• 3-way interaction means the 2-way
interaction changes depending on a 3rd
variable.
40

!"
#!"
$!"
%!"
&!"
'()*+(,"""""""""""""
-)("*./+0123"
4./+012"
!"#$%&#'()*)+'
,-)./$"'()*)+'
!"
#!"
$!"
%!"
&!"
'()*+(,"""""""""""""
-)("*./+0123"
4./+012"
!"#$%&#'()*)+'
,-)./$"'()*)+'
!"
#!"
$!"
%!"
&!"
'()*+(,"""""""""""""
-)("*./+0123"
4./+012"
!"#$%&#'()*)+'
,-)./$"'()*)+'
!"
#!"
$!"
%!"
&!"
'()*+(,"""""""""""""
-)("*./+0123"
4./+012"
!"#$%&#'()*)+'
,-)./$"'()*)+'
!"
#!"
$!"
%!"
&!"
'()*+(,"""""""""""""
-)("*./+0123"
4./+012"
!"#$%&#'()*)+'
,-)./$"'()*)+'
!"
#!"
$!"
%!"
&!"
'()*+(,"""""""""""""
-)("*./+0123"
4./+012"
!"#$%&#'()*)+'
,-)./$"'()*)+'
Introverts Extroverts
Introverts Extroverts
Introverts Extroverts
41

Discussion / Questions
42

8.3 Mixed Multi-Factor
Designs
43

All Participants (N = 20)
Condition 1
(n = 10)
Review:
Between-Subjects Design
Condition 2
(n = 10)
44

All Participants (N=10)
Level 1 (N = 10)
Level 2 (N = 10)
Review:
Within-Subjects Design
45

Within, Between & Mixed
Multi-Factor Designs
• With multiple factors/IVs, one can mix different kinds
of variables (within/between; subject/manipulated,
etc.)
• If all IVs are within-subjects then the design is “fully
within”
• If all IVs are between-subjects then the design is “fully
between”
• Otherwise, it’s a “mixed” design
46

2x2 Fully
Between
Subjects
Design
All Participants (N = 20)
Condition A1B1
(n=5)
Condition A2B1
(n=5)
Condition A1B2
(n=5)
Condition A2B2
(n=5)
47

2x2 Fully
Within
Subjects
Design
Note that orders are not
shown, there would be 24 for
a fully-counterbalanced
design!
All Participants (n = 20)
Condition A1B1
(n = 20)
Condition A2B1
(n = 20)
Condition A1B2
(n = 20)
Condition A2B2
(n = 20)
48

Level B1 (10) All Participants (20) Level B2 (10)
A1B1
(10)
A2B1
(10)
2x2 Mixed Design
A1B2
(10)
A2B2
(10)
49

Fully Within-Subjects
Factorial Design
• a.k.a., Repeated-measures factorial design.
• All subjects are run through all conditions (i.e., all
cells of the factorial matrix).
• Same advantages/disadvantages as single-factor
repeated measures design
50

Example Experiment 1:
Fully Within-Subjects
• Question: Is face recognition more impaired
by inversion than object recognition?
• Method
• Subjects are 20 undergraduates
• Materials are pictures of 25 famous faces
and 25 common objects, either inverted or
not. (So 100 images in all).
51

Example Experiment 1
• Design: 2x2 Fully within-subjects factorial, with
factors being Type of Image (Face or Object)
and View (upright or inverted).
• Procedure: All 20 subjects are shown all 100
images several times in random order and asked
to identify each as quickly as possible.
Repeated-measures factorial design.
• DV is reaction time to name picture.
52

View
Upright
Inverted
Image Type
Face Object
53

Example Experiment 1
• Expected results: RT will be higher for inverted
images than upright ones (main effect). But this
effect will be greater for faces (interaction).
• Implications: Implies that there’s something
different about how people process faces as
compared to objects
54

!"#$%&'%()*"+,%
Possible Results
)!!"
(!!"
'!!"
&!!"
%!!"
$!!"
#!!"
!"
67849" :;

•
•
•
Fully Between-Subjects
Factorial Designs
Each subject run through only one condition (i.e.,
one cell of the factorial matrix)
If all IVs are subject variables, you have a
Nonequivalent groups factorial design
If all IVs are manipulated, decide how equivalent
groups are formed:
• Random assignment:
Independent groups factorial design
• Matching:
Matched groups factorial design
56

•
Example Experiment 2:
Fully Between-Subjects
Question: Same as before, “are faces more
affected by inversion than objects?”
• Method
•
•
Subjects are 80 undergraduates (note higher N
than within-Ss design).
Materials: Same as before, 25 pictures of faces,
25 pictures of objects, shown both upright and
inverted.
57

Example Experiment 2
• Design 2×2 fully between-subjects factorial
design. Assign subjects randomly to one of
four groups of 20. Independent groups
factorial design.
• Procedure: Each group sees 25 pictures
(upright faces, inverted face, upright
objects, or inverted objects).
58

View
Upright
Inverted
Image Type
Face Object
59

Discussion / Questions
60

Mixed Factorial Designs
• At least one IV within-subjects and one
between-subjects.
• Subjects run through all levels of some IVs,
but only single level of other IVs. That is,
each subject goes through one row or
column of the factorial matrix.
• Random assignment, matching,
counterbalancing can all be used.
61

Example Experiment 3:
Mixed Factorial Design
• Question: Is face recognition more impaired by
inversion than object recognition?
• Method
• Subjects are 40 undergraduates (note higher
N than fully within, but lower than fully
between).
• Materials are pictures of 25 famous faces and
25 objects, either inverted or not.
62

Example Experiment 3
• Design: 2x2 Mixed factorial with factors being
Type of Image (face or object, within) and
View (upright or inverted, between)
• Procedure: 20 subjects are shown the 50
inverted images (25 faces and 25 objects),
while 20 other subjects are shown the 50
upright images (25 faces, 25 objects).
63

View
Upright
Inverted
Image Type
Face Object
64

PxE Factorial Designs
• “Person by Environment”
• Variety of fully-between or mixed factorial
design
• At least one subject IV (person) and at least
one manipulated IV (“environment”)
65

•
Example Experiment 4:
PxE Design
Question: Does the effect of assigned study style
interact with preferred study style?
• Method
• Person IV: Ss assigned to groups based on preferred
study style: Crammers or Distributers. This is a subject
IV
• Enviro IV: Half of subjects in each above group are
assigned to study by cramming or by distributing
study. This is manipulated
66

Assigned Style
(manipulated)
Possible Results
Preferred Style (subject)
Crammer Distributer
Cramming 65 65
Distributing 80 90
67

!"#$%&'"
Possible Results
(""#
'"#
&"#
%"#
$"#
!"#
)*+,,5*1# 0-12*-3425*1#
)*+,,-./# 0-12*-342-./#
()*+,-,."/$*'0)%)*10"
Assigned Study Style
68

Interpreting Results From
PxE Designs
• Cannot draw causal links for the subject
variables, can draw causal links for the
manipulated (”environment”) variable.
• So a causal link can be established for
assigned style but not preferred style.
• Cannot draw causal links for interaction
effects.
69

Example 2x3x2 Study
Caspi et al., 2007, PNAS, 104 (47), 18860-18865
70

How Many Participants?
• If I need 50 participants per cell in a 2×2
factorial design, what is the total N?
• What if the design is fully within?
• What if the design is mixed?
• Answer the same questions for a 3×2×3
design with 10 participants per cell.
71

•
•
•
Analyzing Data From Multi-
Factor Designs
As for multi-level designs, multi-factor designs are generally
analyzed via ANOVA procedures:
•
•
•
Pre-tests for normality and other assumptions
2-way (or X-way) ANOVA/MANOVA/ANCOVA...
Post-hoc tests to examine effects in greater detail
Planned comparison techniques may also be involved
Note that there are no well-established techniques for
dealing with multi-factor ordinal-scale data
72

Discussion / Questions
73

8.4 Summary:
Design Complexity
74

Single-Factor, 2-Level
Experimental Designs
• Can’t detect non-linear effects.
• Can’t detect interactions.
• Involve only simple counter-balancing or
simple equivalent groups problems.
75

Single-Factor,
Multilevel Designs
• Can detect non-linear effects
• Can’t detect interactions
• May involve relatively complex counterbalancing
or equivalent groups problems
76

• Multi-factor Designs
• Can detect interactions and main effects
• Can detect non-linear effects where IVs have ≥
3 levels
•
Multi-Factor Designs
May involve both complex counter-balancing
and equivalent groups problems.
77

Conclusion:
Experimental Design
• Experiments and quasi-experiments are just
one way of doing research
• True experiments (not quasi) allow
conclusions about causality
• Next we will turn to observational
research, which is simpler in some ways
78