Presentation: A New DOE Paradigm - JMP

A New DOE Paradigm
• April 12, 2012
• Bradley Jones, SAS
Institute
• Doug Montgomery,
Department of Industrial
Engineering, Arizona
State University
1

New! Improved!
2

Design Optimality
• One of the truly great paradigm shifts in DOX
• Can create custom designs for almost any situation
• Modern software makes this easy for at least some
optimality criteria
• What optimality criteria should we use?
3

Three Criteria:
The D-criterion: M =
X' X
N
Maximize the determinant of M
D
eff
| X´
X |
Max[|
´ |]
X X
1/ p
The G-criterion: Minimize the maximum SPV
over R
NVar[ yˆ(
m
m
SPV
x )] ( )
Nx'
X' X 1
x
2
G
eff
p
max ( SPV )
xR
The I-criterion: Minimize the average prediction variance over R
I
1 NVar[ yˆ
( x)]
d
2
A
x
R
I eff
Min(
APV ( D))
APV ( D)
5

Which Criterion should I Use?
• First-order models, first-order models with interaction
– Objective is usually factor screening
– Parameter estimation is key
– Use the D-criterion
• Second-order models, mixture models
– Response estimation is usually of primary importance
– Use the G or I criterion
6

A problem with a constrained design region –
amount of adhesive and cure temperature
Page 391 & 392

How would we design an experiment for this
problem?
• “Force” a standard design into the experimental region
– May lead to a case of the “square peg and the round hole”
• Generate a unique design just for this particular situation
– Need criteria for constructing the design
– Computer implementation essential

JMP Default RSM Design

Design Comparison
2 reps
D-optimal Design Points
I-optimal Design Points

Design Comparison cont.
D-optimal Design Variance Profiles
I-optimal Design Variance Profile

Let’s back up
What we just showed was an experiment for optimization.
But, usually designed experimentation starts with screening.
We will first review some basics of screening.
Then we will show you some really new stuff.
12

Why do Regular Fractional Factorial
Designs Work for Factor Screening?
• The sparsity of effects principle
– There may be lots of factors, but few are important
– System is dominated by main effects, low-order interactions
• The projection property
– Every fractional factorial contains full factorials in fewer factors
• Sequential experimentation
– Can add runs to a fractional factorial to resolve difficulties (or
ambiguities) in interpretation
13

The Regular One-Half Fraction of the 2 k
• DCM, Design & Analysis of Experiments, 8 th Edition, Wiley 2012
• Notation: because the design has 2 k /2 runs, it’s referred to as a 2 k-1
• Consider a really simple case, the 2 3-1
• Note that I =ABC
14

The Regular One-Half Fraction of the 2 3
For the principal fraction, notice that the contrast for estimating the main
effect A is exactly the same as the contrast used for estimating the BC
interaction.
This phenomena is called aliasing and it occurs in all fractional designs
Aliases can be found directly from the columns in the table of + and - signs
15

Aliasing in the One-Half Fraction of the 2 3
A = BC, B = AC, C = AB (or me = 2fi)
Aliases can be found from the defining relation I = ABC
by multiplication:
AI = A(ABC) = A 2 BC = BC
BI =B(ABC) = AC
CI = C(ABC) = AB
Textbook notation for aliased effects:
[ A] A BC, [ B] B AC, [ C]
C AB
16

The Alternate Fraction of the 2 3-1
• I = -ABC is the defining relation
• Implies slightly different aliases: A = -BC, B= -AC,
and C = -AB
• Both designs belong to the same family, defined by
I ABC
• Suppose that after running the principal fraction, the
alternate fraction was also run
• The two groups of runs can be combined to form a full
factorial – an example of sequential experimentation
17

• Resolution III Designs:
– me = 2fi
– example
• Resolution IV Designs:
– 2fi = 2fi
– example
• Resolution V Designs:
– 2fi = 3fi
– example
Design Resolution
31
2 III
41
2 IV
51
2 V
18

Construction of a Regular One-half Fraction
The basic design; the design generator
19

Projection of Fractional Factorials
Every fractional factorial
contains full factorials in
fewer factors
The “flashlight” analogy
A one-half fraction will
project into a full
factorial in any k – 1 of
the original factors
20

Example 8.1
21

Example 8.1
Interpretation of results often relies on making some assumptions
Ockham’s razor
Confirmation experiments can be important
Adding more runs to resolve ambiguities
22

Confirmation experiment for this example:
Use the model to predict the response at a test combination of interest
in the design space – not one of the points in the current design.
Run this test combination – then compare predicted and observed.
For Example 8.1, consider the point +, +, -, +. The predicted response
is
Actual response is 104.
23

Suppose we believe in the model:
y = X 1
b 1
+ e
But the true model is:
y = X 1
b 1
+ X 2
b 2
+ e
Then, the Alias matrix is:
A = (X t 1
X 1
) -1 X t 1
X 2
The model for our specific example on the next slide is:
y = b 0
+x 1
b 1
+ x 2
b 2
+ x 3
b 3
+ e
24

Main effects aliased
with the 2fis
26

The Alias Matrix:
Columns represent the additional parameters in the full model
Rows represent
the parameters in
the model that
you fit
Effect A*B A*C B*C
Intercept 0 0 0
A 0 0 1
B 0 1 0
C 1 0 0
27

The Regular One-Quarter Fraction of the 2 k 28

The Regular One-Quarter Fraction of the 2 6-2
Complete defining relation: I = ABCE = BCDF = ADEF
This is a regular design 29

The Regular 2 k-p Fractional
Factorial Design
• 2 k-1 = one-half fraction, 2 k-2 = one-quarter fraction, 2 k-3 = oneeighth
fraction, …, 2 k-p = 1/ 2 p fraction
• Add p columns to the basic design; select p independent
generators
• Important to select generators so as to maximize
resolution, see Table 8.14
• Projection – a design of resolution R contains full factorials
in any R – 1 of the factors
31

The Regular 2 k-p Design: Resolution
may not be Sufficient
• Minimum aberration designs
33

Regular Designs may not Always be the Best Choice
for Screening
• In regular designs the alias matrix consists of either 0, +1 or -1
entries
• That means that effects are completely confounded
• Unless the experimenter has some “process knowledge”, effects
cannot be separated without conducting additional experiments
– Fold-over
– Partial fold-over
– Optimal augmentation
34

Number of Orthogonal Designs versus
Number of Factors
Number of Factors
Number of Nonisomorphic Designs
6 27
7 55
8 80
9 87
10 78
11 58
12 36
13 18
14 10
15 5
35

Define Nonisomorphic
Two designs are nonisomorphic if you cannot get one
from the other by:
–Permuting rows
–Permuting columns
–Relabeling the level names
36

Could one of these other orthogonal designs
work better?
Jones and Montgomery 2010 discuss specific designs for 6 – 8
factors and provide recommendations.
We will start with a motivating example and then show you the
designs and how to construct them.
37

Main effects of A, B, C
and E are important
The AB + CE
interaction is important
How do we separate
these interactions?
Unless there is outside
information available,
we’ll need more data
39

The No-Confounding Design
40

Where Did the Data in this Experiment
• Simulated data
Come From?
• We chose the significant main effects A, B, C and E along
with the interaction CE.
• We selected the random component to have the same
standard deviation as the original data
• The result is data that represents closely the original
experiment if the no-confounding design had been run
41

Color Plot for the No-Confounding Design
The design is orthogonal
No two-factor interactions are aliased with each other
There is no complete confounding
42

Stepwise Fit
Response: Shrinkage
Stepwise Regression Control
Prob to Enter
Prob to Leave
Direction:Forward
Rules:
SSE
6659.4375
0.250
0.100
Combine
Current Estimates
DFE
15
LockEnteredParameter
Intercept
A
B
C
D
E
F
A*B
A*C
A*D
A*E
A*F
B*C
B*D
B*E
B*F
C*D
C*E
C*F
D*E
D*F
E*F
Step History
MSE
443.9625
Estimate
27.3125
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
RSquare
0.0000
nDF
1
1
1
1
1
1
1
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
RSquare Adj
0.0000
SS
0
770.0625
5076.563
3.16e-30
3.16e-30
28.89063
28.89063
6410.688
781.4608
885.625
819.0536
809.2569
5076.563
5087.961
5115.757
5125.554
13.78835
2577.449
690.0164
345.2905
2382.766
60.24101
"F Ratio"
0.000
1.831
44.900
0.000
0.000
0.061
0.061
103.086
0.532
0.614
0.561
0.553
12.829
12.951
13.256
13.366
0.008
2.526
0.462
0.219
2.229
0.037
Cp
.
AIC
98.49923
"Prob>F"
1.0000
0.1975
0.0000
1.0000
1.0000
0.8085
0.8085
0.0000
0.6691
0.6192
0.6509
0.6556
0.0005
0.0005
0.0004
0.0004
0.9989
0.1068
0.7138
0.8815
0.1374
0.9902
Step Parameter Action "Sig Prob" Seq SS RSquare Cp p
We can use
stepwise
regression model
fitting
All main effects
and two-factor
interactions are
candidate
variables for the
model
Because there is
no complete
confounding, all
interactions are
potential
candidates 43

Stepwise regression
selects the main
effects of A, B, C and
E along with the CE
interaction
The no-confounding
design correctly
identifies the model
without any ambiguity
and no need for
additional runs
44

No-Confounding Designs
• The 16-run minimum aberration resolution IV designs (6, 7,
and 8 factors) are among the most widely used designs in
practice
• It is possible to find no-confounding designs that are
superior to the standard minimum aberration resolution IV
designs in the sense that they offer a better chance of
detecting significant two-factor interactions
• These designs are constructed from the Hall matrices
45

Hall I 15 Factor Design
Run A B C D E F G H J K L M N P Q
1 -1 -1 1 -1 1 1 -1 -1 1 1 -1 1 -1 -1 1
2 1 -1 -1 -1 -1 1 1 -1 -1 1 1 1 1 -1 -1
3 -1 1 -1 -1 1 -1 1 -1 1 -1 1 1 -1 1 -1
4 1 1 1 -1 -1 -1 -1 -1 -1 -1 -1 1 1 1 1
5 -1 -1 1 1 -1 -1 1 -1 1 1 -1 -1 1 1 -1
6 1 -1 -1 1 1 -1 -1 -1 -1 1 1 -1 -1 1 1
7 -1 1 -1 1 -1 1 -1 -1 1 -1 1 -1 1 -1 1
8 1 1 1 1 1 1 1 -1 -1 -1 -1 -1 -1 -1 -1
9 -1 -1 1 -1 1 1 -1 1 -1 -1 1 -1 1 1 -1
10 1 -1 -1 -1 -1 1 1 1 1 -1 -1 -1 -1 1 1
11 -1 1 -1 -1 1 -1 1 1 -1 1 -1 -1 1 -1 1
12 1 1 1 -1 -1 -1 -1 1 1 1 1 -1 -1 -1 -1
13 -1 -1 1 1 -1 -1 1 1 -1 -1 1 1 -1 -1 1
14 1 -1 -1 1 1 -1 -1 1 1 -1 -1 1 1 -1 -1
15 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1
16 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
46

Hall II 15 Factor Design
Run A B C D E F G H J K L M N P Q
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
2 1 1 1 1 1 1 1 -1 -1 -1 -1 -1 -1 -1 -1
3 1 1 1 -1 -1 -1 -1 1 1 1 1 -1 -1 -1 -1
4 1 1 1 -1 -1 -1 -1 -1 -1 -1 -1 1 1 1 1
5 1 -1 -1 1 1 -1 -1 1 1 -1 -1 1 1 -1 -1
6 1 -1 -1 1 1 -1 -1 -1 -1 1 1 -1 -1 1 1
7 1 -1 -1 -1 -1 1 1 1 1 -1 -1 -1 -1 1 1
8 1 -1 -1 -1 -1 1 1 -1 -1 1 1 1 1 -1 -1
9 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1
10 -1 1 -1 1 -1 1 -1 -1 1 -1 1 -1 1 -1 1
11 -1 1 -1 -1 1 -1 1 1 -1 1 -1 -1 1 -1 1
12 -1 1 -1 -1 1 -1 1 -1 1 -1 1 1 -1 1 -1
13 -1 -1 1 1 -1 -1 1 1 -1 -1 1 -1 1 1 -1
14 -1 -1 1 1 -1 -1 1 -1 1 1 -1 1 -1 -1 1
15 -1 -1 1 -1 1 1 -1 1 -1 -1 1 1 -1 -1 1
16 -1 -1 1 -1 1 1 -1 -1 1 1 -1 -1 1 1 -1
47

Hall III 15 Factor Design
Run A B C D E F G H J K L M N P Q
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
2 1 1 1 1 1 1 1 -1 -1 -1 -1 -1 -1 -1 -1
3 1 1 1 -1 -1 -1 -1 1 1 1 1 -1 -1 -1 -1
4 1 1 1 -1 -1 -1 -1 -1 -1 -1 -1 1 1 1 1
5 1 -1 -1 1 1 -1 -1 1 1 -1 -1 1 1 -1 -1
6 1 -1 -1 1 1 -1 -1 -1 -1 1 1 -1 -1 1 1
7 1 -1 -1 -1 -1 1 1 1 1 -1 -1 -1 -1 1 1
8 1 -1 -1 -1 -1 1 1 -1 -1 1 1 1 1 -1 -1
9 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1
10 -1 1 -1 1 -1 1 -1 -1 1 -1 1 -1 1 -1 1
11 -1 1 -1 -1 1 -1 1 1 -1 -1 1 1 -1 -1 1
12 -1 1 -1 -1 1 -1 1 -1 1 1 -1 -1 1 1 -1
13 -1 -1 1 1 -1 -1 1 1 -1 -1 1 -1 1 1 -1
14 -1 -1 1 1 -1 -1 1 -1 1 1 -1 1 -1 -1 1
15 -1 -1 1 -1 1 1 -1 1 -1 1 -1 -1 1 -1 1
16 -1 -1 1 -1 1 1 -1 -1 1 -1 1 1 -1 1 -1
48

Hall IV 15 Factor Design
Run A B C D E F G H J K L M N P Q
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
2 1 1 1 1 1 1 1 -1 -1 -1 -1 -1 -1 -1 -1
3 1 1 1 -1 -1 -1 -1 1 1 1 1 -1 -1 -1 -1
4 1 1 1 -1 -1 -1 -1 -1 -1 -1 -1 1 1 1 1
5 1 -1 -1 1 1 -1 -1 1 1 -1 -1 1 1 -1 -1
6 1 -1 -1 1 1 -1 -1 -1 -1 1 1 -1 -1 1 1
7 1 -1 -1 -1 -1 1 1 1 1 -1 -1 -1 -1 1 1
8 1 -1 -1 -1 -1 1 1 -1 -1 1 1 1 1 -1 -1
9 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1
10 -1 1 -1 1 -1 -1 1 1 -1 -1 1 -1 1 -1 1
11 -1 1 -1 -1 1 1 -1 -1 1 -1 1 1 -1 -1 1
12 -1 1 -1 -1 1 -1 1 -1 1 1 -1 -1 1 1 -1
13 -1 -1 1 1 -1 1 -1 -1 1 -1 1 -1 1 1 -1
14 -1 -1 1 1 -1 -1 1 -1 1 1 -1 1 -1 -1 1
15 -1 -1 1 -1 1 1 -1 1 -1 1 -1 -1 1 -1 1
16 -1 -1 1 -1 1 -1 1 1 -1 -1 1 1 -1 1 -1
49

Hall V 15 Factor Design
Run A B C D E F G H J K L M N P Q
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
2 1 1 1 1 1 1 1 -1 -1 -1 -1 -1 -1 -1 -1
3 1 1 1 -1 -1 -1 -1 1 1 1 1 -1 -1 -1 -1
4 1 1 1 -1 -1 -1 -1 -1 -1 -1 -1 1 1 1 1
5 1 -1 -1 1 1 -1 -1 1 1 -1 -1 1 1 -1 -1
6 1 -1 -1 1 1 -1 -1 -1 -1 1 1 -1 -1 1 1
7 1 -1 -1 -1 -1 1 1 1 -1 1 -1 1 -1 1 -1
8 1 -1 -1 -1 -1 1 1 -1 1 -1 1 -1 1 -1 1
9 -1 1 -1 1 -1 1 -1 1 1 -1 -1 -1 -1 1 1
10 -1 1 -1 1 -1 1 -1 -1 -1 1 1 1 1 -1 -1
11 -1 1 -1 -1 1 -1 1 1 -1 -1 1 -1 1 1 -1
12 -1 1 -1 -1 1 -1 1 -1 1 1 -1 1 -1 -1 1
13 -1 -1 1 1 -1 -1 1 1 -1 1 -1 -1 1 -1 1
14 -1 -1 1 1 -1 -1 1 -1 1 -1 1 1 -1 1 -1
15 -1 -1 1 -1 1 1 -1 1 -1 -1 1 1 -1 -1 1
16 -1 -1 1 -1 1 1 -1 -1 1 1 -1 -1 1 1 -1
50

Constructing the Recommended 6 Factor
Design
Run A B C D E F G H J K L M N P Q
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
2 1 1 1 1 1 1 1 -1 -1 -1 -1 -1 -1 -1 -1
3 1 1 1 -1 -1 -1 -1 1 1 1 1 -1 -1 -1 -1
4 1 1 1 -1 -1 -1 -1 -1 -1 -1 -1 1 1 1 1
5 1 -1 -1 1 1 -1 -1 1 1 -1 -1 1 1 -1 -1
6 1 -1 -1 1 1 -1 -1 -1 -1 1 1 -1 -1 1 1
7 1 -1 -1 -1 -1 1 1 1 1 -1 -1 -1 -1 1 1
8 1 -1 -1 -1 -1 1 1 -1 -1 1 1 1 1 -1 -1
9 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1
10 -1 1 -1 1 -1 1 -1 -1 1 -1 1 -1 1 -1 1
11 -1 1 -1 -1 1 -1 1 1 -1 1 -1 -1 1 -1 1
12 -1 1 -1 -1 1 -1 1 -1 1 -1 1 1 -1 1 -1
13 -1 -1 1 1 -1 -1 1 1 -1 -1 1 -1 1 1 -1
14 -1 -1 1 1 -1 -1 1 -1 1 1 -1 1 -1 -1 1
15 -1 -1 1 -1 1 1 -1 1 -1 -1 1 1 -1 -1 1
16 -1 -1 1 -1 1 1 -1 -1 1 1 -1 -1 1 1 -1
Hall II – Columns D, E, H, K, M, Q
51

Recommended Nonregular 6 Factor Design
Run A B C D E F
1 1 1 1 1 1 1
2 1 1 -1 -1 -1 -1
3 -1 -1 1 1 -1 -1
4 -1 -1 -1 -1 1 1
5 1 1 1 -1 1 -1
6 1 1 -1 1 -1 1
7 -1 -1 1 -1 -1 1
8 -1 -1 -1 1 1 -1
9 1 -1 1 1 1 -1
10 1 -1 -1 -1 -1 1
11 -1 1 1 1 -1 1
12 -1 1 -1 -1 1 -1
13 1 -1 1 -1 -1 -1
14 1 -1 -1 1 1 1
15 -1 1 1 -1 1 1
16 -1 1 -1 1 -1 -1
52

Constructing Recommended Design
Using the “Generator” Approach
E = 0.5AC+0.5AD+0.5BC-0.5BD F = -0.5AC+0.5AD+0.5BC+0.5BD
Run A B C D
Run
E
Run
F
1 1 1 1 1
1 1
1 1
2 1 1 1 -1
2 1
2 -1
3 1 1 -1 1
3 -1
3 1
4 1 1 -1 -1
4 -1
4 -1
5 1 -1 1 1
5 1
5 -1
6 1 -1 1 -1
6 -1
6 -1
7 1 -1 -1 1
7 1
7 1
8 1 -1 -1 -1
8 -1
8 1
9 -1 1 1 1
9 -1
9 1
10 -1 1 1 -1
10 1
10 1
11 -1 1 -1 1
11 -1
11 -1
12 -1 1 -1 -1
12 1
12 -1
13 -1 -1 1 1
13 -1
13 -1
14 -1 -1 1 -1
14 -1
14 1
15 -1 -1 -1 1
15 1
15 -1
16 -1 -1 -1 -1
16 1
16 1
53

Color Plot for the Standard Minimum
Aberration Resolution IV 7-Factor Design
54

Constructing the Recommended 7 Factor
Design
Run A B C D E F G H J K L M N P Q
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
2 1 1 1 1 1 1 1 -1 -1 -1 -1 -1 -1 -1 -1
3 1 1 1 -1 -1 -1 -1 1 1 1 1 -1 -1 -1 -1
4 1 1 1 -1 -1 -1 -1 -1 -1 -1 -1 1 1 1 1
5 1 -1 -1 1 1 -1 -1 1 1 -1 -1 1 1 -1 -1
6 1 -1 -1 1 1 -1 -1 -1 -1 1 1 -1 -1 1 1
7 1 -1 -1 -1 -1 1 1 1 1 -1 -1 -1 -1 1 1
8 1 -1 -1 -1 -1 1 1 -1 -1 1 1 1 1 -1 -1
9 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1
10 -1 1 -1 1 -1 1 -1 -1 1 -1 1 -1 1 -1 1
11 -1 1 -1 -1 1 -1 1 1 -1 -1 1 1 -1 -1 1
12 -1 1 -1 -1 1 -1 1 -1 1 1 -1 -1 1 1 -1
13 -1 -1 1 1 -1 -1 1 1 -1 -1 1 -1 1 1 -1
14 -1 -1 1 1 -1 -1 1 -1 1 1 -1 1 -1 -1 1
15 -1 -1 1 -1 1 1 -1 1 -1 1 -1 -1 1 -1 1
16 -1 -1 1 -1 1 1 -1 -1 1 -1 1 1 -1 1 -1
Hall III – Columns A, B, D, H, J, M, Q
55

Recommended Nonregular 7 Factor Design
Run A B C D E F G
1 1 1 1 1 1 1 1
2 1 1 1 -1 -1 -1 -1
3 1 1 -1 1 1 -1 -1
4 1 1 -1 -1 -1 1 1
5 1 -1 1 1 -1 1 -1
6 1 -1 1 -1 1 -1 1
7 1 -1 -1 1 -1 -1 1
8 1 -1 -1 -1 1 1 -1
9 -1 1 1 1 1 1 -1
10 -1 1 1 -1 -1 -1 1
11 -1 1 -1 1 -1 1 1
12 -1 1 -1 -1 1 -1 -1
13 -1 -1 1 1 -1 -1 -1
14 -1 -1 1 -1 1 1 1
15 -1 -1 -1 1 1 -1 1
16 -1 -1 -1 -1 -1 1 -1
56

Comparison of Color Plots for the Standard
and No-confounding Designs
The recommended design is orthogonal and does
not have any complete confounding of effects
57

Color Plot for the Standard Minimum
Aberration Resolution IV 8-Factor Design
58

Constructing the Recommended 8 Factor Design
Run A B C D E F G H J K L M N P Q
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
2 1 1 1 1 1 1 1 -1 -1 -1 -1 -1 -1 -1 -1
3 1 1 1 -1 -1 -1 -1 1 1 1 1 -1 -1 -1 -1
4 1 1 1 -1 -1 -1 -1 -1 -1 -1 -1 1 1 1 1
5 1 -1 -1 1 1 -1 -1 1 1 -1 -1 1 1 -1 -1
6 1 -1 -1 1 1 -1 -1 -1 -1 1 1 -1 -1 1 1
7 1 -1 -1 -1 -1 1 1 1 1 -1 -1 -1 -1 1 1
8 1 -1 -1 -1 -1 1 1 -1 -1 1 1 1 1 -1 -1
9 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1
10 -1 1 -1 1 -1 -1 1 1 -1 -1 1 -1 1 -1 1
11 -1 1 -1 -1 1 1 -1 -1 1 -1 1 1 -1 -1 1
12 -1 1 -1 -1 1 -1 1 -1 1 1 -1 -1 1 1 -1
13 -1 -1 1 1 -1 1 -1 -1 1 -1 1 -1 1 1 -1
14 -1 -1 1 1 -1 -1 1 -1 1 1 -1 1 -1 -1 1
15 -1 -1 1 -1 1 1 -1 1 -1 1 -1 -1 1 -1 1
16 -1 -1 1 -1 1 -1 1 1 -1 -1 1 1 -1 1 -1
Hall IV – Columns A, B, D, F, H, J, M, P
59

Recommended Nonregular 8 Factor Design
Run A B C D E F G H
1 1 1 1 1 1 1 1 1
2 1 1 1 1 -1 -1 -1 -1
3 1 1 -1 -1 1 1 -1 -1
4 1 1 -1 -1 -1 -1 1 1
5 1 -1 1 -1 1 -1 1 -1
6 1 -1 1 -1 -1 1 -1 1
7 1 -1 -1 1 1 -1 -1 1
8 1 -1 -1 1 -1 1 1 -1
9 -1 1 1 1 1 1 1 1
10 -1 1 1 -1 1 -1 -1 -1
11 -1 1 -1 1 -1 -1 1 -1
12 -1 1 -1 -1 -1 1 -1 1
13 -1 -1 1 1 -1 -1 -1 1
14 -1 -1 1 -1 -1 1 1 -1
15 -1 -1 -1 1 1 1 -1 -1
16 -1 -1 -1 -1 1 -1 1 1 60

Comparison of Color Plots for the Standard
and No-confounding Designs
The recommended design is orthogonal and does
not have any complete confounding of effects
61

Alternatives to Resolution III Designs
The regular resolution III designs with from 9 to 15 factors in 16 runs are
used frequently in practice
These designs completely confound some interactions with main effects
For example, in the minimum aberration nine factor case, 12 two-factor
interactions are aliased with main effects and 24 two-factor interactions
are confounded in groups with other two-factor interactions
Follow-up experiments are often necessary, and the best augmentation
approach may not be obvious.
Nonregular designs with no pure confounding of main effects and twofactor
interactions are useful alternatives.
We provide a collection of these designs.
62

Recommended 9 Factor Design
Run A B C D E F G H J
1 -1 -1 -1 -1 -1 -1 1 -1 1
2 -1 -1 -1 1 -1 1 -1 1 -1
3 -1 -1 1 -1 1 1 1 1 -1
4 -1 -1 1 1 1 -1 -1 -1 1
5 -1 1 -1 -1 1 1 -1 1 1
6 -1 1 -1 1 1 -1 1 -1 -1
7 -1 1 1 -1 -1 -1 -1 1 -1
8 -1 1 1 1 -1 1 1 -1 1
9 1 -1 -1 -1 1 -1 -1 -1 -1
10 1 -1 -1 1 1 1 1 1 1
11 1 -1 1 -1 -1 1 -1 -1 1
12 1 -1 1 1 -1 -1 1 1 -1
13 1 1 -1 -1 -1 1 1 -1 -1
14 1 1 -1 1 -1 -1 -1 1 1
15 1 1 1 -1 1 -1 1 1 1
16 1 1 1 1 1 1 -1 -1 -1
Correlation of Main Effects and Two-Factor Interactions

Recommended 10 Factor Design
Run A B C D E F G H J K
1 -1 -1 -1 -1 1 -1 -1 1 -1 1
2 -1 -1 -1 1 1 1 -1 -1 1 1
3 -1 -1 1 -1 -1 1 1 1 1 1
4 -1 -1 1 -1 1 -1 1 -1 1 -1
5 -1 1 -1 1 -1 1 1 1 -1 1
6 -1 1 -1 1 1 -1 1 -1 -1 -1
7 -1 1 1 -1 -1 -1 -1 1 -1 -1
8 -1 1 1 1 -1 1 -1 -1 1 -1
9 1 -1 -1 -1 -1 1 1 -1 -1 -1
10 1 -1 -1 1 -1 -1 -1 1 1 -1
11 1 -1 1 1 -1 -1 1 -1 -1 1
12 1 -1 1 1 1 1 -1 1 -1 -1
13 1 1 -1 -1 -1 -1 -1 -1 1 1
14 1 1 -1 -1 1 1 1 1 1 -1
15 1 1 1 -1 1 1 -1 -1 -1 1
16 1 1 1 1 1 -1 1 1 1 1
Correlation of Main Effects and Two-Factor Interactions
64

Recommended 11 Factor Design
Run A B C D E F G H J K L
1 -1 -1 -1 1 1 -1 -1 -1 1 -1 1
2 -1 -1 1 -1 -1 -1 1 -1 1 -1 -1
3 -1 -1 1 -1 1 1 -1 1 -1 -1 -1
4 -1 -1 1 1 -1 1 1 1 -1 1 1
5 -1 1 -1 -1 -1 -1 -1 -1 -1 1 1
6 -1 1 -1 1 -1 1 1 -1 -1 -1 -1
7 -1 1 -1 1 1 1 -1 1 1 1 -1
8 -1 1 1 -1 1 -1 1 1 1 1 1
9 1 -1 -1 -1 -1 1 -1 1 1 -1 1
10 1 -1 -1 -1 1 1 1 -1 -1 1 1
11 1 -1 -1 1 -1 -1 1 1 1 1 -1
12 1 -1 1 1 1 -1 -1 -1 -1 1 -1
13 1 1 -1 -1 1 -1 1 1 -1 -1 -1
14 1 1 1 -1 -1 1 -1 -1 1 1 -1
15 1 1 1 1 -1 -1 -1 1 -1 -1 1
16 1 1 1 1 1 1 1 -1 1 -1 1
Correlation of Main Effects and Two-Factor Interactions
65

Recommended 12 Factor Design
Run A B C D E F G H J K L M
1 -1 -1 -1 -1 1 -1 -1 1 1 -1 1 1
2 -1 -1 -1 1 -1 1 1 1 -1 -1 1 -1
3 -1 -1 1 -1 -1 -1 1 -1 1 1 -1 1
4 -1 -1 1 1 1 1 -1 -1 -1 1 -1 -1
5 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1
6 -1 1 -1 1 1 1 1 -1 1 1 1 1
7 -1 1 1 -1 -1 1 -1 1 1 -1 -1 -1
8 -1 1 1 -1 1 -1 1 1 -1 1 1 -1
9 1 -1 -1 -1 -1 1 -1 -1 1 1 1 -1
10 1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1
11 1 -1 1 1 -1 -1 -1 1 -1 1 1 1
12 1 -1 1 1 1 1 1 1 1 -1 -1 1
13 1 1 -1 -1 -1 1 1 1 -1 1 -1 1
14 1 1 -1 1 1 -1 -1 1 1 1 -1 -1
15 1 1 1 -1 1 1 -1 -1 -1 -1 1 1
16 1 1 1 1 -1 -1 1 -1 1 -1 1 -1
Correlation of Main Effects and Two-Factor Interactions
66

Recommended 13 Factor Design
Run A B C D E F G H J K L M N
1 -1 -1 -1 1 1 -1 -1 1 -1 1 1 -1 1
2 -1 -1 1 -1 -1 -1 -1 -1 1 1 -1 -1 1
3 -1 -1 1 -1 1 1 1 1 1 -1 1 -1 -1
4 -1 -1 1 1 -1 1 1 1 -1 1 -1 1 -1
5 -1 1 -1 -1 -1 1 -1 -1 -1 -1 1 -1 -1
6 -1 1 -1 -1 1 1 1 -1 -1 1 -1 1 1
7 -1 1 -1 1 -1 -1 1 1 1 -1 1 1 1
8 -1 1 1 1 1 -1 -1 -1 1 -1 -1 1 -1
9 1 -1 -1 -1 -1 -1 1 -1 1 1 1 1 -1
10 1 -1 -1 -1 1 -1 -1 1 -1 -1 -1 1 -1
11 1 -1 -1 1 1 1 1 -1 1 -1 -1 -1 1
12 1 -1 1 1 -1 1 -1 -1 -1 -1 1 1 1
13 1 1 -1 1 -1 1 -1 1 1 1 -1 -1 -1
14 1 1 1 -1 -1 -1 1 1 -1 -1 -1 -1 1
15 1 1 1 -1 1 1 -1 1 1 1 1 1 1
16 1 1 1 1 1 -1 1 -1 -1 1 1 -1 -1
Correlation of Main Effects and Two-Factor Interactions
67

Recommended 14 Factor Design
Run A B C D E F G H J K L M N P
1 -1 -1 -1 -1 1 -1 1 1 -1 1 1 -1 -1 1
2 -1 -1 -1 1 -1 -1 1 -1 1 1 -1 1 1 -1
3 -1 -1 1 -1 -1 1 -1 1 1 1 -1 -1 1 1
4 -1 -1 1 1 1 1 1 -1 -1 -1 1 -1 1 -1
5 -1 1 -1 -1 -1 1 1 -1 1 -1 1 1 -1 1
6 -1 1 -1 1 1 1 -1 1 -1 1 -1 1 -1 -1
7 -1 1 1 -1 -1 -1 -1 1 -1 -1 1 1 1 -1
8 -1 1 1 1 1 -1 -1 -1 1 -1 -1 -1 -1 1
9 1 -1 -1 -1 1 1 -1 -1 -1 -1 -1 1 1 1
10 1 -1 -1 1 -1 1 -1 1 1 -1 1 -1 -1 -1
11 1 -1 1 -1 1 -1 -1 -1 1 1 1 1 -1 -1
12 1 -1 1 1 -1 -1 1 1 -1 -1 -1 1 -1 1
13 1 1 -1 -1 1 -1 1 1 1 -1 -1 -1 1 -1
14 1 1 -1 1 -1 -1 -1 -1 -1 1 1 -1 1 1
15 1 1 1 -1 -1 1 1 -1 -1 1 -1 -1 -1 -1
16 1 1 1 1 1 1 1 1 1 1 1 1 1 1
Correlation of Main Effects and Two-Factor Interactions
68

Introducing Definitive Screening
Jones and Nachtsheim (2011) provide a class of screening
designs for 5 factors and up.
These are three level designs for numeric factors.
We will discuss their properties, show how to construct them
and give an example.
69

Motivation: Problems with Standard Screening Designs
Resolution III designs confound main effects and two-factor interactions.
Plackett-Burman designs have “complex aliasing of the main effects by twofactor
interactions.
Resolution IV designs confound two-factor interactions with each other, so if
one is active, you usually need further runs to resolve the active effects.
Center runs give an overall measure of curvature but you do not know which
factor(s) are causing the curvature.
Even the nonregular orthogonal designs we just discussed have aliasing of
+0.5 or -0.5 of some main effects by some two-factor interactions. Plus they
have no way of resolving quadratic effects.

Screening Conundrum – Two Models
The full model containing both 6 first-order and 15 secondorder
terms is:
But n = 12, so we can only fit the intercept and the main
effects:
Standard result: some main effects estimates are biased:
where the “alias” matrix is:
71

Alias Matrix of Plackett-Burman design
PB (non-regular) design has “complex aliasing”

If only there were another six factor 12 run design with
this alias matrix:

Screening Design – Wish List
1. Orthogonal main effects.
2. Main effects uncorrelated with two-factor interactions and
quadratic effects.
3. Estimable quadratic effects – three-level design.
4. Small number of runs – order of the number of factors.
5. Good projective properties.

Design Structure
1. Scaled values of each element of the design are either +1, -1 or 0.
2. Each even numbered row is a mirror image of the previous row.
3. For the kth column, rows 2k and 2k−1 contain zeros.
4. The last row contains all zero elements.
Run A B C D E F
1 0 1 -1 -1 -1 -1
2 0 -1 1 1 1 1
3 1 0 -1 1 1 -1
4 -1 0 1 -1 -1 1
5 -1 -1 0 1 -1 -1
6 1 1 0 -1 1 1
7 -1 1 1 0 1 -1
8 1 -1 -1 0 -1 1
9 1 -1 1 -1 0 -1
10 -1 1 -1 1 0 1
11 1 1 1 1 -1 0
12 -1 -1 -1 -1 1 0
13 0 0 0 0 0 0

How did we find this design? – we used an
optimization algorithm

Problem: Finding orthogonal main effects plans.
Algorithmic approach gave orthogonal main effects
plans for:
6 factors
8 factors
10 factors
but not 12 factors.

Orthogonal Main Effects Design Construction
Conference matrix
From Wikipedia, the free encyclopedia
In mathematics, a conference matrix (also called a C-matrix) is a square
matrix C with 0 on the diagonal and +1 and −1 off the diagonal, such that
C T C is a multiple of the identity matrix I. Thus, if the matrix has order n,
C T C = (n−1)I.
Conference matrices first arose in connection with a problem in
telephony. [3] They were first described by Vitold Belevitch who also gave
them their name. Belevitch was interested in constructing ideal telephone
conference networks from ideal transformers and discovered that such
networks were represented by conference matrices, hence the name. [4]
Other applications are in statistics, [5] and another is in elliptic geometry. [6]

From Wikipedia
article on
Conference
Matrices
Conference Matrices &Telephony

Conference Matrix of Order 6

Statistical Properties
1. Orthogonal for the main effects.
2. The number of required runs is only one more than twice the number of
factors. ***
3. Unlike resolution III designs, main effects are independent of two-factor
interactions.
4. Unlike resolution IV designs, two-factor interactions are not completely
confounded with other two-factor interactions, although they may be
correlated
5. Unlike resolution III, IV and V designs with added center points, all
quadratic effects are estimable in models comprised of any number of
linear and quadratic main effects terms.
6. Quadratic effects are orthogonal to main effects and not completely
confounded (though correlated) with interaction effects.
7. If there are more than six factors, the designs are capable of efficiently
estimating all possible full quadratic models involving three or fewer
factors

Example 1
6 factors and 13 runs.
Run A B C D E F
1 0 1 -1 -1 -1 -1
2 0 -1 1 1 1 1
3 1 0 -1 1 1 -1
4 -1 0 1 -1 -1 1
5 -1 -1 0 1 -1 -1
6 1 1 0 -1 1 1
7 -1 1 1 0 1 -1
8 1 -1 -1 0 -1 1
9 1 -1 1 -1 0 -1
10 -1 1 -1 1 0 1
11 1 1 1 1 -1 0
12 -1 -1 -1 -1 1 0
13 0 0 0 0 0 0
Note that even numbered row mirrors the previous row.
D-efficiency is 85.5% and the design is orthogonal for the main effects.

Alias Matrices
The D-optimal design with one added center point has substantial aliasing of
each main effect with a number of two-factor interactions.
For our design there is no aliasing between main effects and two-factor interactions.

Column Correlations
0.25
0.5
0.4655
0.1333

Example 2
12 factors – 25 runs.

Column Correlations
Note the zero
correlations
among main
effects

Example from JQT paper
87

Where did the data come from?
We simulated it using the model below…
88

Stepwise Results
89

Recapitulation – Definitive Screening Design
1. Orthogonal main effects plans.
2. Two-factor interactions are uncorrelated with main effects.
3. Quadratic effects are uncorrelated with main effects.
4. All quadratic effects are estimable.
5. The number of runs is only one more than twice the number of
factors.
6. For six factors or more, the designs can estimate all possible full
quadratic models involving three or fewer factors

References
1. Box, G. E. P., and Hunter, J. S. (1961), “The 2k−p Fractional Factorial Designs,”Technometrics,
3, 449–458.
2. Goethals, J. and Seidel, J. (1967). ”Orthogonal matrices with zero diagonal”. Canadian Journal
of Mathematics, 19, pp. 1001–1010.
3. Hall, M. Jr. (1961). Hadamard matrix of order 16. Jet Propulsion Laboratory Research Summary,
1, 21–26.
4. Jones, B. A. and Montgomery, D. C. (2010) “Alternatives to Resolution IV Screening Designs in
16 Runs.”, International Journal of Experimental Design and Process Optimization, 2010; Vol. 1
No. 4: 285-295.
5. Jones, B. and Nachtsheim, C. J. (2011) “Efficient Designs with Minimal Aliasing” Technometrics,
53. 62-71.
6. Jones, B and Nachtsheim, C. (2011) “A Class of Three-Level Designs for Definitive Screening in
the Presence of Second-Order Effects” Journal of Quality Technology, 43. 1-15.
7. Plackett, R. L., and Burman, J. P. (1946), “The Design of Optimum Multifactor Experiments,”
Biometrika, 33, 305–325.
8. Montgomery, D.C. (2012) “Design and Analysis of Experiments” 8 th Edition, Wiley Hoboken,
New Jersey.
9. Sun, D. X., Li, W., and Ye, K. Q. (2002), “An Algorithm for Sequentially Constructing Non-
Isomorphic Orthogonal Designs and Its Applications,” Technical Report SUNYSB-AMS-02-13,
State University of New York at Stony Brook, Dept. of Applied Mathematics and Statistics.