Presentation: A New DOE Paradigm - JMP
Presentation: A New DOE Paradigm - JMP
Presentation: A New DOE Paradigm - JMP
Create successful ePaper yourself
Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.
A <strong>New</strong> <strong>DOE</strong> <strong>Paradigm</strong><br />
• April 12, 2012<br />
• Bradley Jones, SAS<br />
Institute<br />
• Doug Montgomery,<br />
Department of Industrial<br />
Engineering, Arizona<br />
State University<br />
1
<strong>New</strong>! Improved!<br />
2
Design Optimality<br />
• One of the truly great paradigm shifts in DOX<br />
• Can create custom designs for almost any situation<br />
• Modern software makes this easy for at least some<br />
optimality criteria<br />
• What optimality criteria should we use?<br />
3
Three Criteria:<br />
The D-criterion: M =<br />
X' X<br />
N<br />
Maximize the determinant of M<br />
D<br />
eff<br />
<br />
| X´<br />
X | <br />
<br />
Max[|<br />
´ |]<br />
<br />
X X <br />
1/ p<br />
The G-criterion: Minimize the maximum SPV<br />
over R<br />
NVar[ yˆ(<br />
m<br />
m<br />
SPV<br />
x )] ( ) <br />
<br />
Nx'<br />
X' X 1<br />
x<br />
2<br />
<br />
G<br />
eff<br />
<br />
p<br />
max ( SPV )<br />
xR<br />
The I-criterion: Minimize the average prediction variance over R<br />
I <br />
1 NVar[ yˆ<br />
( x)]<br />
d<br />
2<br />
A<br />
<br />
x<br />
<br />
R<br />
I eff<br />
Min(<br />
APV ( D))<br />
<br />
APV ( D)<br />
5
Which Criterion should I Use?<br />
• First-order models, first-order models with interaction<br />
– Objective is usually factor screening<br />
– Parameter estimation is key<br />
– Use the D-criterion<br />
• Second-order models, mixture models<br />
– Response estimation is usually of primary importance<br />
– Use the G or I criterion<br />
6
A problem with a constrained design region –<br />
amount of adhesive and cure temperature<br />
Page 391 & 392
How would we design an experiment for this<br />
problem?<br />
• “Force” a standard design into the experimental region<br />
– May lead to a case of the “square peg and the round hole”<br />
• Generate a unique design just for this particular situation<br />
– Need criteria for constructing the design<br />
– Computer implementation essential
<strong>JMP</strong> Default RSM Design
Design Comparison<br />
2 reps<br />
D-optimal Design Points<br />
I-optimal Design Points
Design Comparison cont.<br />
D-optimal Design Variance Profiles<br />
I-optimal Design Variance Profile
Let’s back up<br />
What we just showed was an experiment for optimization.<br />
But, usually designed experimentation starts with screening.<br />
We will first review some basics of screening.<br />
Then we will show you some really new stuff.<br />
12
Why do Regular Fractional Factorial<br />
Designs Work for Factor Screening?<br />
• The sparsity of effects principle<br />
– There may be lots of factors, but few are important<br />
– System is dominated by main effects, low-order interactions<br />
• The projection property<br />
– Every fractional factorial contains full factorials in fewer factors<br />
• Sequential experimentation<br />
– Can add runs to a fractional factorial to resolve difficulties (or<br />
ambiguities) in interpretation<br />
13
The Regular One-Half Fraction of the 2 k<br />
• DCM, Design & Analysis of Experiments, 8 th Edition, Wiley 2012<br />
• Notation: because the design has 2 k /2 runs, it’s referred to as a 2 k-1<br />
• Consider a really simple case, the 2 3-1<br />
• Note that I =ABC<br />
14
The Regular One-Half Fraction of the 2 3<br />
For the principal fraction, notice that the contrast for estimating the main<br />
effect A is exactly the same as the contrast used for estimating the BC<br />
interaction.<br />
This phenomena is called aliasing and it occurs in all fractional designs<br />
Aliases can be found directly from the columns in the table of + and - signs<br />
15
Aliasing in the One-Half Fraction of the 2 3<br />
A = BC, B = AC, C = AB (or me = 2fi)<br />
Aliases can be found from the defining relation I = ABC<br />
by multiplication:<br />
AI = A(ABC) = A 2 BC = BC<br />
BI =B(ABC) = AC<br />
CI = C(ABC) = AB<br />
Textbook notation for aliased effects:<br />
[ A] A BC, [ B] B AC, [ C]<br />
C AB<br />
16
The Alternate Fraction of the 2 3-1<br />
• I = -ABC is the defining relation<br />
• Implies slightly different aliases: A = -BC, B= -AC,<br />
and C = -AB<br />
• Both designs belong to the same family, defined by<br />
I ABC<br />
• Suppose that after running the principal fraction, the<br />
alternate fraction was also run<br />
• The two groups of runs can be combined to form a full<br />
factorial – an example of sequential experimentation<br />
17
• Resolution III Designs:<br />
– me = 2fi<br />
– example<br />
• Resolution IV Designs:<br />
– 2fi = 2fi<br />
– example<br />
• Resolution V Designs:<br />
– 2fi = 3fi<br />
– example<br />
Design Resolution<br />
31<br />
2 III<br />
41<br />
2 IV<br />
51<br />
2 V<br />
18
Construction of a Regular One-half Fraction<br />
The basic design; the design generator<br />
19
Projection of Fractional Factorials<br />
Every fractional factorial<br />
contains full factorials in<br />
fewer factors<br />
The “flashlight” analogy<br />
A one-half fraction will<br />
project into a full<br />
factorial in any k – 1 of<br />
the original factors<br />
20
Example 8.1<br />
21
Example 8.1<br />
Interpretation of results often relies on making some assumptions<br />
Ockham’s razor<br />
Confirmation experiments can be important<br />
Adding more runs to resolve ambiguities<br />
22
Confirmation experiment for this example:<br />
Use the model to predict the response at a test combination of interest<br />
in the design space – not one of the points in the current design.<br />
Run this test combination – then compare predicted and observed.<br />
For Example 8.1, consider the point +, +, -, +. The predicted response<br />
is<br />
Actual response is 104.<br />
23
Suppose we believe in the model:<br />
y = X 1<br />
b 1<br />
+ e<br />
But the true model is:<br />
y = X 1<br />
b 1<br />
+ X 2<br />
b 2<br />
+ e<br />
Then, the Alias matrix is:<br />
A = (X t 1<br />
X 1<br />
) -1 X t 1<br />
X 2<br />
The model for our specific example on the next slide is:<br />
y = b 0<br />
+x 1<br />
b 1<br />
+ x 2<br />
b 2<br />
+ x 3<br />
b 3<br />
+ e<br />
24
Main effects aliased<br />
with the 2fis<br />
26
The Alias Matrix:<br />
Columns represent the additional parameters in the full model<br />
Rows represent<br />
the parameters in<br />
the model that<br />
you fit<br />
Effect A*B A*C B*C<br />
Intercept 0 0 0<br />
A 0 0 1<br />
B 0 1 0<br />
C 1 0 0<br />
27
The Regular One-Quarter Fraction of the 2 k 28
The Regular One-Quarter Fraction of the 2 6-2<br />
Complete defining relation: I = ABCE = BCDF = ADEF<br />
This is a regular design 29
The Regular 2 k-p Fractional<br />
Factorial Design<br />
• 2 k-1 = one-half fraction, 2 k-2 = one-quarter fraction, 2 k-3 = oneeighth<br />
fraction, …, 2 k-p = 1/ 2 p fraction<br />
• Add p columns to the basic design; select p independent<br />
generators<br />
• Important to select generators so as to maximize<br />
resolution, see Table 8.14<br />
• Projection – a design of resolution R contains full factorials<br />
in any R – 1 of the factors<br />
31
The Regular 2 k-p Design: Resolution<br />
may not be Sufficient<br />
• Minimum aberration designs<br />
33
Regular Designs may not Always be the Best Choice<br />
for Screening<br />
• In regular designs the alias matrix consists of either 0, +1 or -1<br />
entries<br />
• That means that effects are completely confounded<br />
• Unless the experimenter has some “process knowledge”, effects<br />
cannot be separated without conducting additional experiments<br />
– Fold-over<br />
– Partial fold-over<br />
– Optimal augmentation<br />
34
Number of Orthogonal Designs versus<br />
Number of Factors<br />
Number of Factors<br />
Number of Nonisomorphic Designs<br />
6 27<br />
7 55<br />
8 80<br />
9 87<br />
10 78<br />
11 58<br />
12 36<br />
13 18<br />
14 10<br />
15 5<br />
35
Define Nonisomorphic<br />
Two designs are nonisomorphic if you cannot get one<br />
from the other by:<br />
–Permuting rows<br />
–Permuting columns<br />
–Relabeling the level names<br />
36
Could one of these other orthogonal designs<br />
work better?<br />
Jones and Montgomery 2010 discuss specific designs for 6 – 8<br />
factors and provide recommendations.<br />
We will start with a motivating example and then show you the<br />
designs and how to construct them.<br />
37
Main effects of A, B, C<br />
and E are important<br />
The AB + CE<br />
interaction is important<br />
How do we separate<br />
these interactions?<br />
Unless there is outside<br />
information available,<br />
we’ll need more data<br />
39
The No-Confounding Design<br />
40
Where Did the Data in this Experiment<br />
• Simulated data<br />
Come From?<br />
• We chose the significant main effects A, B, C and E along<br />
with the interaction CE.<br />
• We selected the random component to have the same<br />
standard deviation as the original data<br />
• The result is data that represents closely the original<br />
experiment if the no-confounding design had been run<br />
41
Color Plot for the No-Confounding Design<br />
The design is orthogonal<br />
No two-factor interactions are aliased with each other<br />
There is no complete confounding<br />
42
Stepwise Fit<br />
Response: Shrinkage<br />
Stepwise Regression Control<br />
Prob to Enter<br />
Prob to Leave<br />
Direction:Forward<br />
Rules:<br />
SSE<br />
6659.4375<br />
0.250<br />
0.100<br />
Combine<br />
Current Estimates<br />
DFE<br />
15<br />
LockEnteredParameter<br />
Intercept<br />
A<br />
B<br />
C<br />
D<br />
E<br />
F<br />
A*B<br />
A*C<br />
A*D<br />
A*E<br />
A*F<br />
B*C<br />
B*D<br />
B*E<br />
B*F<br />
C*D<br />
C*E<br />
C*F<br />
D*E<br />
D*F<br />
E*F<br />
Step History<br />
MSE<br />
443.9625<br />
Estimate<br />
27.3125<br />
0<br />
0<br />
0<br />
0<br />
0<br />
0<br />
0<br />
0<br />
0<br />
0<br />
0<br />
0<br />
0<br />
0<br />
0<br />
0<br />
0<br />
0<br />
0<br />
0<br />
0<br />
RSquare<br />
0.0000<br />
nDF<br />
1<br />
1<br />
1<br />
1<br />
1<br />
1<br />
1<br />
3<br />
3<br />
3<br />
3<br />
3<br />
3<br />
3<br />
3<br />
3<br />
3<br />
3<br />
3<br />
3<br />
3<br />
3<br />
RSquare Adj<br />
0.0000<br />
SS<br />
0<br />
770.0625<br />
5076.563<br />
3.16e-30<br />
3.16e-30<br />
28.89063<br />
28.89063<br />
6410.688<br />
781.4608<br />
885.625<br />
819.0536<br />
809.2569<br />
5076.563<br />
5087.961<br />
5115.757<br />
5125.554<br />
13.78835<br />
2577.449<br />
690.0164<br />
345.2905<br />
2382.766<br />
60.24101<br />
"F Ratio"<br />
0.000<br />
1.831<br />
44.900<br />
0.000<br />
0.000<br />
0.061<br />
0.061<br />
103.086<br />
0.532<br />
0.614<br />
0.561<br />
0.553<br />
12.829<br />
12.951<br />
13.256<br />
13.366<br />
0.008<br />
2.526<br />
0.462<br />
0.219<br />
2.229<br />
0.037<br />
Cp<br />
.<br />
AIC<br />
98.49923<br />
"Prob>F"<br />
1.0000<br />
0.1975<br />
0.0000<br />
1.0000<br />
1.0000<br />
0.8085<br />
0.8085<br />
0.0000<br />
0.6691<br />
0.6192<br />
0.6509<br />
0.6556<br />
0.0005<br />
0.0005<br />
0.0004<br />
0.0004<br />
0.9989<br />
0.1068<br />
0.7138<br />
0.8815<br />
0.1374<br />
0.9902<br />
Step Parameter Action "Sig Prob" Seq SS RSquare Cp p<br />
We can use<br />
stepwise<br />
regression model<br />
fitting<br />
All main effects<br />
and two-factor<br />
interactions are<br />
candidate<br />
variables for the<br />
model<br />
Because there is<br />
no complete<br />
confounding, all<br />
interactions are<br />
potential<br />
candidates 43
Stepwise regression<br />
selects the main<br />
effects of A, B, C and<br />
E along with the CE<br />
interaction<br />
The no-confounding<br />
design correctly<br />
identifies the model<br />
without any ambiguity<br />
and no need for<br />
additional runs<br />
44
No-Confounding Designs<br />
• The 16-run minimum aberration resolution IV designs (6, 7,<br />
and 8 factors) are among the most widely used designs in<br />
practice<br />
• It is possible to find no-confounding designs that are<br />
superior to the standard minimum aberration resolution IV<br />
designs in the sense that they offer a better chance of<br />
detecting significant two-factor interactions<br />
• These designs are constructed from the Hall matrices<br />
45
Hall I 15 Factor Design<br />
Run A B C D E F G H J K L M N P Q<br />
1 -1 -1 1 -1 1 1 -1 -1 1 1 -1 1 -1 -1 1<br />
2 1 -1 -1 -1 -1 1 1 -1 -1 1 1 1 1 -1 -1<br />
3 -1 1 -1 -1 1 -1 1 -1 1 -1 1 1 -1 1 -1<br />
4 1 1 1 -1 -1 -1 -1 -1 -1 -1 -1 1 1 1 1<br />
5 -1 -1 1 1 -1 -1 1 -1 1 1 -1 -1 1 1 -1<br />
6 1 -1 -1 1 1 -1 -1 -1 -1 1 1 -1 -1 1 1<br />
7 -1 1 -1 1 -1 1 -1 -1 1 -1 1 -1 1 -1 1<br />
8 1 1 1 1 1 1 1 -1 -1 -1 -1 -1 -1 -1 -1<br />
9 -1 -1 1 -1 1 1 -1 1 -1 -1 1 -1 1 1 -1<br />
10 1 -1 -1 -1 -1 1 1 1 1 -1 -1 -1 -1 1 1<br />
11 -1 1 -1 -1 1 -1 1 1 -1 1 -1 -1 1 -1 1<br />
12 1 1 1 -1 -1 -1 -1 1 1 1 1 -1 -1 -1 -1<br />
13 -1 -1 1 1 -1 -1 1 1 -1 -1 1 1 -1 -1 1<br />
14 1 -1 -1 1 1 -1 -1 1 1 -1 -1 1 1 -1 -1<br />
15 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1<br />
16 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1<br />
46
Hall II 15 Factor Design<br />
Run A B C D E F G H J K L M N P Q<br />
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1<br />
2 1 1 1 1 1 1 1 -1 -1 -1 -1 -1 -1 -1 -1<br />
3 1 1 1 -1 -1 -1 -1 1 1 1 1 -1 -1 -1 -1<br />
4 1 1 1 -1 -1 -1 -1 -1 -1 -1 -1 1 1 1 1<br />
5 1 -1 -1 1 1 -1 -1 1 1 -1 -1 1 1 -1 -1<br />
6 1 -1 -1 1 1 -1 -1 -1 -1 1 1 -1 -1 1 1<br />
7 1 -1 -1 -1 -1 1 1 1 1 -1 -1 -1 -1 1 1<br />
8 1 -1 -1 -1 -1 1 1 -1 -1 1 1 1 1 -1 -1<br />
9 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1<br />
10 -1 1 -1 1 -1 1 -1 -1 1 -1 1 -1 1 -1 1<br />
11 -1 1 -1 -1 1 -1 1 1 -1 1 -1 -1 1 -1 1<br />
12 -1 1 -1 -1 1 -1 1 -1 1 -1 1 1 -1 1 -1<br />
13 -1 -1 1 1 -1 -1 1 1 -1 -1 1 -1 1 1 -1<br />
14 -1 -1 1 1 -1 -1 1 -1 1 1 -1 1 -1 -1 1<br />
15 -1 -1 1 -1 1 1 -1 1 -1 -1 1 1 -1 -1 1<br />
16 -1 -1 1 -1 1 1 -1 -1 1 1 -1 -1 1 1 -1<br />
47
Hall III 15 Factor Design<br />
Run A B C D E F G H J K L M N P Q<br />
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1<br />
2 1 1 1 1 1 1 1 -1 -1 -1 -1 -1 -1 -1 -1<br />
3 1 1 1 -1 -1 -1 -1 1 1 1 1 -1 -1 -1 -1<br />
4 1 1 1 -1 -1 -1 -1 -1 -1 -1 -1 1 1 1 1<br />
5 1 -1 -1 1 1 -1 -1 1 1 -1 -1 1 1 -1 -1<br />
6 1 -1 -1 1 1 -1 -1 -1 -1 1 1 -1 -1 1 1<br />
7 1 -1 -1 -1 -1 1 1 1 1 -1 -1 -1 -1 1 1<br />
8 1 -1 -1 -1 -1 1 1 -1 -1 1 1 1 1 -1 -1<br />
9 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1<br />
10 -1 1 -1 1 -1 1 -1 -1 1 -1 1 -1 1 -1 1<br />
11 -1 1 -1 -1 1 -1 1 1 -1 -1 1 1 -1 -1 1<br />
12 -1 1 -1 -1 1 -1 1 -1 1 1 -1 -1 1 1 -1<br />
13 -1 -1 1 1 -1 -1 1 1 -1 -1 1 -1 1 1 -1<br />
14 -1 -1 1 1 -1 -1 1 -1 1 1 -1 1 -1 -1 1<br />
15 -1 -1 1 -1 1 1 -1 1 -1 1 -1 -1 1 -1 1<br />
16 -1 -1 1 -1 1 1 -1 -1 1 -1 1 1 -1 1 -1<br />
48
Hall IV 15 Factor Design<br />
Run A B C D E F G H J K L M N P Q<br />
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1<br />
2 1 1 1 1 1 1 1 -1 -1 -1 -1 -1 -1 -1 -1<br />
3 1 1 1 -1 -1 -1 -1 1 1 1 1 -1 -1 -1 -1<br />
4 1 1 1 -1 -1 -1 -1 -1 -1 -1 -1 1 1 1 1<br />
5 1 -1 -1 1 1 -1 -1 1 1 -1 -1 1 1 -1 -1<br />
6 1 -1 -1 1 1 -1 -1 -1 -1 1 1 -1 -1 1 1<br />
7 1 -1 -1 -1 -1 1 1 1 1 -1 -1 -1 -1 1 1<br />
8 1 -1 -1 -1 -1 1 1 -1 -1 1 1 1 1 -1 -1<br />
9 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1<br />
10 -1 1 -1 1 -1 -1 1 1 -1 -1 1 -1 1 -1 1<br />
11 -1 1 -1 -1 1 1 -1 -1 1 -1 1 1 -1 -1 1<br />
12 -1 1 -1 -1 1 -1 1 -1 1 1 -1 -1 1 1 -1<br />
13 -1 -1 1 1 -1 1 -1 -1 1 -1 1 -1 1 1 -1<br />
14 -1 -1 1 1 -1 -1 1 -1 1 1 -1 1 -1 -1 1<br />
15 -1 -1 1 -1 1 1 -1 1 -1 1 -1 -1 1 -1 1<br />
16 -1 -1 1 -1 1 -1 1 1 -1 -1 1 1 -1 1 -1<br />
49
Hall V 15 Factor Design<br />
Run A B C D E F G H J K L M N P Q<br />
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1<br />
2 1 1 1 1 1 1 1 -1 -1 -1 -1 -1 -1 -1 -1<br />
3 1 1 1 -1 -1 -1 -1 1 1 1 1 -1 -1 -1 -1<br />
4 1 1 1 -1 -1 -1 -1 -1 -1 -1 -1 1 1 1 1<br />
5 1 -1 -1 1 1 -1 -1 1 1 -1 -1 1 1 -1 -1<br />
6 1 -1 -1 1 1 -1 -1 -1 -1 1 1 -1 -1 1 1<br />
7 1 -1 -1 -1 -1 1 1 1 -1 1 -1 1 -1 1 -1<br />
8 1 -1 -1 -1 -1 1 1 -1 1 -1 1 -1 1 -1 1<br />
9 -1 1 -1 1 -1 1 -1 1 1 -1 -1 -1 -1 1 1<br />
10 -1 1 -1 1 -1 1 -1 -1 -1 1 1 1 1 -1 -1<br />
11 -1 1 -1 -1 1 -1 1 1 -1 -1 1 -1 1 1 -1<br />
12 -1 1 -1 -1 1 -1 1 -1 1 1 -1 1 -1 -1 1<br />
13 -1 -1 1 1 -1 -1 1 1 -1 1 -1 -1 1 -1 1<br />
14 -1 -1 1 1 -1 -1 1 -1 1 -1 1 1 -1 1 -1<br />
15 -1 -1 1 -1 1 1 -1 1 -1 -1 1 1 -1 -1 1<br />
16 -1 -1 1 -1 1 1 -1 -1 1 1 -1 -1 1 1 -1<br />
50
Constructing the Recommended 6 Factor<br />
Design<br />
Run A B C D E F G H J K L M N P Q<br />
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1<br />
2 1 1 1 1 1 1 1 -1 -1 -1 -1 -1 -1 -1 -1<br />
3 1 1 1 -1 -1 -1 -1 1 1 1 1 -1 -1 -1 -1<br />
4 1 1 1 -1 -1 -1 -1 -1 -1 -1 -1 1 1 1 1<br />
5 1 -1 -1 1 1 -1 -1 1 1 -1 -1 1 1 -1 -1<br />
6 1 -1 -1 1 1 -1 -1 -1 -1 1 1 -1 -1 1 1<br />
7 1 -1 -1 -1 -1 1 1 1 1 -1 -1 -1 -1 1 1<br />
8 1 -1 -1 -1 -1 1 1 -1 -1 1 1 1 1 -1 -1<br />
9 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1<br />
10 -1 1 -1 1 -1 1 -1 -1 1 -1 1 -1 1 -1 1<br />
11 -1 1 -1 -1 1 -1 1 1 -1 1 -1 -1 1 -1 1<br />
12 -1 1 -1 -1 1 -1 1 -1 1 -1 1 1 -1 1 -1<br />
13 -1 -1 1 1 -1 -1 1 1 -1 -1 1 -1 1 1 -1<br />
14 -1 -1 1 1 -1 -1 1 -1 1 1 -1 1 -1 -1 1<br />
15 -1 -1 1 -1 1 1 -1 1 -1 -1 1 1 -1 -1 1<br />
16 -1 -1 1 -1 1 1 -1 -1 1 1 -1 -1 1 1 -1<br />
Hall II – Columns D, E, H, K, M, Q<br />
51
Recommended Nonregular 6 Factor Design<br />
Run A B C D E F<br />
1 1 1 1 1 1 1<br />
2 1 1 -1 -1 -1 -1<br />
3 -1 -1 1 1 -1 -1<br />
4 -1 -1 -1 -1 1 1<br />
5 1 1 1 -1 1 -1<br />
6 1 1 -1 1 -1 1<br />
7 -1 -1 1 -1 -1 1<br />
8 -1 -1 -1 1 1 -1<br />
9 1 -1 1 1 1 -1<br />
10 1 -1 -1 -1 -1 1<br />
11 -1 1 1 1 -1 1<br />
12 -1 1 -1 -1 1 -1<br />
13 1 -1 1 -1 -1 -1<br />
14 1 -1 -1 1 1 1<br />
15 -1 1 1 -1 1 1<br />
16 -1 1 -1 1 -1 -1<br />
52
Constructing Recommended Design<br />
Using the “Generator” Approach<br />
E = 0.5AC+0.5AD+0.5BC-0.5BD F = -0.5AC+0.5AD+0.5BC+0.5BD<br />
Run A B C D<br />
Run<br />
E<br />
Run<br />
F<br />
1 1 1 1 1<br />
1 1<br />
1 1<br />
2 1 1 1 -1<br />
2 1<br />
2 -1<br />
3 1 1 -1 1<br />
3 -1<br />
3 1<br />
4 1 1 -1 -1<br />
4 -1<br />
4 -1<br />
5 1 -1 1 1<br />
5 1<br />
5 -1<br />
6 1 -1 1 -1<br />
6 -1<br />
6 -1<br />
7 1 -1 -1 1<br />
7 1<br />
7 1<br />
8 1 -1 -1 -1<br />
8 -1<br />
8 1<br />
9 -1 1 1 1<br />
9 -1<br />
9 1<br />
10 -1 1 1 -1<br />
10 1<br />
10 1<br />
11 -1 1 -1 1<br />
11 -1<br />
11 -1<br />
12 -1 1 -1 -1<br />
12 1<br />
12 -1<br />
13 -1 -1 1 1<br />
13 -1<br />
13 -1<br />
14 -1 -1 1 -1<br />
14 -1<br />
14 1<br />
15 -1 -1 -1 1<br />
15 1<br />
15 -1<br />
16 -1 -1 -1 -1<br />
16 1<br />
16 1<br />
53
Color Plot for the Standard Minimum<br />
Aberration Resolution IV 7-Factor Design<br />
54
Constructing the Recommended 7 Factor<br />
Design<br />
Run A B C D E F G H J K L M N P Q<br />
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1<br />
2 1 1 1 1 1 1 1 -1 -1 -1 -1 -1 -1 -1 -1<br />
3 1 1 1 -1 -1 -1 -1 1 1 1 1 -1 -1 -1 -1<br />
4 1 1 1 -1 -1 -1 -1 -1 -1 -1 -1 1 1 1 1<br />
5 1 -1 -1 1 1 -1 -1 1 1 -1 -1 1 1 -1 -1<br />
6 1 -1 -1 1 1 -1 -1 -1 -1 1 1 -1 -1 1 1<br />
7 1 -1 -1 -1 -1 1 1 1 1 -1 -1 -1 -1 1 1<br />
8 1 -1 -1 -1 -1 1 1 -1 -1 1 1 1 1 -1 -1<br />
9 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1<br />
10 -1 1 -1 1 -1 1 -1 -1 1 -1 1 -1 1 -1 1<br />
11 -1 1 -1 -1 1 -1 1 1 -1 -1 1 1 -1 -1 1<br />
12 -1 1 -1 -1 1 -1 1 -1 1 1 -1 -1 1 1 -1<br />
13 -1 -1 1 1 -1 -1 1 1 -1 -1 1 -1 1 1 -1<br />
14 -1 -1 1 1 -1 -1 1 -1 1 1 -1 1 -1 -1 1<br />
15 -1 -1 1 -1 1 1 -1 1 -1 1 -1 -1 1 -1 1<br />
16 -1 -1 1 -1 1 1 -1 -1 1 -1 1 1 -1 1 -1<br />
Hall III – Columns A, B, D, H, J, M, Q<br />
55
Recommended Nonregular 7 Factor Design<br />
Run A B C D E F G<br />
1 1 1 1 1 1 1 1<br />
2 1 1 1 -1 -1 -1 -1<br />
3 1 1 -1 1 1 -1 -1<br />
4 1 1 -1 -1 -1 1 1<br />
5 1 -1 1 1 -1 1 -1<br />
6 1 -1 1 -1 1 -1 1<br />
7 1 -1 -1 1 -1 -1 1<br />
8 1 -1 -1 -1 1 1 -1<br />
9 -1 1 1 1 1 1 -1<br />
10 -1 1 1 -1 -1 -1 1<br />
11 -1 1 -1 1 -1 1 1<br />
12 -1 1 -1 -1 1 -1 -1<br />
13 -1 -1 1 1 -1 -1 -1<br />
14 -1 -1 1 -1 1 1 1<br />
15 -1 -1 -1 1 1 -1 1<br />
16 -1 -1 -1 -1 -1 1 -1<br />
56
Comparison of Color Plots for the Standard<br />
and No-confounding Designs<br />
The recommended design is orthogonal and does<br />
not have any complete confounding of effects<br />
57
Color Plot for the Standard Minimum<br />
Aberration Resolution IV 8-Factor Design<br />
58
Constructing the Recommended 8 Factor Design<br />
Run A B C D E F G H J K L M N P Q<br />
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1<br />
2 1 1 1 1 1 1 1 -1 -1 -1 -1 -1 -1 -1 -1<br />
3 1 1 1 -1 -1 -1 -1 1 1 1 1 -1 -1 -1 -1<br />
4 1 1 1 -1 -1 -1 -1 -1 -1 -1 -1 1 1 1 1<br />
5 1 -1 -1 1 1 -1 -1 1 1 -1 -1 1 1 -1 -1<br />
6 1 -1 -1 1 1 -1 -1 -1 -1 1 1 -1 -1 1 1<br />
7 1 -1 -1 -1 -1 1 1 1 1 -1 -1 -1 -1 1 1<br />
8 1 -1 -1 -1 -1 1 1 -1 -1 1 1 1 1 -1 -1<br />
9 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1<br />
10 -1 1 -1 1 -1 -1 1 1 -1 -1 1 -1 1 -1 1<br />
11 -1 1 -1 -1 1 1 -1 -1 1 -1 1 1 -1 -1 1<br />
12 -1 1 -1 -1 1 -1 1 -1 1 1 -1 -1 1 1 -1<br />
13 -1 -1 1 1 -1 1 -1 -1 1 -1 1 -1 1 1 -1<br />
14 -1 -1 1 1 -1 -1 1 -1 1 1 -1 1 -1 -1 1<br />
15 -1 -1 1 -1 1 1 -1 1 -1 1 -1 -1 1 -1 1<br />
16 -1 -1 1 -1 1 -1 1 1 -1 -1 1 1 -1 1 -1<br />
Hall IV – Columns A, B, D, F, H, J, M, P<br />
59
Recommended Nonregular 8 Factor Design<br />
Run A B C D E F G H<br />
1 1 1 1 1 1 1 1 1<br />
2 1 1 1 1 -1 -1 -1 -1<br />
3 1 1 -1 -1 1 1 -1 -1<br />
4 1 1 -1 -1 -1 -1 1 1<br />
5 1 -1 1 -1 1 -1 1 -1<br />
6 1 -1 1 -1 -1 1 -1 1<br />
7 1 -1 -1 1 1 -1 -1 1<br />
8 1 -1 -1 1 -1 1 1 -1<br />
9 -1 1 1 1 1 1 1 1<br />
10 -1 1 1 -1 1 -1 -1 -1<br />
11 -1 1 -1 1 -1 -1 1 -1<br />
12 -1 1 -1 -1 -1 1 -1 1<br />
13 -1 -1 1 1 -1 -1 -1 1<br />
14 -1 -1 1 -1 -1 1 1 -1<br />
15 -1 -1 -1 1 1 1 -1 -1<br />
16 -1 -1 -1 -1 1 -1 1 1 60
Comparison of Color Plots for the Standard<br />
and No-confounding Designs<br />
The recommended design is orthogonal and does<br />
not have any complete confounding of effects<br />
61
Alternatives to Resolution III Designs<br />
The regular resolution III designs with from 9 to 15 factors in 16 runs are<br />
used frequently in practice<br />
These designs completely confound some interactions with main effects<br />
For example, in the minimum aberration nine factor case, 12 two-factor<br />
interactions are aliased with main effects and 24 two-factor interactions<br />
are confounded in groups with other two-factor interactions<br />
Follow-up experiments are often necessary, and the best augmentation<br />
approach may not be obvious.<br />
Nonregular designs with no pure confounding of main effects and twofactor<br />
interactions are useful alternatives.<br />
We provide a collection of these designs.<br />
62
Recommended 9 Factor Design<br />
Run A B C D E F G H J<br />
1 -1 -1 -1 -1 -1 -1 1 -1 1<br />
2 -1 -1 -1 1 -1 1 -1 1 -1<br />
3 -1 -1 1 -1 1 1 1 1 -1<br />
4 -1 -1 1 1 1 -1 -1 -1 1<br />
5 -1 1 -1 -1 1 1 -1 1 1<br />
6 -1 1 -1 1 1 -1 1 -1 -1<br />
7 -1 1 1 -1 -1 -1 -1 1 -1<br />
8 -1 1 1 1 -1 1 1 -1 1<br />
9 1 -1 -1 -1 1 -1 -1 -1 -1<br />
10 1 -1 -1 1 1 1 1 1 1<br />
11 1 -1 1 -1 -1 1 -1 -1 1<br />
12 1 -1 1 1 -1 -1 1 1 -1<br />
13 1 1 -1 -1 -1 1 1 -1 -1<br />
14 1 1 -1 1 -1 -1 -1 1 1<br />
15 1 1 1 -1 1 -1 1 1 1<br />
16 1 1 1 1 1 1 -1 -1 -1<br />
Correlation of Main Effects and Two-Factor Interactions
Recommended 10 Factor Design<br />
Run A B C D E F G H J K<br />
1 -1 -1 -1 -1 1 -1 -1 1 -1 1<br />
2 -1 -1 -1 1 1 1 -1 -1 1 1<br />
3 -1 -1 1 -1 -1 1 1 1 1 1<br />
4 -1 -1 1 -1 1 -1 1 -1 1 -1<br />
5 -1 1 -1 1 -1 1 1 1 -1 1<br />
6 -1 1 -1 1 1 -1 1 -1 -1 -1<br />
7 -1 1 1 -1 -1 -1 -1 1 -1 -1<br />
8 -1 1 1 1 -1 1 -1 -1 1 -1<br />
9 1 -1 -1 -1 -1 1 1 -1 -1 -1<br />
10 1 -1 -1 1 -1 -1 -1 1 1 -1<br />
11 1 -1 1 1 -1 -1 1 -1 -1 1<br />
12 1 -1 1 1 1 1 -1 1 -1 -1<br />
13 1 1 -1 -1 -1 -1 -1 -1 1 1<br />
14 1 1 -1 -1 1 1 1 1 1 -1<br />
15 1 1 1 -1 1 1 -1 -1 -1 1<br />
16 1 1 1 1 1 -1 1 1 1 1<br />
Correlation of Main Effects and Two-Factor Interactions<br />
64
Recommended 11 Factor Design<br />
Run A B C D E F G H J K L<br />
1 -1 -1 -1 1 1 -1 -1 -1 1 -1 1<br />
2 -1 -1 1 -1 -1 -1 1 -1 1 -1 -1<br />
3 -1 -1 1 -1 1 1 -1 1 -1 -1 -1<br />
4 -1 -1 1 1 -1 1 1 1 -1 1 1<br />
5 -1 1 -1 -1 -1 -1 -1 -1 -1 1 1<br />
6 -1 1 -1 1 -1 1 1 -1 -1 -1 -1<br />
7 -1 1 -1 1 1 1 -1 1 1 1 -1<br />
8 -1 1 1 -1 1 -1 1 1 1 1 1<br />
9 1 -1 -1 -1 -1 1 -1 1 1 -1 1<br />
10 1 -1 -1 -1 1 1 1 -1 -1 1 1<br />
11 1 -1 -1 1 -1 -1 1 1 1 1 -1<br />
12 1 -1 1 1 1 -1 -1 -1 -1 1 -1<br />
13 1 1 -1 -1 1 -1 1 1 -1 -1 -1<br />
14 1 1 1 -1 -1 1 -1 -1 1 1 -1<br />
15 1 1 1 1 -1 -1 -1 1 -1 -1 1<br />
16 1 1 1 1 1 1 1 -1 1 -1 1<br />
Correlation of Main Effects and Two-Factor Interactions<br />
65
Recommended 12 Factor Design<br />
Run A B C D E F G H J K L M<br />
1 -1 -1 -1 -1 1 -1 -1 1 1 -1 1 1<br />
2 -1 -1 -1 1 -1 1 1 1 -1 -1 1 -1<br />
3 -1 -1 1 -1 -1 -1 1 -1 1 1 -1 1<br />
4 -1 -1 1 1 1 1 -1 -1 -1 1 -1 -1<br />
5 -1 1 -1 1 -1 -1 -1 -1 -1 -1 -1 1<br />
6 -1 1 -1 1 1 1 1 -1 1 1 1 1<br />
7 -1 1 1 -1 -1 1 -1 1 1 -1 -1 -1<br />
8 -1 1 1 -1 1 -1 1 1 -1 1 1 -1<br />
9 1 -1 -1 -1 -1 1 -1 -1 1 1 1 -1<br />
10 1 -1 -1 -1 1 -1 1 -1 -1 -1 -1 -1<br />
11 1 -1 1 1 -1 -1 -1 1 -1 1 1 1<br />
12 1 -1 1 1 1 1 1 1 1 -1 -1 1<br />
13 1 1 -1 -1 -1 1 1 1 -1 1 -1 1<br />
14 1 1 -1 1 1 -1 -1 1 1 1 -1 -1<br />
15 1 1 1 -1 1 1 -1 -1 -1 -1 1 1<br />
16 1 1 1 1 -1 -1 1 -1 1 -1 1 -1<br />
Correlation of Main Effects and Two-Factor Interactions<br />
66
Recommended 13 Factor Design<br />
Run A B C D E F G H J K L M N<br />
1 -1 -1 -1 1 1 -1 -1 1 -1 1 1 -1 1<br />
2 -1 -1 1 -1 -1 -1 -1 -1 1 1 -1 -1 1<br />
3 -1 -1 1 -1 1 1 1 1 1 -1 1 -1 -1<br />
4 -1 -1 1 1 -1 1 1 1 -1 1 -1 1 -1<br />
5 -1 1 -1 -1 -1 1 -1 -1 -1 -1 1 -1 -1<br />
6 -1 1 -1 -1 1 1 1 -1 -1 1 -1 1 1<br />
7 -1 1 -1 1 -1 -1 1 1 1 -1 1 1 1<br />
8 -1 1 1 1 1 -1 -1 -1 1 -1 -1 1 -1<br />
9 1 -1 -1 -1 -1 -1 1 -1 1 1 1 1 -1<br />
10 1 -1 -1 -1 1 -1 -1 1 -1 -1 -1 1 -1<br />
11 1 -1 -1 1 1 1 1 -1 1 -1 -1 -1 1<br />
12 1 -1 1 1 -1 1 -1 -1 -1 -1 1 1 1<br />
13 1 1 -1 1 -1 1 -1 1 1 1 -1 -1 -1<br />
14 1 1 1 -1 -1 -1 1 1 -1 -1 -1 -1 1<br />
15 1 1 1 -1 1 1 -1 1 1 1 1 1 1<br />
16 1 1 1 1 1 -1 1 -1 -1 1 1 -1 -1<br />
Correlation of Main Effects and Two-Factor Interactions<br />
67
Recommended 14 Factor Design<br />
Run A B C D E F G H J K L M N P<br />
1 -1 -1 -1 -1 1 -1 1 1 -1 1 1 -1 -1 1<br />
2 -1 -1 -1 1 -1 -1 1 -1 1 1 -1 1 1 -1<br />
3 -1 -1 1 -1 -1 1 -1 1 1 1 -1 -1 1 1<br />
4 -1 -1 1 1 1 1 1 -1 -1 -1 1 -1 1 -1<br />
5 -1 1 -1 -1 -1 1 1 -1 1 -1 1 1 -1 1<br />
6 -1 1 -1 1 1 1 -1 1 -1 1 -1 1 -1 -1<br />
7 -1 1 1 -1 -1 -1 -1 1 -1 -1 1 1 1 -1<br />
8 -1 1 1 1 1 -1 -1 -1 1 -1 -1 -1 -1 1<br />
9 1 -1 -1 -1 1 1 -1 -1 -1 -1 -1 1 1 1<br />
10 1 -1 -1 1 -1 1 -1 1 1 -1 1 -1 -1 -1<br />
11 1 -1 1 -1 1 -1 -1 -1 1 1 1 1 -1 -1<br />
12 1 -1 1 1 -1 -1 1 1 -1 -1 -1 1 -1 1<br />
13 1 1 -1 -1 1 -1 1 1 1 -1 -1 -1 1 -1<br />
14 1 1 -1 1 -1 -1 -1 -1 -1 1 1 -1 1 1<br />
15 1 1 1 -1 -1 1 1 -1 -1 1 -1 -1 -1 -1<br />
16 1 1 1 1 1 1 1 1 1 1 1 1 1 1<br />
Correlation of Main Effects and Two-Factor Interactions<br />
68
Introducing Definitive Screening<br />
Jones and Nachtsheim (2011) provide a class of screening<br />
designs for 5 factors and up.<br />
These are three level designs for numeric factors.<br />
We will discuss their properties, show how to construct them<br />
and give an example.<br />
69
Motivation: Problems with Standard Screening Designs<br />
Resolution III designs confound main effects and two-factor interactions.<br />
Plackett-Burman designs have “complex aliasing of the main effects by twofactor<br />
interactions.<br />
Resolution IV designs confound two-factor interactions with each other, so if<br />
one is active, you usually need further runs to resolve the active effects.<br />
Center runs give an overall measure of curvature but you do not know which<br />
factor(s) are causing the curvature.<br />
Even the nonregular orthogonal designs we just discussed have aliasing of<br />
+0.5 or -0.5 of some main effects by some two-factor interactions. Plus they<br />
have no way of resolving quadratic effects.
Screening Conundrum – Two Models<br />
The full model containing both 6 first-order and 15 secondorder<br />
terms is:<br />
But n = 12, so we can only fit the intercept and the main<br />
effects:<br />
Standard result: some main effects estimates are biased:<br />
where the “alias” matrix is:<br />
71
Alias Matrix of Plackett-Burman design<br />
PB (non-regular) design has “complex aliasing”
If only there were another six factor 12 run design with<br />
this alias matrix:
Screening Design – Wish List<br />
1. Orthogonal main effects.<br />
2. Main effects uncorrelated with two-factor interactions and<br />
quadratic effects.<br />
3. Estimable quadratic effects – three-level design.<br />
4. Small number of runs – order of the number of factors.<br />
5. Good projective properties.
Design Structure<br />
1. Scaled values of each element of the design are either +1, -1 or 0.<br />
2. Each even numbered row is a mirror image of the previous row.<br />
3. For the kth column, rows 2k and 2k−1 contain zeros.<br />
4. The last row contains all zero elements.<br />
Run A B C D E F<br />
1 0 1 -1 -1 -1 -1<br />
2 0 -1 1 1 1 1<br />
3 1 0 -1 1 1 -1<br />
4 -1 0 1 -1 -1 1<br />
5 -1 -1 0 1 -1 -1<br />
6 1 1 0 -1 1 1<br />
7 -1 1 1 0 1 -1<br />
8 1 -1 -1 0 -1 1<br />
9 1 -1 1 -1 0 -1<br />
10 -1 1 -1 1 0 1<br />
11 1 1 1 1 -1 0<br />
12 -1 -1 -1 -1 1 0<br />
13 0 0 0 0 0 0
How did we find this design? – we used an<br />
optimization algorithm
Problem: Finding orthogonal main effects plans.<br />
Algorithmic approach gave orthogonal main effects<br />
plans for:<br />
6 factors<br />
8 factors<br />
10 factors<br />
but not 12 factors.
Orthogonal Main Effects Design Construction<br />
Conference matrix<br />
From Wikipedia, the free encyclopedia<br />
In mathematics, a conference matrix (also called a C-matrix) is a square<br />
matrix C with 0 on the diagonal and +1 and −1 off the diagonal, such that<br />
C T C is a multiple of the identity matrix I. Thus, if the matrix has order n,<br />
C T C = (n−1)I.<br />
Conference matrices first arose in connection with a problem in<br />
telephony. [3] They were first described by Vitold Belevitch who also gave<br />
them their name. Belevitch was interested in constructing ideal telephone<br />
conference networks from ideal transformers and discovered that such<br />
networks were represented by conference matrices, hence the name. [4]<br />
Other applications are in statistics, [5] and another is in elliptic geometry. [6]
From Wikipedia<br />
article on<br />
Conference<br />
Matrices<br />
Conference Matrices &Telephony
Conference Matrix of Order 6
Statistical Properties<br />
1. Orthogonal for the main effects.<br />
2. The number of required runs is only one more than twice the number of<br />
factors. ***<br />
3. Unlike resolution III designs, main effects are independent of two-factor<br />
interactions.<br />
4. Unlike resolution IV designs, two-factor interactions are not completely<br />
confounded with other two-factor interactions, although they may be<br />
correlated<br />
5. Unlike resolution III, IV and V designs with added center points, all<br />
quadratic effects are estimable in models comprised of any number of<br />
linear and quadratic main effects terms.<br />
6. Quadratic effects are orthogonal to main effects and not completely<br />
confounded (though correlated) with interaction effects.<br />
7. If there are more than six factors, the designs are capable of efficiently<br />
estimating all possible full quadratic models involving three or fewer<br />
factors
Example 1<br />
6 factors and 13 runs.<br />
Run A B C D E F<br />
1 0 1 -1 -1 -1 -1<br />
2 0 -1 1 1 1 1<br />
3 1 0 -1 1 1 -1<br />
4 -1 0 1 -1 -1 1<br />
5 -1 -1 0 1 -1 -1<br />
6 1 1 0 -1 1 1<br />
7 -1 1 1 0 1 -1<br />
8 1 -1 -1 0 -1 1<br />
9 1 -1 1 -1 0 -1<br />
10 -1 1 -1 1 0 1<br />
11 1 1 1 1 -1 0<br />
12 -1 -1 -1 -1 1 0<br />
13 0 0 0 0 0 0<br />
Note that even numbered row mirrors the previous row.<br />
D-efficiency is 85.5% and the design is orthogonal for the main effects.
Alias Matrices<br />
The D-optimal design with one added center point has substantial aliasing of<br />
each main effect with a number of two-factor interactions.<br />
For our design there is no aliasing between main effects and two-factor interactions.
Column Correlations<br />
0.25<br />
0.5<br />
0.4655<br />
0.1333
Example 2<br />
12 factors – 25 runs.
Column Correlations<br />
Note the zero<br />
correlations<br />
among main<br />
effects
Example from JQT paper<br />
87
Where did the data come from?<br />
We simulated it using the model below…<br />
88
Stepwise Results<br />
89
Recapitulation – Definitive Screening Design<br />
1. Orthogonal main effects plans.<br />
2. Two-factor interactions are uncorrelated with main effects.<br />
3. Quadratic effects are uncorrelated with main effects.<br />
4. All quadratic effects are estimable.<br />
5. The number of runs is only one more than twice the number of<br />
factors.<br />
6. For six factors or more, the designs can estimate all possible full<br />
quadratic models involving three or fewer factors
References<br />
1. Box, G. E. P., and Hunter, J. S. (1961), “The 2k−p Fractional Factorial Designs,”Technometrics,<br />
3, 449–458.<br />
2. Goethals, J. and Seidel, J. (1967). ”Orthogonal matrices with zero diagonal”. Canadian Journal<br />
of Mathematics, 19, pp. 1001–1010.<br />
3. Hall, M. Jr. (1961). Hadamard matrix of order 16. Jet Propulsion Laboratory Research Summary,<br />
1, 21–26.<br />
4. Jones, B. A. and Montgomery, D. C. (2010) “Alternatives to Resolution IV Screening Designs in<br />
16 Runs.”, International Journal of Experimental Design and Process Optimization, 2010; Vol. 1<br />
No. 4: 285-295.<br />
5. Jones, B. and Nachtsheim, C. J. (2011) “Efficient Designs with Minimal Aliasing” Technometrics,<br />
53. 62-71.<br />
6. Jones, B and Nachtsheim, C. (2011) “A Class of Three-Level Designs for Definitive Screening in<br />
the Presence of Second-Order Effects” Journal of Quality Technology, 43. 1-15.<br />
7. Plackett, R. L., and Burman, J. P. (1946), “The Design of Optimum Multifactor Experiments,”<br />
Biometrika, 33, 305–325.<br />
8. Montgomery, D.C. (2012) “Design and Analysis of Experiments” 8 th Edition, Wiley Hoboken,<br />
<strong>New</strong> Jersey.<br />
9. Sun, D. X., Li, W., and Ye, K. Q. (2002), “An Algorithm for Sequentially Constructing Non-<br />
Isomorphic Orthogonal Designs and Its Applications,” Technical Report SUNYSB-AMS-02-13,<br />
State University of <strong>New</strong> York at Stony Brook, Dept. of Applied Mathematics and Statistics.