Selecting the Right Objective Measure for Association Analysis*

Table 12. Data sets used in our experiments.
Name Description Number of Variables
re0 Reuters-21578 articles 2886
la1 LA-Times articles 31472
product Retail data 14462
S&P 500 Stock market data 976
E-Com Web data 6664
Census Survey data 59
results indicate that the distance function D decreases with increasing sample
size, mainly because the larger the sample size, the more similar it is to the entire
data set. Furthermore, the DISJOINT algorithm does a substantially better job
than random sampling in terms of choosing the right tables to be presented to
the domain experts. This is because it tends to select tables that are furthest
apart in terms of their relative rankings and tables that create a huge amount of
ranking conflicts. Even at k = 20, there is little difference (D < 0.15) between
the similarity matrices S s and S T .
0.9
0.8
RANDOM
DISJOINT (p=10)
DISJOINT (p=20)
0.7
Average Distance, D
0.6
0.5
0.4
0.3
0.2
0.1
0
0 10 20 30 40 50 60 70 80 90 100
Sample size, k
Fig. 8. Average distance between similarity matrix computed from the subset (S s) and
the similarity matrix computed from the entire set of contingency tables (S T ) for the
re0 data set.
We complement our evaluation above by showing that the ordering of measures
produced by the DISJOINT algorithm on even a small sample of 20 tables