Machine Learning - DISCo

search, there is a danger that a suboptimal choice will be made at any step. To
reduce this risk, we can extend the algorithm to perform a beam search; that is,
a search in which the algorithm maintains a list of the k best candidates at each
step, rather than a single best candidate. On each search step, descendants (specializations)
are generated for each of these k best candidates, and the resulting
set is again reduced to the k most promising members. Beam search keeps track
of the most promising alternatives to the current top-rated hypothesis, so that all
of their successors can be considered at each search step. This general to specific
beam search algorithm is used by the CN2 program described by Clark and Niblett
(1989). The algorithm is described in Table 10.2.
LEARN-ONE-RULE(T~~~~~~~~~~~U~~,
Attributes, Examples, k)
Returns a single rule that covers some of the Examples. Conducts a generalJotospec$c
greedy beam search for the best rule, guided by the PERFORMANCE metric.
a Initialize Besthypothesis to the most general hypothesis 0
a Initialize Candidatehypotheses to the set (Besthypothesis)
a While Candidatehypotheses is not empty, Do
I. Generate the next more spec@ candidatehypotheses
a Allronstraints c the set of all constraints of the form (a = v), where a is a member
of Attributes, and v is a value of a that occurs in the current set of Examples
a Newrandidatehypotheses c
for each h in Candidatehypotheses,
for each c in Alll-onstraints,
create a specialization of h by adding the constraint c
a Remove from Newl-andidatehypotheses any hypotheses that are duplicates, inconsistent,
or not maximally specific
2. Update Besthypothesis
a For all h in Newnandidatehypotheses do
a If PERFORMANCE(^, Examples, Targetattribute)
z PERFORMANCE(Besthypothesis, Examples, Targetattribute))
Then Besthypothesis t h
3. Update Candidatehypotheses
a Candidatehypotheses c the k best members of New-candidatehypotheses, according
to the PERFORMANCE measure.
a Return a rule of the form
"IF Best hypothesis THEN prediction"
where prediction is the most frequent value of Targetattribute among those Examples
that match Besthypothesis.
PERM)RMANCE(~, Examples, Target attribute)
a hxxamples t the subset of Examples that match h
return -Entropy(hxxarnples), where entropy is with respect to Targetattribute
TABLE 10.2
One implementation for LEARN-ONE-RULE is a general-to-specific beam search. The frontier of current
hypotheses is represented by the variable Candidatehypotheses. This algorithm is similar to that
used by the CN2 program, described by Clark and Niblett (1989).