An efficient algorithm for maximal margin clustering - ResearchGate

J Glob Optim
logic, the maximal margin clustering with a fixed number of features can also be cast as an
approximate solution to the original maximal margin clustering problem and be used in a
semi-supervised learning setting. Later, we investigate an algorithm based on this idea and
analyze its convergence properties.
The rest of the paper is organized as follows. In Sect. 2, we discuss the complexity of
the maximal margin problem and present an algorithm. Then, we analyze how the idea of
maximal margin separation can be extended to some existing graph-cut models for clustering.
In Sect. 3, we discuss the maximal margin clustering problem with a fixed number
of features, which can be viewed as a combination of the maximal margin clustering and
feature selection. In Sect. 4, we propose a new successive procedure to approximately solve
the original hard maximal margin clustering problem and study its convergence properties.
We also discuss how to deal with the semi-supervised learning case. Experimental results
based on these ideas are reported in Sect. 5 and we conclude the paper in Sect. 6.
2 Complexity of hard maximal margin clustering
In this section, we investigate the complexity of the hard maximal margin clustering. Throughout
the paper, we make the following assumption:
Assumption 1 [General Position] All the points in the input data set V ={v 1 ,v 2 ,...,v n }
are in general position, i.e., any d points in V will define precisely one hyperplane in R d .
We remark that the above assumption is quite reasonable, because if the data set V does
not satisfy the assumption, we may perturb the data set slightly so that the assumption holds.
We can now state the following result.
Theorem 1 Suppose that the input data set satisfies Assumption 1 and (w ∗ ,b ∗ ) is the global
solution to problem (1). Then, there must exist a subset V ∗ ⊂ V with either d or d + 1 points
such that at least one of the following two conditions hold:
(1) The hyperplane defined by the points in V ∗ will create the same separation as the
optimal separating hyperplane.
(2) The optimal separating hyperplane is also the global solution of problem (1) where V
is replaced by V ∗ .
Proof Suppose that (w ∗ ,b ∗ ) is the global solution to problem (1), we can then separate the
data set V into two subsets V 1 and V 2 such that
V 1 ={v ∈ V : (w ∗ ) T v + b ∗ > 0},
V 2 ={v ∈ V : (w ∗ ) T v + b ∗ < 0}. (3)
Further, by the definition of maximal margin, there exist two subsets Ṽ 1 , Ṽ 2 such that ∀ v 1 ,v 2 :
v 1 ∈ Ṽ 1 ⊂ V 1 ,v 2 ∈ Ṽ 2 ⊂ V 2 ,
(
(w ∗ ) T v 1 + b ∗ =− (w ∗ ) T v 2 + b ∗) = 1. (4)
For a given data set V, let|V| denote the number of points in V. Since all points in V are in
general position, we can claim that
|Ṽ 1 |≤d, |Ṽ 2 |≤d, |Ṽ 1 |+|Ṽ 2 |≥d. (5)
If there exist at least d + 1 points in Ṽ 1 ∪ Ṽ 2 , we may select d + 1 points from Ṽ 1 ∪ Ṽ 2 and
these d + 1 points will define the separating hyperplane (w ∗ ) T v + b ∗ = 0 due to the general
123