Foundations of Data Science

Corollary 6.16 (VC-dimension sample bound) For any class H and distribution D,
a training sample S of size
( )
1
O
ɛ [VCdim(H) log(1/ɛ) + log(1/δ)]
is sufficient to ensure that with probability ≥ 1 − δ, every h ∈ H with err D (h) ≥ ɛ has
err S (h) > 0 (equivalently, every h ∈ H with err S (h) = 0 has err D (h) < ɛ).
For any class H, VCdim(H) ≤ log 2 (|H|) since H must have at least 2 k concepts in
order to shatter k points. Thus Corollary 6.16 is never too much worse than Theorem 6.1
and can be much better.
6.9.2 Examples: VC-Dimension and Growth Function
Rectangles with axis-parallel edges
As we saw above, the class of axis-parallel rectangles in the plane has VC-dimension
4 and growth function C[n] = O(n 4 ).
Intervals of the reals
Intervals on the real line can shatter any set of two points but no set of three points
since the subset of the first and last points cannot be isolated. Thus, the VC-dimension
of intervals is two. Also, C[n] = O(n 2 ) since we have O(n 2 ) choices for the left and right
endpoints.
Pairs of intervals of the reals
Consider the family of pairs of intervals, where a pair of intervals is viewed as the set
of points that are in at least one of the intervals, in other words, their set union. There
exists a set of size four that can be shattered but no set of size five since the subset of first,
third, and last point cannot be isolated. Thus, the VC-dimension of pairs of intervals is
four. Also we have C[n] = O(n 4 ).
Convex polygons
Consider the set system of all convex polygons in the plane. For any positive integer
n, place n points on the unit circle. Any subset of the points are the vertices of a convex
polygon. Clearly that polygon will not contain any of the points not in the subset. This
shows that convex polygons can shatter arbitrarily large sets, so the VC-dimension is
infinite. Notice that this also implies that C[n] = 2 n .
Half spaces in d-dimensions
209