Anomaly Detection for Monitoring

Clustering
Not all anomaly detection is based on time series of metrics. Clustering,
or cluster analysis is one way of grouping elements together
to try to find the odd ones out. Netflix has written about their
anomaly detection methods based on cluster analysis. 2 They apply
cluster analysis techniques on server clusters to identify anomalous,
misbehaving, or underperforming servers.
K-Means clustering is a common algorithm that’s fairly simple to
implement. Here’s an example:
Non-Parametric Analysis
Not all anomaly detection techniques need models to draw useful
conclusions about metrics. Some avoid models altogether! These are
called non-parametric anomaly detection methods, and use theory
from a larger field called non-parametric statistics.
The Kolmogorov-Smirnov test is one non-parametric method that
has gained popularity in the monitoring community. It tests for
changes in the distributions of two samples. An example of a type of
question that it can answer is, “is the distribution of CPU usage this
week significantly different from last week?” Your time intervals
don’t necessarily have to be as long as a week, of course.
We once learned an interesting lesson while trying to solve a sticky
problem with a non-Gaussian distribution of values. We wanted to
2 “Tracking down the Villains: Outlier Detection at Netflix”
56 | Chapter 6: The Broader Landscape