Anomaly Detection for Monitoring

More documents

Recommendations

Info

letter μ represents a constant mean, and ɛ is a random variable representing noise or error in the system. In the case of the basic control chart model, ɛ is assumed to be a Gaussian distributed random variable. Control charts have the following characteristics: • They assume a fixed or known mean and spread of values. • The values are assumed to be Gaussian (normally) distributed around the mean. • They can detect one or multiple points that are outside the desired range. Figure 3-2. A basic control chart with fixed control limits, which are represented with dashed lines. Values are considered to be anomalous if they cross the control limits. Moving Window Control Chart The major problem with a basic control chart is the assumption of stability. In time series analysis, the usual term is stationary, which means the values have a consistent mean and spread over time. Many systems change rapidly, so you can’t assume a fixed mean <strong>for</strong> the metrics you’re monitoring. Without this key assumption holding true, you will either get false positives or fail to detect true anoma‐ 18 | Chapter 3: Modeling and Predicting
lies. To fix this problem, the control chart needs to adapt to a changing mean and spread over time. There are two basic ways to do this: • Slice up your control chart into smaller time ranges or fixed windows, and treat each window as its own independent fixed control chart with a different mean and spread. The values within each window are used to compute the mean and standard deviation <strong>for</strong> that window. Within a small interval, everything looks like a regular fixed control chart. At a larger scale, what you have is a control chart that changes across windows. • Use a moving window, also called a sliding window. Instead of using predefined time ranges to construct windows, at each point you generate a moving window that covers the previous N points. The benefit is that instead of having a fixed mean within a time range, the mean changes after each value yet still considers the same number of points to compute the mean. Moving windows have major disadvantages. You have to keep track of recent history because you need to consider all of the values that fall into a window. Depending on the size of your windows, this can be computationally expensive, especially when tracking a large number of metrics. Windows also have poor characteristics in the presence of large spikes. When a spike enters a window, it causes an abrupt shift in the window until the spike eventually leaves, which causes another abrupt shift. Statistical Process Control | 19
Page 3 and 4: Anomaly Detection for Monitoring A
Page 5 and 6: Table of Contents Foreword. . . . .
Page 7: Foreword Monitoring is currently un
Page 10 and 11: tion” to anomaly detection is imp
Page 12 and 13: Why do we assume these things? Are
Page 14 and 15: Conclusions If you are like most of
Page 17 and 18: CHAPTER 2 A Crash Course in Anomaly
Page 19 and 20: How can you achieve similar results
Page 21 and 22: cause most anomaly detection techni
Page 23 and 24: CHAPTER 3 Modeling and Predicting A
Page 25: (say, the size of the drill bit), a
Page 29 and 30: ally decaying window. This is made
Page 31 and 32: ack again, meaning that they smooth
Page 33 and 34: emains consistent across all of the
Page 35 and 36: Evaluating Predictions One of the m
Page 37 and 38: No. You’ve stumbled into statisti
Page 39: As an aside, there’s a rumor goin
Page 42 and 43: Dealing with Trend Trends break mod
Page 44 and 45: pletely out of phase with the seaso
Page 46 and 47: mon situations. You can probably gu
Page 49 and 50: CHAPTER 5 Practical Anomaly Detecti
Page 51 and 52: If you can get close to that, you m
Page 53 and 54: We previously discussed how frequen
Page 55 and 56: First, observe how odd this metric
Page 57 and 58: After differencing, it looks like w
Page 59 and 60: Perhaps, instead of differencing, w
Page 61 and 62: CHAPTER 6 The Broader Landscape As
Page 63 and 64: We could apply a EWMA control chart
Page 65 and 66: figure out how unlikely it was for
Page 67 and 68: predict or find structure in data w
Page 69 and 70: Graphite and RRDTool Graphite and R
Page 71 and 72: APPENDIX A Appendix Code Control Ch
Page 73 and 74: var ma = new movingAverage(0.5); ma
Page 75: Acknowledgments We’d like to than

Anomaly Detection for Monitoring

Create successful ePaper yourself

Delete template?

Save as template?