Anomaly Detection for Monitoring

More documents

Recommendations

Info

In this plot, you’ll notice that throughput increases with an approximately linear trend, with a slight drop around the two-thirds mark. In the middle of the plot, you can see large drops in throughput. These drops, which happen on a short timescale, are the anomalies that we’ve decided to try detecting. Now that we know the scope of our anomaly detection problem, let’s try to use the decision tree to determine what kinds of methods might be useful. For “Detection Time,” we know we’re interested in a short timescale method so we’ll follow the path on the right. With a linear trend, we don’t have a fixed mean, so we’ll follow the “no” path for “Known mean.” Now we’re down to “Variability.” The metric is highly variable, and exponential smoothing tends to generate better results in situations like this. If our data set didn’t have trend, we may have ended up with a fixed control chart method, which is simpler than using exponential smoothing. Previously, we mentioned that trend is something that can be removed with a first difference or derivative. Let’s try that out. 48 | Chapter 5: Practical Anomaly Detection for Monitoring
After differencing, it looks like we eliminated the trend. Great! Now we have a known, fixed mean at 0 and we can use an ordinary control chart. Because the spread changes over time and isn’t something we know a priori, we’ll use an EWMA control chart, which uses a EWMA to draw the limits based mostly on recent data values. A Worked Example | 49
Page 3 and 4:
Anomaly Detection for Monitoring A
Page 5 and 6: Table of Contents Foreword. . . . .
Page 7: Foreword Monitoring is currently un
Page 10 and 11: tion” to anomaly detection is imp
Page 12 and 13: Why do we assume these things? Are
Page 14 and 15: Conclusions If you are like most of
Page 17 and 18: CHAPTER 2 A Crash Course in Anomaly
Page 19 and 20: How can you achieve similar results
Page 21 and 22: cause most anomaly detection techni
Page 23 and 24: CHAPTER 3 Modeling and Predicting A
Page 25 and 26: (say, the size of the drill bit), a
Page 27 and 28: lies. To fix this problem, the cont
Page 29 and 30: ally decaying window. This is made
Page 31 and 32: ack again, meaning that they smooth
Page 33 and 34: emains consistent across all of the
Page 35 and 36: Evaluating Predictions One of the m
Page 37 and 38: No. You’ve stumbled into statisti
Page 39: As an aside, there’s a rumor goin
Page 42 and 43: Dealing with Trend Trends break mod
Page 44 and 45: pletely out of phase with the seaso
Page 46 and 47: mon situations. You can probably gu
Page 49 and 50: CHAPTER 5 Practical Anomaly Detecti
Page 51 and 52: If you can get close to that, you m
Page 53 and 54: We previously discussed how frequen
Page 55: First, observe how odd this metric
Page 59 and 60: Perhaps, instead of differencing, w
Page 61 and 62: CHAPTER 6 The Broader Landscape As
Page 63 and 64: We could apply a EWMA control chart
Page 65 and 66: figure out how unlikely it was for
Page 67 and 68: predict or find structure in data w
Page 69 and 70: Graphite and RRDTool Graphite and R
Page 71 and 72: APPENDIX A Appendix Code Control Ch
Page 73 and 74: var ma = new movingAverage(0.5); ma
Page 75: Acknowledgments We’d like to than
show all

Anomaly Detection for Monitoring

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?