Edition

Recommendations

Info

IFTA JOURNAL 2017 EDITION Trend Without Hiccups—A Kalman Filter Approach By Eric Benhamou, Ph.D., CFTe, CAIA, CMT, MFTA Eric Benhamou, Ph.D., CFTE, CAIA, CMT eric.benhamou@ibinence.com Ibinence, 35 Boulevard d'Inkermann 92200 Neuilly sur Seine, France +33 1 73 63 70 09 Abstract Have you ever felt miserable because of a sudden whipsaw in the price that triggered an unfortunate trade? In an attempt to remove this noise, technical analysts have used various types of moving averages (simple, exponential, adaptive one or using Nyquist criterion). These tools may have performed decently, but we show in this paper that this can be improved dramatically thanks to the optimal filtering theory of Kalman filters (KF). We explain the basic concepts of KF and its optimum criterion. We provide a pseudo code for this new technical indicator that demystifies its complexity. We show that this new smoothing device can be used to better forecast price moves as lag is reduced. We provide four Kalman filter models and their performance on the SP500 mini- future contract. Results are quite illustrative of the efficiency of KF models, with better net performance achieved by the KF model combining smoothing and extremum position. The author would like to acknowledge support from Thomson Reuters. Data are from Thomson Reuters Eikon, while screenshots and source codes are done with TR Eikon Trading Robot. The author also thanks Denis Dollfus for fruitful conversations. The author also warns that the views and opinions expressed in this article are those of the author and do not necessarily reflect the official position or policy of Thomson Reuters. Introduction Have you ever felt angry because a sudden price whipsaw triggered an unfortunate signal and a resulting bad trade? Prices have inherent blips and jerks that are not easy to control. Moreover, prices are inputs for technical analysis indicators. This can result in corrupted or non-efficient indicators. In an ideal world, one would like prices heading to a clear direction. Remember the old adage: “ trade with the trend” . But in real life, price hiccups create noise and perturb the signal. A first attempt to remove these yanks and jolts is to smoothen prices with moving averages. However, moving averages suffer from two flaws: lags and no dynamics. The first drawback—delay in moving average response—is widely known as moving averages used past data. Adaptations to moving averages have been suggested (exponential, adaptive, zero lag or Nyquist criterion based moving averages). Dürschner (2012) suggested the use of Nyquist criterion to create moving average 3.0 with no lag. This is intellectually very enticing, as the lag is completely removed. This improves moving averages from Patrick Mulloy (Mulloy, 1994) with zero lag or the attempts by John Ehlers to provide sophisticated moving averages (Ehlers, 2001a or Ehlers, 2001b). But this does not address the second problem of capturing price dynamics. What we mean by price dynamics is the price movement. If we can identify that prices are moving upwards (respectively downwards), then a good guess for the next price observation should be higher (respectively lower) than the current price. Let us pause for a moment and imagine that instead of prices, we were looking at car position using a GPS. We measure the car position with a GPS but with some noise, as the signal is not perfectlyaccurate. Could we capture the car dynamics to compute the best guess at next time step 1 and hence, reduce noise in car position? The answer is yes! And guess what, this is what your car GPS is doing. This theory simply explained is referred to as Kalman filter, from its inventor, Kalman (1960), shortened to KF in this paper. It was created for the spatial industry to remove noise and capture shuttle movements. In a scientific way, the Kalman filter is an efficient recursive filter that estimates the state of a dynamic system from a series of incomplete and noisy measurements to estimate the best forecast according to an assumed distribution. In the original paper, Kalman assumes a Gaussian distribution of noise, but an extended version can now cope with more advanced distribution (see Wikipedia, Kalman Filter). In this article, we first revisit moving averages and then present different Kalman filter models and their implementation to create trading strategies. We then provide performance results for our four KF models on one year of data of the E-mini-SP continuation future. Motivation for Smoothing Smoothing prices is natural. The basic idea is to remove noise from prices to better identify important patterns or trends. Remember, when we trade, we want the big picture. So smoothing enables us to remove bumps, bangs, bounces, and shocks and get an average clean signal. If we believe that prices do not follow a random walk model, the smoothened signal provides us a clear directional signal. Impact for Trading Strategies Conversely, if we do not smoothen prices, we could act on tugs, wrenches, or snatches that are against the trend and result in bad trades. Smoothing is the right way! But we need to be careful. If we smoothen with lag (one of the major drawbacks of moving averages), we act with delay and enter trades too late, potentially facing reverse direction markets. In an ideal world, we would like the smoothing technique to have zero lag and to provide a first move advantage. PAGE 38 IFTA.ORG
ND METHODS IFTA JOURNAL 2017 EDITION RIAL AND METHODS ing Average Material and Methods ERIAL of Moving AND Average METHODS Introduction to Kalman Filter ERIAL AND METHODS Review of Moving Average ual w g of averages moving Moving averages Average ew of Moving Average Basic concepts The usual moving averages sophisticated dynamics, sual o ERIAL remove way moving AND to remove averages noise METHODS The in usual prices way is to with remove noise prices is with moving Kalman like filter a is linear a recursive Gaussian algorithm model, that was KF invented is the optimal choice a noise in prices is with moving averages. Let us denote weights sual moving averages sophisticated dynamics, in the 1960s like to a track linear a moving Gaussian target, model, remove KF any noisy the optimal choice a ew averages. Let us denote weights by w i for the time T i , where goes sual of or e the way Moving , where time to remove Average most , where noise between goes between goes in prices 0 and between is with N Then, to the to moving . Then, . Then, averages. the the moving Let us sophisticated efficient computational given average denote by is weights dynamics, is given by measurements like solution of a linear its position, for Gaussian finding and predict model, the its model KF future is parameters. position. the optimal choice and sual way to remove noise in prices is with moving averages. Let us denote weights by sual moving averages sophisticated dynamics, most dynamics, efficient like like computational a In linear a finance, linear Gaussian KF Gaussian solution has been model, used model, for finding by KF the KF is asset the the is management the optimal model optimal parameters. choice industry choice and and the the for for the the time time , where where goes goes between between to to . Then, . Then, the the moving moving average average KF most has is given is given efficient also by (EQ by been computational 2.1) for used various over purposes. the solution last KF decade for is an finding optimal by different the choice model in authors. many parameters. cases Martinelli and Rh MATERIAL AND METHODS usual way to remove noise in prices is with moving (EQ 2.1) most averages. most efficient efficient Let us denote KF computational has weights also (EQ been solution and 2.1) solution used does over for at for least finding the finding better last the decade than the model a model moving by parameters. different parameters. average authors. smoothing. Martinelli and Rho Review of Moving Average lag is (Martinelli, KF (EQ has (EQ 2.1) also 2.1) 2006) been Dao used and et al. over (Martinelli (Bruder, the Dao, last decade and Richard, Rhoads, by and different Roncalli, 2010) authors. 2011) used and Kalman Martinelli (Dao, filter and Rhoad to f for the time , where Whose goes lag between is to . Then, the moving average is given sophisticated by The usual moving averages 2011) dynamics, showed that like for a linear price following Gaussian random model, walk KF with is the noise, optimal choice and se e lag is KF KF has has also also been (Martinelli, been used used over over 2006) the the last and last decade (Martinelli decade by different by different and Rhoads, authors. authors. 2010) Martinelli Martinelli used and Kalman and Rhoads Rhoads filter in in to fi optimal (Martinelli, guess 2006) for KF trading is and equivalent (Martinelli strategies to the on optimal and stocks. Rhoads, exponential Haleh 2010) et moving al. used (2011) average Kalman used filter Extended to findK The usual way to remove noise in prices is with moving averages. Let us denote weights (EQ sophisticated most 2.1) efficient dynamics, computational like a linear solution Gaussian for finding model, KF the is model the optimal parameters. choice and the (EQ 2.2) (EQ 2.2) with parameter equal to Kalman gain. However, for more (Martinelli, 2006) optimal 2006) and and guess (Martinelli for trading and and strategies Rhoads, Rhoads, on 2010) stocks. 2010) used Haleh used Kalman Kalman et al. filter (2011) filter to used find to find Extended the the K se lag by is for the time , where goes between to . Then, the moving average filter optimal is given (EQ most for guess by (EQ 2.2) forecasting for 2.2) sophisticated trading (EQ 2.2) stock strategies dynamics, prices, on combining like stocks. a linear Haleh Gaussian technical et al. model, (2011) and KF fundamental used is the Extended data Kal KF efficient computational solution for finding the model parameters. sophisticated has also dynamics, been used like over a linear the last Gaussian decade model, by different KF is the authors. optimal Martinelli choice and and the Rhoa optimal choice and the most efficient computational solution for (EQ 2.1) o a moving average If we of do a a moving average, of the a moving optimal equation averaºge, guess filter for (2.1) becomes the filter for equation trading for for forecasting strategies forecasting finding on stock the on stocks. model stocks. prices, prices, parameters. Haleh Haleh combining combining et al. et (2011) al. (2011) technical technical used used Extended and and Extended fundamental fundamental Kalman Kalman data. data. KF has also been used over the last decade by different authors. Martinelli and Rhoads in T showed (Martinelli, most that efficient it outperformed 2006) computational and (Martinelli solution regression for finding and Rhoads, and the model neural 2010) parameters. networks. used Kalman Ernie filter Chan to find do a moving average (2.1) becomes (EQ 2.2) Whose lag is of a moving average, the the equation (2.1) (2.1) becomes In finance, KF has also been used over the last decade by g average of a moving average, the equation filter filter for (2.1) for forecasting becomes showed showed (Martinelli, stock that that stock it prices, 2006) it outperformed (EQ 2.3) different prices, and combining (Martinelli authors. regression Martinelli and technical Rhoads, and and Rhoads and 2010) and neural fundamental used in (Martinelli, Kalman networks. filter 2006) data. Ernie data. to Ernie They find Chan They the Chan (2 suggested optimal KF has also using guess been KF for used trading over the pair correlation strategies last decade on by trading, stocks. different while Haleh authors. Cazalet al. (2011) and and used Rhoads Zheng Extended in (2014) Ka u (EQ (EQ 2.3) do a moving average of a moving average, the equation (2.1) becomes (EQ optimal 2.2) 2.3) guess and for (Martinelli trading strategies and Rhoads, on stocks. 2010) Haleh used et Kalman al. (2011) filter used to Extended find Kalman showed that that suggested it it outperformed (Martinelli, using using KF 2006) the KF regression for optimal for and pair (Martinelli correlation guess and and for and trading neural Rhoads, neural trading, strategies networks. 2010) networks. while used on stocks. Cazalet Ernie Kalman Ernie Haleh Chan and filter et Chan Zheng to al. (2013) find (2013) (2014) the used filter forecasting stock prices, combining technical and fundamental data. e corresponding lag is for hedge (EQ 2.3) the corresponding lag is filter fund for forecasting replication. And is the corresponding lag is (EQ 2.3) (2011) used stock Extended prices, Kalman combining filter technical for forecasting and fundamental stock prices, data. They optimal guess for trading strategies on stocks. Haleh et al. (2011) used Extended Kalman If we do a moving average of a moving average, the equation suggested (2.1) becomes using using for KF for hedge KF for showed for pair fund fund pair correlation replication. that combining it outperformed trading, technical trading, while and regression while fundamental Cazalet Cazalet and and data. neural and Zheng They Zheng networks. showed (2014) (2014) that used Ernie used KF Chan KF (2 In a general showed way, that it onding the corresponding lag is lag is (EQ 2.3) it Kalman outperformed outperformed filter regression is regression considered and and neural a neural linear networks. networks. dynamic Ernie Ernie system Chan Chan given (2013) filter for forecasting stock prices, combining technical and fundamental data. They by for (EQ 2.4) (EQ 2.4) for hedge hedge fund In fund replication. In a general a suggested (EQ 2.4) way, using 2.4) (2013) Kalman KF suggested for filter pair is using is correlation considered KF for pair trading, a correlation linear while dynamic Cazalet trading, system and while Zheng given (2014) by by use suggested showed that using it KF outperformed for pair correlation regression trading, and while neural Cazalet networks. and Zheng Ernie (2014) Chan used (2013) KF And the corresponding lag is Cazalet and Zheng (2014) used KF for hedge fund replication. (E In (EQ 2.4) In a general way, Kalman filter is considered a linear dynamic an easily derive similar formula for for a recursive moving In a general moving average a general way, average at the way, Kalman for for hedge hedge at order Kalman filter fund fund the kth: order (EQ filter is replication. replication. (EQ suggested using considered KF kth: 2.4) is considered for pair correlation a linear a linear trading, dynamic dynamic while system Cazalet system given and Zheng given by (2014) by used KF n easily derive a similar We can formula easily derive for a recursive a similar formula moving average for a recursive at the (EQ order moving 2.4) kth: system given by (E In In for a a hedge general fund way, way, replication. Kalman filter filter is considered is considered a linear a dynamic linear dynamic system given system by given by (EQ average at the order kth: an easily derive a similar formula for a recursive moving average at the order (EQ kth: 2.5) (EQ (EQ 2.5) 2.5) (EQ (EQ 3.1) (EQ 3.1) (E Where is the state transition matrix, the measurement matrix, the 3.1) mode We can easily derive a similar formula for a recursive moving average at the order Where In a general kth: is the way, state Kalman transition filter is matrix, considered a the linear measurement dynamic system matrix, given by (EQ 3.1) the model (EQn erive esulting a lag similar is formula for a recursive moving average Where at the (EQ order is the 2.5) kth: state transition matrix, the measurement matrix, (EQ (EQ 3.2) 3.2) the model 3.2) sulting lag is (EQ the 2.5) state vector, the measurement vector, the measurement (EQ noise, 3.1) 3.2) the state vector, Where the is the measurement state transition vector, matrix, H the the measurement noise, (EQ and resulting lag is Where The resulting lag The is resulting lag is Where is is the the state the Where state state transition vector, is the transition (EQ matrix, 2.5) matrix, state matrix, w transition the measurement the the measurement measurement matrix, matrix, the model the noise, t the model matrix, noise, X the vector, t the measurement the state vector, matrix, measurement Y t the the model noise, noise, a (EQ the independent noise, 3.2) (EQ Where 2.6) is white the state noises transition with zero matrix, mean the and measurement their variance matrix, matrices the given model by n the independent measurement white noises vector, with zero v the state vector, (EQ the (EQ 2.6) t the mean measurement and their noise, variance w t and matrices v t the given by the independent Where the state measurement is 2.6) white vector, the state noises transition the measurement vector, with matrix, zero mean vector, the measurement and the their measurement matrix, variance noise, noise, the matrices model and and noise, given by is the state vector, (EQ (EQ 2.6) the measurement independent white vector, noises with zero measurement mean and their variance respectively. the state vector, noise, and 2.6) , respectively the measurement is the vector, drift of the measurement state vector, noise, respectiv an respectively. the independent matrices given by Q and R respectively. c t , respectively d t , the mpled with equidistant time steps . Formulae (EQ.2.6) can be easily the state vector, white , respectively noises the measurement with zero , mean is vector, the and drift their the variance of measurement the matrices state noise, vector, given by and respectively and pled cit Lag with Computation equidistant time steps . Formulae the independent (EQ.2.6) white respectively. can be noises easily with drift , zero respectively of the mean state vector, and their , respectively is variance the drift the matrices measurement of the state given vector. vector, by and respective the independent white the independent noises with white zero noises mean with and zero their mean variance and their matrices variance given matrices by given and by t it Lag Lag Computation Explicit Lag Computation Explicit Lag Computation measurement the respectively. independent vector. The white corresponding The , The respectively noises corresponding with Kalman zero , is mean the filter Kalman and drift is: their of filter the variance state is: is: matrices vector, respectively given by and the icit terms pled Lag of with Computation first order value, as follows: rms of first equidistant order Prices value, time are as sampled steps follows: with equidistant . Formulae respectively. time (EQ.2.6) steps Formulae measurement can be , easily respectively. vector. The respectively. , (EQ 2.6) , , is corresponding respectively the drift of , Kalman the is the state drift filter vector, of is: the respectively state vector, the respectively Prediction , step: is the drift of the state the uidistant time steps .(EQ.2.6) Formulae (EQ.2.6) can be easily can be easily computed in terms of measurement first Prediction respectively. (EQ order vector. The corresponding Kalman filter is: step: , respectively , is the drift of the state vector, (EQ.3.3) respectively the (EQ(E 2.7) terms of first order value, value, as follows: as follows: (See Proof A.1:) measurement vector. Prediction Prediction measurement The (EQ step: corresponding 2.7) step: vector. vector. The The Kalman corresponding corresponding filter Kalman is: Kalman filter is: filter is: (EQ.3.3) (E order value, as follows: measurement vector. The corresponding Kalman filter is: .1:) With With (EQ.3.4) (EQ 4 :) (EQ 2.7) 4 (EQ 2.7) mputation Prediction step: With With Prediction step: (EQ.3.3) (EQ.3.3) (EQ.3.4) (EQ (E , if we combine recursive moving averages, 4 it is easy Prediction to find back step: Correction the results step: (EQ.3.3) (EQ(E 4 Correction step: Correction step: (EQ.3.5) (EQ.3.5) .1:) 4 f we combine recursive Furthermore, moving if we averages, combine recursive it is easy moving to find averages, Correction With back the it is results n e recursive the case moving of a averages, it is easy to find back the results With With With step: (EQ.3.4) (EQ (EQ.3.4) easy moving to find average back the of moving results of average, Mulloy. In the the only case possible of a moving choice With if we combine recursive moving averages, it is easy With Correction step: (EQ.3.5) to find back With the results Correction With step: (EQ.3.4) (EQ the moving case average of a of moving average average, of moving the only of average, possible moving the choice average, only Correction possible the choice only step: possible with zero choice (EQ.3.5) whose coefficient sum is equal to 1 is the double moving average: With With Kalman With Kalman Kalman gain gain gain (EQ.3.6) (EQ(E cient the sum case is equal of a to moving lag 1 is the whose double average coefficient moving of average: moving sum is equal average, to Correction 1 the is the only double possible step: moving With choice (EQ.3.5) hose coefficient average: sum (See is equal Proof to A.2) 1 is the double With With moving average: - - With With Kalman With With Kalman gain With gain Kalman gain (EQ.3.6) (EQ.3.6) (EQ.3.7) (EQ (EQ 2.8) With (EQ 2.8) (E With Kalman gain (EQ whose coefficient sum is equal to 1 is the double moving average: With With (EQ.3.7) (E .2) - 4 With Kalman gain (EQ With (EQ 2.8) 2.8) With (EQ.3.7) (EQ.3.6) (EQ - With Kalman gain (EQ 2.8) (EQ.3.6) average (if we impose the additional constraint that the third With (EQ.3.7) triple ) moving average And for (if the we triple impose moving the average additional (if we constraint impose the that additional the third 6 .2) rage coefficient is 1), we have With KF works in a two-step process (EQ.3.7) constraint that the third order recursive moving average 6 (prediction and correction 6 iple ve moving average - coefficient coefficient (if is we 1), we is impose 1), have we (See the have (EQ Proof additional 2.9) A.3) constraint that the third steps). The algorithm is recursive 6 and can run in real time, using riple moving average (if we impose the additional constraint that the third only the present input measurements, 6 the previously calculated 6 moving average coefficient is - 1), we have (EQ 2.9) state, and its uncertainty matrix. e moving average coefficient is 1), we have 6 .3) ilter - (EQ 2.9) - (EQ 2.9) 6 IFTA.ORG PAGE 39 .3) )
Page 1 and 2: IFTA Journal 2017 Edition A Profess
Page 3 and 4: IFTA JOURNAL 2017 EDITION Letter Fr
Page 5 and 6: IFTA JOURNAL 2017 EDITION Letter Fr
Page 7 and 8: IFTA JOURNAL 2017 EDITION Table 1:
Page 9 and 10: IFTA JOURNAL 2017 EDITION Whipsaws
Page 11 and 12: IFTA JOURNAL 2017 EDITION Table 9.
Page 13 and 14: IFTA JOURNAL 2017 EDITION Pull back
Page 15 and 16: IFTA JOURNAL 2017 EDITION Reference
Page 17 and 18: IFTA JOURNAL 2017 EDITION Figure 1.
Page 19 and 20: IFTA JOURNAL 2017 EDITION was corre
Page 21 and 22: IFTA JOURNAL 2017 EDITION JGB futur
Page 23 and 24: IFTA JOURNAL 2017 EDITION Crude Oil
Page 25 and 26: IFTA JOURNAL 2017 EDITION JGB futur
Page 27 and 28: IFTA JOURNAL 2017 EDITION The Compo
Page 37 and 38: IFTA JOURNAL 2017 EDITION Discussio
Page 39: IFTA JOURNAL 2017 EDITION suggested
Page 43 and 44: IFTA JOURNAL 2017 EDITION Pseudo co
Page 45 and 46: IFTA JOURNAL 2017 EDITION Table 1:
Page 47 and 48: IFTA JOURNAL 2017 EDITION Discussio
Page 49 and 50: IFTA JOURNAL 2017 EDITION Wrestling
Page 51 and 52: IFTA JOURNAL 2017 EDITION Research
Page 53 and 54: IFTA JOURNAL 2017 EDITION In additi
Page 55 and 56: IFTA JOURNAL 2017 EDITION Figure 11
Page 57 and 58: IFTA JOURNAL 2017 EDITION The RSI (
Page 59 and 60: IFTA JOURNAL 2017 EDITION The RSI (
Page 61 and 62: IFTA JOURNAL 2017 EDITION Conclusio
Page 69 and 70: IFTA JOURNAL 2017 EDITION The Signi
Page 71 and 72: IFTA JOURNAL 2017 EDITION event, sh
Page 73 and 74: IFTA JOURNAL 2017 EDITION most nota
Page 75 and 76: IFTA JOURNAL 2017 EDITION Every typ
Page 77 and 78: IFTA JOURNAL 2017 EDITION Formula o
Page 79 and 80: IFTA JOURNAL 2017 EDITION Figure 14
Page 81 and 82: IFTA JOURNAL 2017 EDITION Forecasti
Page 83 and 84: IFTA JOURNAL 2017 EDITION Trend Ide
Page 85 and 86: IFTA JOURNAL 2017 EDITION The Calcu
Page 89 and 90: IFTA JOURNAL 2017 EDITION The Perio
Page 91 and 92:
IFTA JOURNAL 2017 EDITION Results f
Page 93 and 94:
IFTA JOURNAL 2017 EDITION 4. What i
Page 95 and 96:
IFTA JOURNAL 2017 EDITION Conclusio
Page 97 and 98:
IFTA JOURNAL 2017 EDITION Construct
Page 99 and 100:
IFTA JOURNAL 2017 EDITION Results A
Page 101 and 102:
IFTA JOURNAL 2017 EDITION A Point-a
Page 103 and 104:
IFTA JOURNAL 2017 EDITION The ticke
Page 105 and 106:
IFTA JOURNAL 2017 EDITION Figure 3.
Page 107 and 108:
IFTA JOURNAL 2017 EDITION An Empiri
Page 109 and 110:
IFTA JOURNAL 2017 EDITION q m=5 m=7
Page 111 and 112:
IFTA JOURNAL 2017 EDITION As you ca
Page 113 and 114:
IFTA JOURNAL 2017 EDITION The F-rat
Page 115 and 116:
IFTA JOURNAL 2017 EDITION model has
Page 117 and 118:
IFTA JOURNAL 2017 EDITION allocatio
Page 119 and 120:
IFTA JOURNAL 2017 EDITION delved de
Page 121 and 122:
IFTA JOURNAL 2017 EDITION Author Pr
Page 123 and 124:
IFTA JOURNAL 2017 EDITION Prashant
show all

Edition

Create successful ePaper yourself

Delete template?

Save as template?