Back Testing Technical Strategies

Discussion in 'Strategy Development' started by Soros, May 16, 2007.

  1. Soros


    There is a lot of literature debating weather technical analysis really works or not and if it does which indicators or theories are useful and which are just black magic? One of the best ways to address this question is to use today’s powerful computers together with specialized software and test the indicators, theories and strategies that have been developed and published over time. Over a period of one year together with a colleague we tested a number of renowned as well as some not so famous indicators and strategies. This was done using Metastcok Professional 9.0 as well as Trade sim, add plug in software written for Metastock to analyse the statistical results. The strategies were tested on US stocks based on 20 years of historical data from AMEX, NASDAQ and the NYSE. Indicators tested included Moving Averages, MACD, RSI, Stochastic, Volume, McClellan Oscillator & Summation Index, Candlestick Formations, Range Beak Outs, Volatility Breakouts, Support Resistance, Rate of Change or Momentum, Relative Strength, Average True Range Stops and Gaps among others. The data was then compiled and the parameters for the variables were optimised. The problem of optimisation and curve fitting was recognized. A way that we used to get around this issue of curve fitting was by picking the best parameters for variables, which also had good parameters around them. For example if the 35 day exponential moving average worked well but the 34 and 35 day didn’t, we would conclude that this was not significant and looked for other parameters which were. We also ran a backrest for the 20 years as well as an out of sample test for around 6 months. The results we obtained were very interesting and in lighting. After having read many technical analysis books and having seen such strategies advocated by authors who proclaimed them to be reliable, it was interesting to see which indicators and strategies actually worked in the past and possibly had a chance of working in the future. As opposed to which indicators showed poor results.

    From our results we found such popular indicators as MACD, Stochastic and candlesticks as having little value or being wrong more times then they were right in predicting future price action. The analysis showed that based on are algorithms and the criteria specified accepted indicators such as Moving Averages, ROC and RSI had returns and draw downs that were unimpressive, but in general could have been utilized in a profitable manner over the long term. Further more from our analysis we found that pretty much most indicators did poorly to average when used by them selves. When indicators were combined with other such technical indicators or specific entry and exit rules based on price, we started to obtain some attractive results. Indicators when combined together with others or specific price action proved to be quiet valuable. With out going into too much statistical detail indicators such as Volume, Momentum, RSI, Range Break Outs, Support Resistance, Rate of Change or Momentum, Relative Strength, Average True Range Stops showed promising results when combined into to specific groups or with the right buy and sell criteria. Indicators such as Gaps McClellan Oscillator & Summation Index and Volatility breakouts gave mixed results. Gaps worked well if you were able to get in at the open of the gap day and predict weather the day would end up being a true gap day rather then one, which moves through the price of yesterdays high. Unfortunately this is not possible in real life. The indicators and strategies that worked best in back testing also ultimately proved to work best in forward testing. Although the results were less impressive which was to be expected they still proved to be statistically significant.

    Some of the problems in the tests included using specific entry and exit rules. Different rules or other criteria if added may have improved some of the results of the poor performing indicators and strategies. Also most likely not all the curve fitting was eliminated an indication of this is the forward test results being worse then the ones in the back tests. Other criteria such as market capitalization were not considered since they are not part of the technical analysis world but we believe could have important implications for the use of such strategies. Based on are results we consider technical analysis as a valid form of analysing and trading the US stock market and quiet possibly other financial markets. Further more we observed that if the right indicators are combined as well as specific buy and entry rules it is possible to create high return generating strategies. Overall the testing was very in lighting and a lot of knowledge was gained in regards to some of mainstream as well as less used technical indicators and strategies in technical analysis.
  2. How did you calculate transaction costs and slippage in your study?