Technical analysis is frequently used as a somewhat good way to design trading strategies provided one has done a serious backtesting, does not fall into the overfitting trap, knows the probabilities well and sticks to the odds. I decided to investigate the combination of technical analysis and machine learning to design seriously profitable trading strategies. The Quantitative trading strategy i talk about in this post is being actively traded in the investment club I belong to. So unfortunately, I won’t be able to explain the ins and outs of the strategy. However, I will share the backtesting results and mention a few elements. So in a nutshell, the strategy is pretty much simple. It is based on using daily market data for particular asset clases: Futures contract for commodities and indices. Here, I will show the results for Gold futures. The trading strategy came about as I was looking at daily price chart (candlesticks) for multiple asset classes. I realized that whenever the daily closed price was above the Ema7, price had the tendency to go upwards the following days. So I decided to test it. The data I used for gold goes from 2016 to October 2018. See price chart below. I got the data from my broker (which I won’t mention here) but you can obtain such daily price data for gold futures easily online. You can see in the chart below that the price goes up and down over the last couple of years. I used the statistical programming language called R to carry out the research and backtest of the strategy. The strategy consisted of buying 1 gold contract the next day if the current day close price is above the Ema7. I would also short 1 gold contract the next day if current day closing price is below EMA7. No trade is carried/kept till the next day which means that all trades are open and closed the same day. So the EMA7 was the only technical indicators used for the strategy. The next bit of it was to use machine learning. I used a particular algorithm to tune the strategy such that the profitability would increase and the duration and trough of the drawdowns would decrease. The figure below shows the resulting equity curve excluding commission costs. Equity curve of the Gold machine learning enhanced Ema7 strategy This equity curve looks great and this strategy turns out to be highly profitable. But the equity curve is not enough, it is important to look at other risk elements like drawdowns, trades stats, sharpe ratio etc… Here I will focus on the drawdown which can be quite significant and intimidating. If proper analysis of the drawdowns is not done, one can be exposed to the risk of failure by now not understanding fully the returns and stopping the strategy prematurely. In the figure below, we can see that the drawdowns (red line chart) can such in some instances -50% of the starting capital which in this case was 10000$. The length and trough varies obviously depending on market volatility. Also as you can obverse on the distribution, the strategy has a lot of zero $ trades. These are times when no trading happen and there given certain signals, the algorithm would not execute a trade. Overall the strategy looks really robust and highly profitable delivering close to 800% return in no less than 2.5 years.
So how does this “not fall into the overfitting trap” when it looks exactly like an overfitting trap? There are various software programs that find strategies that look very similar to this and with similar equity curve, then immediately stop working because they’re overfit. I’ve seen so many of these that just taking a look at yours immediately tells me “oh no, not another one of these.” And really, you can’t consider any strategy not overfit when it wasn’t even back tested for 10-20 years, and forward tested or ran live for another year. I have never seen a strategy that was backtested for less than 3 years and wasn’t overfit and wouldn’t fail soon afterwards, and I’ve sifted through more than 600 million strategies.
What guru said. You should have no confidence unless it performs well on out-of-sample. VERY easy to enter curve-fit realm even while gently "training" and "tuning." Or at least, trade only small amounts while forward testing in the market.
does your curve fitting seem to show a lessening most recently from trades ~550 to nearly 700, where cumulative % return seems to be plateauing a it? Could reflect an exhaustion in price growth?
how can you generate so many trades in 2 years using daily data unless u r churning every single day?
While on the topic of overfitting, have you repeated the backtest for all moving averages? For example from Ema2 all the way up to Ema50? Another question is related to the title of this thread: in what way is Machine Learning involved in this?
I understand you guys. I have to say that overfitting is trap that one can easily fall into. I indeed fitted a more than 2 states markov model algorithm to the 2 years of daily market data. Now the equity curve I am showing you is assuming I was making daily trades knowing the output of the markov model which would have been fitted on past data up to yesterday's. I am certainly putting my money where my mouth is since I am currently running 4 other strategies live. \ The equity curve seems to be plateauing because there are many period of time where the strategy recommends to take no trade (Flat line). I will share some results in the weeks to come. Regarding this post, I am still developing this strategy on Gold further and doing some Bayesian statistics on it.