Hi guys I have been working on equity systems for a few months now. When creating a fully mechanical system, what are the stand out stat's that make you say "yeah this is probably going to make money"? I have been trying out different ideas in Amibroker, some of which have returned great profits over the last few years even unoptimised. Amibroker does not give me the details I think necessary to evaluate a system, I have to copy and paste the trades into an excel template which works out what I want to know. This is what I look for (and have not found yet): an average of 8-9 profitable months per year over the last 5 years. Assuming a constant position size, what is the average monthly win/loss? in dollars. Small standard deviation of monthly win/loss, maybe about 1.5x the mean. Example of good system performance: Position Size: $5000 per trade Average win/loss per month: $700 Standard Deviation of Monthly win/loss: $900 Before I was mainly looking at the metrics that amibroker gave you, such as expectancy per trade, annual return and max drawdown. What I found when I looked deeper into the backtest data for my best system was: > Below average trades per month in 2005-2007, 1.2% Exp > Very few trades during the GFC (which is good), ~-0.2% Exp. > A whoppingly massive amount of trades during 2009, 4.8% Exp. > Back to average performance in 2010-2011 1% Exp. This basically tells me that the system, although profitable for a large portion of the last 5 years, is fantastic for 2009 market conditions (a bounce out of a crash) but pretty average for sideways markets (2010-2011) and even boom times (2005-2007). So if I took this system live today, my historical average expectancy is 2% per trade, but can I realistically expect to encounter the same market conditions as I have back tested over in the last 5 years? No, 2009 is not likely to happen again soon is it? and by then you could expect the current market inefficiencies exploited by my system to have disappeared into the ether. Sorry for the wall of text, my question to you is: What metrics really stand out to you when back testing to indicate a system that would be worth taking live? and How do you determine the relevant time frames for effective back testing? I am almost exclusively looking for swing trading systems (1-2 week trades) and not looking at day trading systems (too expensive) or LT trend following systems.
hi nathansz, <a href="http://www.datatime.eu/public/gbot/Strats%20G-BOT/Strategy_CT_T_LS_E/Strategy_CT_T_LS_E.htm" target="blank" >Here</a> is a sample of metrics i personally use and find sufficient for precise assessment. ( See bottom of page. ) Tom
Here are some previous threads that may add to your thinking, too. http://www.elitetrader.com/vb/showthread.php?s=&threadid=204564&perpage=6&pagenumber=1 http://www.elitetrader.com/vb/showthread.php?s=&threadid=204548 http://www.elitetrader.com/vb/showthread.php?s=&threadid=204507 http://www.elitetrader.com/vb/showthread.php?s=&threadid=200527
You have to determine what stats you find most useful on your own. The absolute most important thing is that you do not allow any data peeking at all! That's forwards or backwards. Any data peeking renders results totally unreliable. Furthermore, you must understand that even out of sample testing can and often does show random results when dealing in speculative systems. IOW take all results of speculative system building with a huge grain of salt. That is the most important metric, if you can call it that.
The fantastic 2009 bounce you mentioned, basically a "V" bottom, is an aberation. Not likely to be replicated anytime soon in an index (but indeed possible with individual issues). How does that impact your back-testing?
It gives me phenomenal numbers on almost every system I conceive of. 2010 until today seems to be the most important timeframe for a swing trading system to perform on if you expect to get anywhere close to your backtested expectancy in the next year.