Gooday everybody, I would like to know if my system is ROBUST, but I do not know what specific performance measurements to look at. I use tradestation 2000i for my backtesting. Any thoughts would be greately apprciated. Thank you. "Cut your losses and let your profits run"
Post a system report. Did you optimize any variables? This can be useful but will curve fit your system results. How is your p/l distributed over your trades, if you have only a few trades that account for a majority of the profit that is bad. Are you using TS coded or built in stops? There is a large mail list for omega users that has many pros on it, mail me and I can give you info. RC
I do system development with my own C++ tools, without any of that TS or Omega fancy stuff. I mainly look at max drawdown and max string of losing days, because that is the major gauge of if you will be able to stick to the system or not. As long as you are comfortable with those, you can always step up your share size for more profit.
Look under the hood, if its making money where is it coming from? calculate R and chart a graph of all of them. Do you have one or two massive R trades that provide most of the system profitability? Recommend: Ultimate Trading Guide by Hill, Pruitt, Hill Title is deceptive as the book is on system development/testing.
Aaf, Here are the things I most like to look at, some are very obvious... 1) I always calculate my expectancy for each set-up that I trade...for more of an explanation check out Van Tharps book 2) Drawdowns.... 3) Flat periods (the amount of time it takes to reach a new high in euity) 4) Oppurtunities....I don't know about your objectives, but I knw this is a big factor for me. I have had some very nice expectancy set-ups (swings) that simply did not produce enough opps to meet my goals, I could not risk enough of my equity(due to extreme overnight exposure) to make the scarce trades produce the type of returns that I was seeking 5) I also graph all of my R-gains to see where most of my gains are coming from...For instance I may of had a system that produced a nice return, but if after a sample size of 100 78% of the gains came from one winner I do not think I would be to estatic about.... 6) I also like to visualy view my equity curve...I don't want a roller coaster or something with huge swings, I seek consistency...I want a nice smooth ascending line up to the upper hard right edge..... Those are some of the things I look for, there are much much much more....good luck on your search, some books that I have found very helpful are both of Van Tharps books PEACE and good trading, Commisso
Thank you all for your advice. I understand about R multiples and calculating expectancies, I have read Vans books several times over. My system is based on trading retracements, using a 20 and 50 simple ma to determine the trend, and pullbacks in the RSI for entry. I was wondering wheter I need to focus on things like standard deviations of the average trade, and wheter I need to export data to excel to create surface charts to choose the most stable parameters for entry, exits, and stops. "cut your loses and let your profits run"
All the previous posts are very good imo. One piece of caution, very, very important and easy to forget : Optimize as little as possible. It is tempting to adjust all your parameters to get the biggest.. whatever (perf, R ratio etc..). You mention stability so I guess you are aware of the pitfall.. nevertheless I'd add extra emphasis. Optimizing on past data is counter productive and possibly dangerous. It is better to keep the system simple and sound : ie explain to yourself the rational. Or why should it work ? Retracement in a trend is rational. In this case your goal (as usual) is to control how much you loose for how much you win and that to me is the most important ratio. forget all the rest if expectancy is not there. Drawdowns and flat periods are very good points from earlier posts too. But only as a measure of what you must expect to stomach (and eventually forget about the system if you can't take the heat). Optimizing more will drive you in losses, that's the system trading paradox. A better way : try your system on several DIFFERENT markets at different point in time. Then try forward testing, aka simulation. Consider slippage, and make it bigger if your time frame is smaller or your size bigger (size does matter ) neo
There may be a lot of good advice here, but I would go back to the original question and ask what is meant by "robust"? What is it exactly that you are worried about? Would you tell us more about the type of trading your system pertains too? There is a lot of difference between a system that trades one trading vehicle sequentially (e.g. an index future or QQQ) and one that trades several non-correlated vehicles (multiple futures) or one that allows several parallel trades in one vehicle (stock trading).
Once again thank you for your replies, they are of great help. To answer dufferdon's questions: 1. I would go back to the original question and ask what is meant by "robust"? What is it exactly that you are worried about? WHAT I MEAN BY ROBUST : is my system catches the same type of moves (retracements) on out-sample data as in-sample data, and on different markets. WHAT I AM WORRIED ABOUT: is that I may have chosen the values for my moving averages and RSI based on the wrong criteria. For example I may select the 6 period RSI as opposed to the 12 period RSI, because it has a higher net profit. But net profit may not be the most important criteria, it may be having lower standard deviation. I am trying to find out which criteria to focus on so I don't build a curve fitted system. 2. Would you tell us more about the type of trading your system pertains too? I am building my sytem to trade the e-mini s&p 500 futures. I am focusing on 1 - 5min timeframe, I will decide which is the most profitable to trade baessd on my testing. Thank you
A "robust" trading model produces consistent profits over a wide range of variables and market conditions. The more robust the model, the "hardier" it is, or the better it will perform when markets change. A robust model id not necessarily the most profitable but is the most consistent over time. Aa, these might interest you: http://clubs.yahoo.com/clubs/investingwithdeepinsight Book: Design, Testing, Optimization of Trading Systems Robert Pardo