when people post backtesting results, i'm always a little unsure of what all the results mean. i want to start getting into backtesting ideas, so i need to know what the results mean. for example, when looking at backtesting results, which numbers are the most important? in the past i would look at "return on account" the most. i am going to attach a tradestation report for a system that has a "return on account" of over 800%. at first i thought this was great, but i was later told that number can be meaningless. so now i'm interested in finding out what all the results mean and how to interpret them. to start, can someone explain why a "return on account" of over 800% in this example can be meaningless? how do you know when you have something good? thanks
Gordon, One of the first things I look at, especially when I get big returns like that are, equity curve and drawdowns, for me I don't want to see that a substantial portion of the profits were made on a small % of the trades. I like my equity curves nice and smooth with minimal drawdowns. I also look to make sure that the programming is not looking forward at events that couldn't have taken place yet when making buy/sell decisions. Just some thoughts, this is also a very individual thing as each person has a different trading personality and can live with some issues and not with others, also bankroll affects our choices i.e. drawdowns etc.... wdbaker
GG, You might want to check-out: http://traderclub.com/discus/board.html There is a ton of systems related discussions over there - an evening with the archives might prove useful. I found LeBeau's book "Computer Analysis of the Futures Market" to be a good all-around read for eyeing some of the common pitfalls in testing. (it is his board I referenced) (you have said in previous posts that you have read most of what is out there, so apologies if you have read this one) In short, the % return of a system can be a bit misleading. Max drawdown is not a joke -- could you (financially or psychologically) hold through that period? I've seen a lot of systems that look great but upon closer investigation show that to net 100,000 you would have to have be able to sustain a 500,000 loosing streak ... different story than just the return figure could indicate. I'll second what wdbaker stated about winners: you need to look at the per trade results ... was one or two winners responsible for a lot of the gain? Some people will take out the largest winner from their results and gauge things that way. There's more, of course -- systems trading is a whole other world. Curious as I have read some of your past postings: are you looking to trade systems or just looking to test strategies?
im just paper trading still but -- just looking at the results you have 200k profit with a max intraday drawdown of 25k - thats 100% of starting equity - obviously then 25k is not the account size required and 800% is not the return if say 100k is what is required to trade this system then return would be 100% over about 4 1/2 years. 24 losers in a row - that would hurt
I like the number of consecutive winners to be much higher than the number of consectutive losers. I think if you system has more than 9 or consecutive losers, your entry and exit criteria is not good enough. Max drawdown is only relative to account size and total return. Increase the account size with the same number of contracts and your max drawdown is reduced Runningbear
thanks guys, this is exactly what i wanted to get into. slowly but surely i'll understand this stuff. could someone explain profit factor? i know it's gains/losses, and obviously you want it as high as possible.....but why do you want this number high? for example, if it's low, what does that mean? thanks
PF = (Win%*AvgW) / (Loss%*AvgL) whereas Expectancy = Win%*AvgW - Loss%*AvgL I find PF to be more intuitive.
Gordon Gekko, there are several important aspects in judging results, that are not covered by standard software: 1. the key of all trading is return:risk. nobody professional cares too much about return only. 50% return per year, ignoring 75% maximum draw down spells: hands away. 2. in order to get 1. into a single figure, Sharpe Ratio was introduced (quite some time ago), which is SR = (return - riskFreeRate)/annualised standarddeviation of returns Sharpe Ratios in the long run of above 1.5 are good, above 2 are very good, above 3 are excellent. High figures run with significant (!) money are rare. I assume that daytraders with small accounts can do significantly better. My numbers are valid for hedge funds. 3. in addition to sharpe ratio there is a variety of things that divide return by another figure, max draw down, or by the average of the three biggest draw downs - you get the idea. Return/Risk figures give you a much clearer picture than return only. High Sharpe ratios means smooth equity curves. You most likely have seen tests with great return, fitted to a handful of trades. 4. avoid too high returns. In my opinion, making stable 15 to 25% after fees for yourself or for clients over ten years indicates professional investment management. Again, I am aware of daytraders beating these numbers by far. But I assume that there is a high drop out rate (=-100%), which is hardly ever mentioned, since that does not attract public attention too often. 5. avoid thinking in ordinary percentage terms. Consider the following equity curve: day 1 1000 day 2 1500 day 3 1000 normal percentage terms calculating daily yields would find an increase by 50% and a consective decline by 33%, leaving the account with about +17%, whereas the real result is 0. This is due to a weakness of ordinary percentage calculation, once it comes to bigger numbers. Note: in small numbers differences become neglectible, but when you compare results over ten years, using compounding, it might become an important thing to be aware of. A way to get aroung this is using log returns. easy in excel. 6. portfolio effect - most neglected thing in the day trading arena. the single most powerful technique. add different styles, different trading objects in order to improve sharpe ratio. horrible results on the single market might look very well in noncorrelated context. take two systems that are non correlated (<=0.3), add them up and calculate the sharpe improvement. Do that with twenty markets and you get into the game of professional money management. peace
Vast treasures on the board but the culture is extremely abrasive. Put on your thick skin if you dive in. Good trading.
I have been using E trade, but I am looking to start trading daily. I think I need a more robust product. Mostly trading stocks and the e-minis. Can anybody explain the advantages to using Tradestation? Interested in getting into backtesting in the future. Thanks. James