i bought mine from disktrading http://disktrading.is99.com/disktrading/ but i'll be happy to hear about other people resources
For my systems, I use the "modified sharpe ratio" mentioned by Acrary in his system development thread. The formula is: modified sharpe = mean / variance or more explicitly: modified sharpe = average trade return / standard deviation of trade return Like another poster on this thread said, expect that as your backtesting trade quantity goes up, your average return, largest loss, and historical max drawdown will usually have a negative drift. I understand the negative drift intuitively like this: if an extreme negative event for my system has something like a 1 in 1,000 chance of happening, I probably won't see it if I'm only looking at a 500 trade sample. Even if I'm looking at a sample of 10,000 trades, odds are that there is an even worse extreme event that has a 1 in 20,000 chance of happening and I can't see its effect in the backtest because my sample is too small. With this in mind, you can try to factor in the negative drift based on the size of trading samples your backtest is using. A related area of statistics that you might find helpful for implementing this kind of sample size based "fudge factor" is the "confidence interval", which I'm sure you're already familiar with. Don't forget about the fat tailed distribution of returns if you choose to implement something like this though.
thanks for sharing. what would you consider as a tradeable system in terms of "modified sharpe" values? and do you find any difference between EOD systems and intraday systems?
I look at how each system contributes to the modified sharpe of a basket of systems. If it improves the overall modified sharpe of the basket, then it is probably good to trade if the other metrics aren't bad because it makes a positive contribution towards a smoother equity curve. Each system shouldn't necessarily control an equal portion of your margin though. I think it is a mistake to pick one time horizon and ignore the others because doing so blinds you to a lot of information and causes you to miss opportunities. Many people assume that just by trading the short time horizon that they will implicitly get the benefits of the longer term moves, but that is an incorrect assumption. So my answer is yes, I do find a difference between EOD and intraday systems. Each are making decisions based on different information, and if your systems are based on TA signals, then they are modeling fundamentally different volatility and liquidity characteristics.
I agree. 42 trades is not enough. I could get a profit factor and win ratio like that flipping coins with 42 trades. The lowest # I would use would be about 200 trades, and only if the profit factor and win rate was very very good. 1000 trades is solid to get some good stats if applied to lots of market conditions. My best systems are in the 65% win, PF 2.5, 6% max DD range and thats with several thousand trades, over years of data, tested against down markets, up markets and sideways markets. System 2 - not good enough to trade. You need more "buffer" in case you missed something.
I think sharpe ratios, etc, are over rated. Just post a picture of the equity curve (by time) along with the max DD #. This will visually tell me more than any sharpe ratio or any other number can. This is all about understanding the "behavior" of your system.
When you are running through 10,000+ different variations and system combinations, you can't look at every equity and drawdown chart, so what metric do you use for optimizing? I optimize for the modified sharpe and eyeball the more interesting equity and drawdown charts (chiefly the optimal one). I'd be interested to hear how other people do it.
thank you dragon for sharing are you referring to futures intraday systems? i'm attaching the curve of the intraday system. any comment are welcome.