Can everyone explain some of the technical aspects of backtesting strategies. Is it more tied into programming automated systems? My perception of backtesting pretty much involves testing a technical/charting strategy against old data. I would like to create a great backtesting strategy before I get back into the market and actually backtest religiously. It seems alot of the great traders seem to live by it. Thanks.
without backtesting u better b very familiar with non linear mathematics and or finite elements and modeling of same. so yea backtesting is what you have - example! darn did u see me just shoot my foot off? I just learned from history that didn't work so well. Dont think I'll try that again! It could be said you can learn something from backtesting. mb
be careful about ''death by tweakage'' and by that i mean that you will find the best nuances for the data you have on hand and then the moment you go live you tank your account. try to find parameters that works on different segments of your data and that can be compared against similar markets i.e a strategy that works on the emini sp500 should work on the mini dow too, maybe not as much, but you should have convergent results.
I suggest the following paper by Michael Harris: http://www.tradingpatterns.com/About_Us/articles/synthesis.pdf He explains in the paper how backtesting is used to develop and analyze trading systems and he includes a flow chart along with some simple examples. Ron
I'm pretty much a pure mechanical investor and been doing it for about 10 years now. I use the approach you're talking about. Develop a plan, backtest it, then work the plan. You seem to be asking "what does it mean to backtest?" In it's ideal form backtesting is developing a set of rules, then applying those rules starting at some day in the past -just like one would have actually traded. Then see how well you do. The idea is that past performance will be an indicator of future performance. Now, if your rules are sufficiently simple, and doesn't trade a lot (like the Dogs of the DOW strategy) you may be able to backtest by hand. Looking up historical data and plugging the numbers into a spreadsheet can work. I did that for many years downloading data from Yahoo. There are a number of sources for price and volume (technicals) like Yahoo or MSN, but historical fundamentals like sales or earnigs are less easy to come by. Historical earnings estimates (needed for valuation calcs) are even harder. So, if your rules are more complex or if you use any fundamental data in your strategy then you will need to use some backtesting software. There are a couple of online tools I'm familiar with. I like Stockworm.com. It has both technical and fundamental data going back over 10 years. Another site to look at would be stockfetcher.com. They only have technical data though. Now remember, I said "in it's ideal form". Backtesting like this will give you the exact buys and sells you would have made each day over the entire span of your test. A less rigorous approach would be to simply do two-point backtesting. Apply a set of rules to "buy" some stocks at some point in the past then see what those stocks are worth now. There is also an approach that (typically technical trading software uses) where the rules are applied to just one stock. Then see when that stock would have gotten bought or sold. In any case, backtesting is simply applying the rules you plan on using to trade with to historical data then analyzing the results. Good luck!
My opinion about backtesting is that out-of-sample test should be done in the last step of development. Otherwise, if you tweak your system after out-of-sample test, you're just overfitting it to the whole dataset, and out-of-sample testing becomes meaningless.
I fully agree (missed that step). My strategy development is typically done with "in-sample" data and then validated against "out-of-sample" data. Stockworm has a cool feature for this. All stocks in the universe are randomly assigned to set A or set B. I develop using set A and then test using set B. I don't trade a strategy unless the two results are similar. I also backtest using the the total universe of stocks. Ideally, the results using the total universe come out a bit better then either of the parts (I attribute this to having a larger better pool of stocks to choose from at any given time). Good catch!
Even though the out / in-sample is implemented as part of the system development process. It doesn't guarantee the system will work in the future. As a matter of fact... there is never a guarantee that ANY system will work in the future. It's only telling you that it is more probable / likely to work in the future. A discretionary trader works the market move. A system trader works the market tendency. The rest is very "similar".
Westward, please check out this thread, it might help you: http://www.elitetrader.com/vb/showthread.php?threadid=104393