Registered: Dec 2001
07-29-04 03:52 PM
First, before I forget, let me address one of the biggest mistakes people make when developing a system: optimization.
Typical traders will take a program like TradeStation or WealthLab or the equivalent, and create a spectacular trading system with clever and creative ideas, which has a near-linear P&L curve, and with very few minor drawdowns... They are excited and can't wait to start making the 'easy money' with live trading.
Will they actually make money with their new methods? Actually, we don't have a clue, as they've never tested whether their new method will make money in the future on not. All they have actually done is prove that their new system does any excellent job of being optimized to fit the past data, which in and of itself, is worthless.
How do we improve upon this? Test your system in the future! Hmmm... the future, you ask, how would we do that?
Let's assume that you are doing your backtesting on 5 years of data, and the system generates 150 trades per year, for a total of 750 trades. In order to tell how well the system performs in the future, you must withhold all of your data from the optimization tests. Instead, for example, only optimize on the first two years of data (300 trades, pretty decent number). Then, use your optimized system to now test on the 'future' => i.e. test year 3 of data with the system optimized on years 1 and 2.
Next, reoptimize your system on the data from years 2 and 3, and give it a future test with the data from year 4. Similarly, optimize your system with the data from year 3 and 4, giving it a 'real-time' test against the data from year 5. In this way, you can see how your system will perform in the 'future' based upon whatever optimization methods you are using.
This sort of method should give you increased confidence when you go to live activation that it should work well in the 'future'.