Bill, There is an even simpler way to resolve the opinions on Optimization. It's called OOS or Out of Sample testing. To do this: 1) take a set of market data and create the best system you can, with 2 rules or 20 rules. Make it simple, make it complex, whatever. 2) Freeze the rules and logic. 3) Apply you new system to a new data set that you have not seen in step 1 and 2. 4) If you had a winner in step 1 but now you have a system not worth trading or even a looser, then you system is not valid. It is over optimized or curve fitted. However if your system still works, then take another OSS period of even longer length and test. IF you system continues to work then you have created something that is very likely to work in real trading regardless of the simplicity or complexity of the system. The process above is the basis of neural networks and genetic algorithms and these tools have created some very profitable trading systems. Jerry -------------------------------------------------------------------------------- Quote from Donchian: Curtis does a thorough explanation of this paradox in his book Way of the Turtle, chapters 11-13. You may read an extract at: http://www.tradingblox.com/tradingb...ion_paradox.htm -------------------------------------------------------------------------------- The paradox is only in his mind. There is no paradox. Optimized systems are fitted for vest past performance. Past performance is no guarantee of futures results I heard. Simple. Paradox is an apparent contradiction that is resolved when one of the false premises is removed and replaced by a true one. In this case, we have a true conclusion, no paradox. Bill
Actually I'm still puzzled because there were around 9000 nonoptimized trades on in sample data and my out of sample trades were not significantly different than the in sample. The strategy itself makes sense and is very simple. The timeframes that I'm concerned about were in the past and maybe due to changes in the interworkings in the market. The thing that bothers me is that I can't understand why and the more I try to, the more likely it'll be biased. I was reading a post by Acrary mentioning random testing for the same data period for the same duration of trade length and then using chi square for determining significant difference. It seemed to me he was relating it to preventing against overfitting, but this appears to be more useful for significane testing with regard to the market character (sorry I lost his post). If anybody has any feedback, I'd appreciate it. Jeff
A system would be fairly common and generate some common results when merely based on commonly agreed/ found stuff.
I keep on repeating the same ol' thing but: Understand your "edge". Depending on the edge you are using, how you test (optimize and etc.) and how you set the risk management criteria changes. To make things easier, let's say I found the specific edge: Let's say you exposed a specific tendency in the equity market like... "All stocks that hit the 'Daily Bar' 20-day moving average on the 3rd Friday of the month, bounces off the MA." After some research you, understand that funds liquidate their position on the 3rd Friday and they use 20-day MA. (Note that this is just an example) I test the system with a code like below under intraday data: If (DayofWeek == "Friday" && WeekofMonth == '3' and Price == Average(DailyDataClose, 20) { Trade(); // Trade checks if it's bouncing below or above and deciding whether to buy/sell } ExitAllPositionAtEndofDay(); Running the above on all symbol universe under: - 30 years of data. - Set contract size. - All statistic values are calculated within the symbol. (eg. AvgTrade or RiskRewardR. will not be cross referenced with other symbols) Total Profitability = 70%. Average Risk/Reward Ratio: 3.4 Average Sharpe: 1.9 Now... what would you optimize? and based on what fitness? Would you do it the same way as an trend-following model? Scalping/Market Maker? Would Out-sample... Walk-Forward be significant (relatively high Sharpe).