Has anyone ever seen an exhaustive list of the ways in which systems can be curve fit? Or, are there an infinite number of ways, so no list would ever be exhaustive? I want to avoid curve-fitting, but it would help in that endeavor to know exactly what are the curve-fitting pitfalls for each type of strategy.
As soon as you choose a system you are already in some type of curve-fitting because you make some arbitrary fit to the data and you optimize some arbitrary objective function. Most of those who talk about curve-fitting and say it is bad for trading systems know nothing about that they are talking about. They say that because they heard someone else saying that. It is part of mass behavior. The subject is too complicated for forum discussion. Things like "if you change a parameter you are curve-fitting" or if the system performs in out_of_sample well without parameter adjustments then it is not fitted", are false statements, too simplistic and basically out of the as* of someone who then was presented as an authority in the subject and many adopted his unjustifiable BS.
That list would be a blank piece of paper. You can't curve-fit a strategy or a system. You curve-fit data. You overfit a strategy or a system. Of course the best thing would be to avoid both curve-fitting data and overfitting strategies. To avoid overfitting: http://www.elitetrader.com/vb/showthread.php?s=&threadid=192056
Curve fitting is just a term. All strategies are based on the assumptions you make about the market. When your assumption is not valid and does not represent reality then the strategy falls apart. People give it a name and call it curve fitting when it fact it could be better termed as having "invalid market assumptions/observations". Since there are as many assumptions possible as imagination allows there is no exhausive list of curve fitting pitfalls. The only way to avoid curve fitting is to make sure your assumptions are valid. There is no substitute for experience in this.
Each of your strategies is going to made up of a set of rules/parameters. You can score each strategy with something like... Strategy Score = (correct prob.) / (# of rules) Most likely the strategy with the highest score will be the least curve fitted.
Do what works for you. There is no formal mathematical proof that over-fitted system fail more than any non over-fitted system.
Why is it the best thing to do? I would like to see a formal proof, not because someone just said that and the rest of us took it for granted. This is what happened I believe. When system trading started in the 1980s people attributed wrongly the high rate of failures to some mathematical concept called "curve-fitting". The real cause of failure was undercapitalized accounts. The industry wanted to attribute failures to some vague notion rather than to admit that you need at least 50K to trade comfortably and be able to exercise reasonable risk and money management. That was not good for business. So they blamed some bogus phenomenon and stupid speculators continued looking for the system that would make them rich with 5K down. The key to success is risk and money management. Fitting is a distraction from the real important issues of account capitalization and proper risk and money management. http://en.wikipedia.org/wiki/Money_management http://www.tradingmarkets.com/.site/stocks/how_to/articles/-74644.cfm http://www.priceactionlab.com/Blog/...t-capitalization-is-primary-cause-of-failure/
Rather than looking for a list of "don'ts" you should focus on tests that indicate your system will work in the future. All trading systems, to some extent, are curve fitted because they are data driven. The issue is to test the robustness of your rule set. This is done in two ways. You can test your rules over several market cycles to avoid having them dependent on a specific market condition. Many will stop there. I like to add to this by using two sets of data. The first set is used to "develop" your system. The second set is to "verify" that your rules perform the same within statistical significance limits on both sets. If your system performs similarly on data it has never seen before, you have more confidence it will perform well in the future where you hope to make money with these rules.
Your beliefs lack formal proof as well. If you want to add proofs to your understanding then study the mathematics of positive expectation and the gambler's ruin theorem. You may find that your conclusion is incomplete.
Why, have you seen a lot of formal proofs in forums? What is it exactly you need a formal proof for? I may be able to direct you to professional help. For example, the proof that you are a troll is the undeniable fact that in about a month's time since you have registered on 09/24/2010 you average 5.13 posts/day of total crap, I mean total waste, meaningless posts.