You are completely right and it has been explained a couple of postings above. Comming back to the original question of OP how to avoid curve fitting. I like this chart which visualizes the traps waiting for some big model builders. http://en.wikipedia.org/wiki/File:PiratesVsTemp(en).svg So how to avoid curver fitting? A useful insight can be gained by asking a question "how to avoid building a good model as on the chart above"? Creating blind trading rules by selecting the "best" rule from all possible combinations of O(0), H(0), L(0), C(0) ...O(-4), H(-4), L(-4), C(-4) is an almost sure way to the disaster (in trading, as a student exercise it can make fun).
I think you are wrong. Selection bias is not equivalent to curve fitting. Selection bias is a process followed by nature and peope every day in their life. You select your mate, your house, your job, you college. But you may be from some country where they fitted on you your mate, your house, your job and your college. In that case, it was curve fitting I guess. Too bad for you. You deserved it. As far as the pirates the other imbecile mentioned, I keep looking at price data and can't find any pirates. He must be under serious delussional state. I see you are tring to match that hard. The more traders that think simple rules are curve fits, the better for me. I let them sweat and dig their life looking for miracles.
The rule of intradaybill is not curve fitted and there is no place to curve fitting. However in no way it is full trading system. Exit, position sizing, stop loss - where is it ? Another problem: I think intradaybill's rule is not enough even as enter rule. And what we do - we gonna add additional rules like: if close was near low or if volume was big enough, etc. No doubt price actions system has less chances to be curve fitted, compared with indicators based system. But there is another problem with price action system. Let me give an example. I think I can consider Japanese Candlestick as price action system. What was very good for trading rice a few hundreds years ago when only professionals (I mean people that really knew the subject) it is not good for trading today when after few words of Bernanke herds gets high degree of optimism and after few days back to pessimism - All symptoms of manic-depression disease. In purest form using hammers has low positive expectancy - not tradable. Using it with B Bands or over bought/sold indicators - not pure price action trading. BTW, my trading timeframe is daily - what I said about Candlestick is checked only on daily time frame.
no matter what rules are used, i think everyone would like to see rising equity curve of the system. if the equity curve is rising it automatically means, the system was fitted to the historical chart. otherwise it would not make any profits. a system with 100% random entries (not curve fitted) will produce zero profit over the long run. 99.9% of traders cannot grasp what i've wrote above!
i'd love to see all other traders developing systems with not-optimized inputs. lets take stop losses. PLEASE, please trade with a 2 tick or 900 tick SL. this way, your accounts will quickly be destroyed and more money will flow to traders who actually put some effort into their strategy design
Well, gosh, the chart makes perfect sense to me. Outdoor work is always more comfortable when the temperature is cooler. Pirating isn't nearly as fun in the heat.
This discussion gives me the déjà vu feeling. It's been around ET more than once before. Selection bias is indeed not the same as curve fitting, but it is perfectly possible to describe the created model as an instance of a function (or more correctly as a relation) with parameters from a universe of possible functions. This could be interpreted as fitting to a "curve", though it is not totally clear to what "curve" we fit to, hence the confusion. To further add to the confusion optimization search methods are often use to find the best function. And, as you probably know, selection bias can cause similar problems as curve fitting. And maybe the OP really meant to ask the question "how do we avoid problems when we start trading a system that has been constructed using hisorical data?". IMHO this is really the interesting question... So maybe everyone involved should bury the hatchets and maybe start discussing this instead.
I think the only one who cannot grasp what he wrote is you. I bet many read your false dichotomy above and laughed on the floor. False dichotomy it is because you classify all systems either as curve fit or random. There are also systems that are intelligent enough to make consistent high probability forecasts. without any curve fit. I suggest you visit any on the sites that offer historical fund data and take a look at mechanical trading programs that have produced consistent high returns for many years. ...and stop trying to enforce your false ideas on others without any justification whatsoever. I use a trend following system with 4 parameters I have optimized and it has worked for many years very well. I also use very simple rules like buying after a few consecutive lows and a two-bar high and they work exceptionally well in some markets.
I agree but it is not curve-fitting and that was my point. Selection bias may turn out to be a good and even necessary process for trading system development. On the other hand, optimization should be avoided. Two different things. Some imbeciles confuse the two but I do not know their hidden agenda. For many years indicator vendors have tried to convince traders that price action trading is also curve fitting.