optimising Does it work

I had a conversation with a former discretionary daytrader who has spent a year developing systems. We seem to have a friendly disagreement about optimizing. What constitutes curve fitting and what type of optimizing might be useful in indicating future profitability. Also when do adaptive tools become over fitting.

If anyone wants to advise others on this subject, I for one will listen as I have not done too much system designing and would be afraid to trade any system that would be optimized because I know sometimes optimization leads to curve fitting. So when if ever is it a a good thing.

(Side note) I would be able to trade say a breakout system that was tested over many parameters and I got to choose a setting from within an area that showed solid returns. However, to me that is not what my friend is doing when he talks about optimization. (I believe he has been mislead by Indigo and would like to see him stop wasting his time, however I am very aware I could be wrong because this is not my forte by any stretch.)

As I write this I see the need for definitions and explanations and I am all ears. Thanks any help will be appreciated.

 

If you want a hard edge, you have to design a system that does nothing but optimize itself. In other words, the system starts with nothing and keeps optimizing that.

 

jem,

I believe much of the bad press about otimization came from the days when people used only end of day data. It is so easy to over fit EOD data it isn't even funny! Contrary to that, if you find a system that works well on a very small time frame it is much more likely to have strong predictive value.

One other thing, tell you friend to get a clue and drop indigo. It is a POS.

Also, as I understand it, curve fitting is not the same as over fitting.

 

Yes - but I would do one more thing - I would filter out those stocks whose price action are likely to confuse my system. This is almost as trading the damn thing myself....

nitro

 

That's obviously a very good idea. I was thinking about stock index futures, since I have yet to find a system that even appears to me to have hope for working on stocks. Maybe your suggestion is just what I have been missing so far.

 

Hmmm,

I do not buy this argument for the reason you give at least - my suspicion is that the reason that daily systems in the past had more trouble is because you had to guess where in the bar the system would get filled - a nearly worthless assumption.

As far as being able to over optimizing a 1 min bar or less being OK, I don't agree.

BTW - what is "Indigo" ?

nitro

 

Hmm,

Yes, in that case that would work...

nitro

 

1) I didn't say it coudn't be done. I am saying it is less likely to be over fit. It really depends on your number parameters, etc, etc

2) www.indigoinvestor.com is a 2k pile of shit my step bought a few years back. He is more math inclined than I, we both agreed it is worthless.

 

PubliasEnigma
Senior Member

Registered: Jul 2002
Posts: 118


07-10-02 11:29 PM
Back Testing....."Paradoxically" Speaking
Of all the paradoxes that exist in trading, the notion of “Back Testing” could quite possibly use some “Back Testing” of its own. The literal meaning of “Back Testing” is rather self explanatory and actually defines itself with little explanation. “Back” is referring to past occurrences and “Test”, is a trial of a model.

Regardless of the level of sophistication, rigidity, flexibility, or time frame of your model, you help to define it and ultimately accept it. You are simply testing a method against past occurrences and analyzing the results. Conventional wisdom supports that the result of such testing can be optimized, and
when applied to the future, produce a positive expectancy. None the less, one obvious fact still remains. The market, which is what your modeling into, remains a variable. Upon further scrutiny though, a less obvious change also occurs. Your perspective. Unknowingly, you have begun to reinforce in your mind what should happen as a result of your positive experience in literal backtesting.


Once satisfied with your results, your back tested model will still require one more trial. A trial by "The Here and Now.” You have learned the literal definition of back testing above. It is now time for “You the Trader” to learn the traders definition of
back testing. In the traders definition of “Back Testing”, “Testing” simply means to put on the trade, as dictated by the system of your invention. "Testing" remains constant and mechanical as a function of your system. “Back”, on the other hand, is actually a variable.


Now with everything to your satisfaction and your methods in place, your back tested system triggers a trade and your day of discovery begins. This is what you can expect to discover. Everything you thought your system to be, wished it to be, hoped it to be, ever dreamed or believed it could be is absolutely irrelevant under a trial by “The Here And Now”. Go back and read that again.


Enter “Back”, the variable of the traders definition of “Back Testing”. Under a trial by “The Here And Now”, once entered into a trade, you are given but 3 choices. But wait a minute. This is strange. None of choices are related to the literal definition of back testing, or the positive "perspective" from which you saw your trades resulting in. “Back” (traders definition) becomes to mean.....

1) Back Off 2) Back Down 3) Back The Hell Out.

Back Off: This refers to reduction of trading frequency.

Back Down: This refers to reducing your share or contract size.

Back The Hell Out: It means just that. Neither you nor your system are in the flow of the market. Get out and re-enter when
conditions improve. Both yours and the markets.


There in lies a paradox of back testing. From the traders definition of "Back Testing" you derive a finite and constant set of prepared responses on every trade, every time. From the literal definition of "Back Testing" you derive a deceptively skewed departure from "The Here and Now". You only see what should happen, not what could happen.


Now ask yourself this..... If the correct response to any trade is a constant, regardless of all the variable methods, signal generators, systems, and market conditions, where does one find a constant? The answer is quite simple. You become the constant, constant "WITH" the market.


It is one thing to say you trade the markets. It is something completely different to say you trade "WITH" the markets. If you have a bias, a belief, or behavioral response from the past, regardless of origin, of what the market will do, should do, or could do, you are not trading "WITH" the markets, you are trading your past perspective of the markets. The day you decide to trade the markets perspective in the moment, and not yours, is the day you will be trading "WITH" the markets.


Oh, by the way. There is one more definition of “Back Testing.” This one is provided to us courtesy of the markets. It is for all those traders who choose to ignore the traders definition of back testing. The markets actually re-structure “Back Testing” to “Backer Testing”. Backer testing occurs when you, the trader, have to go out and find new ”backers” to fund your account, the one you just blew up, as you chose to “test” your trading account for the sake of “Back Testing.” Literally, that is.

 

I'm not a big fan of computerized back-testing. I do it myself because:

a) It forces me to look at a lot of charts and, the more I view them, the more I will recognize familiar patterns.

b) No mechanical system works as well as a intuitive based system.

c) Back-testing encourages one to tweak a system to massage the back-tested data to such a point that a system is developed which is 90% precise but entirely untradable (i.e. buy when the RSI crosses over the tangent line of the 42.8 MA as it oscillates in a semiwave resembling 2sin(4.5x) - 4coss(9.3) but only between 9:35am and 9:37pm on just Thursdays where the previous day rained.


RIGHT

aphie