 |
trader3cnd
Registered: Apr 2008
Posts: 136 |
03-07-12 07:33 PM
Does it make any sense? Has anyone used it with success or is it just another buzzword of the software industry?
IMO an optimized system - is an optimized system, whether that is done stepwise or not. More importantly, I could think of some cases where walk forward optimization would produce worse results than optimizing over the whole sample.
|
| |
|
Edit/Delete • Quote • Complain |
jcl
Registered: Jan 2012
Posts: 407 |
03-07-12 10:39 PM
The answer of your questions is yes. Walk forward optimization makes sense, is necessary for successful strategy development, and produces worse test results.
When you develop and optimize a strategy, you don't want good test results. You want realistic results that reproduce real trading. Backtests can't do this. This can only be achieved with walk forward tests.
|
| |
|
Edit/Delete • Quote • Complain |
intradaybill
Registered: Feb 2008
Posts: 2962 |
03-08-12 09:40 AM
Quote from jcl:
The answer of your questions is yes. Walk forward optimization makes sense, is necessary for successful strategy development, and produces worse test results.
Produces worse test results? I'm not sure what you want to say.
Regardless, nobody is going to take your word for it. Do you have some examples or you're talking under the influence? 
|
| |
|
Edit/Delete • Quote • Complain |
jcl
Registered: Jan 2012
Posts: 407 |
03-08-12 09:50 AM
Quote from intradaybill:
Produces worse test results? I'm not sure what you want to say.
Regardless, nobody is going to take your word for it. Do you have some examples or you're talking under the influence?
I'm always talking under the influence of the mind ;).
I have plenty examples, as I've meanwhile used WFO for about 25 strategies since last October. Here's one example, with a short WFO tutorial:
Link Deleted Spam
There are two reasons why a walk forward test often gives you worse test results than a backtest or a parallel out-of-sample test. First, it eliminates most overfitting effects, and second, it tests in a slightly different market situation than you had when optimizing. Both effects make your tests more realistic, which usually means a worse test result.
|
| |
|
Edit/Delete • Quote • Complain |
goodgoing
Registered: Jun 2009
Posts: 340 |
03-08-12 01:53 PM
Quote from jcl:
Both effects make your tests more realistic, which usually means a worse test result.
If "realistic" means worse, would that make "reality" mean disaster? Because in reality there are unknown effects that make things even worse.
Why do you think optimizing during a choppy market period and then trading during a trending market is any good? It is a bad idea. WFO is a bad idea in general. A parameter optimization session should include a wide variety of market conditions. Like most things in trading, some guy once mentioned the method in his book and some other guys continue talking about it. Most of those who use it in actual trading fail. This is the truth.
There is an interesting ongoing discussion in this forum:
http://groups.google.com/group/adap...bf68c5ea2?hl=en
Some people who actually work with systems claim OOS testing is useless.
|
| |
|
Edit/Delete • Quote • Complain |
jcl
Registered: Jan 2012
Posts: 407 |
03-08-12 02:59 PM
Quote from goodgoing:
If "realistic" means worse, would that make "reality" mean disaster? Because in reality there are unknown effects that make things even worse.
Why do you think optimizing during a choppy market period and then trading during a trending market is any good? It is a bad idea. WFO is a bad idea in general. A parameter optimization session should include a wide variety of market conditions. Like most things in trading, some guy once mentioned the method in his book and some other guys continue talking about it. Most of those who use it in actual trading fail. This is the truth.
There is an interesting ongoing discussion in this forum:
http://groups.google.com/group/adap...bf68c5ea2?hl=en
Some people who actually work with systems claim OOS testing is useless.
Ok, there are some valid arguments but also some misconceptions.
The effects that amount to a difference between testing and trading are well known and based on relatively simple math. WFO is one method to deal with those effects. There are several other methods, but they are more complicated or less effective.
Some years ago, algo developers believed that strategy parameters should be optimized for a long time span and cover as many different market conditions as possible. Then the strategy was considered stable. As it turned out, it was not. Strategies optimized this way tended to become unprofitable in real trading after a remarkably short time.
No matter how many market conditions you want to cover with your parameters, you'll always encounter a completely new market situation in real trading after a while.
You can do a little experiment: Take one of your profitable strategies, and optimize and test it in the traditional way with a simulation period of 4 years, 6 years, and 8 years. You will almost always find that the 4 years test is the best, 6 years is worse, and 8 years is the worst - no matter in which year you start the simulation. This should answer the question about the usefulness of WFO.
|
| |
|
Edit/Delete • Quote • Complain |
| Receive
an email whenever a new post is added to this thread by subscribing
to it. |
|
|
|
|
