How do you decide the in-sample and out-sample sizes (two extra parameters) in Walk Forward Optimization without being curve-fitting?
the best method of avoiding curve fitting is not having inputs at all. if you want to optimize then you should do it properly. make a plan with your available data and split it up according for in and out of sample runs. 1. Optimize over X period 2. Run setting over out of sample 3. Repeat there are no shortcuts.
Unless your strategy is dynamic adjusting to market changes of course your going to have different results. ES trading at 1200 - 1500 is very different from ES trading 600 - 900. Option strikes are still 5 points but the contract notational value is $30K vs $75K. How well does your strategy adjust, manage risk and perform trading random numbers? After all that is really what your after.
It all has to do with whether you found something, that given the current market structure, has a REAL 99.9+% of things backtested will prove to be spurious. And most of the things that are walkforward test will still fail. The size of the sample is to give SOME statistical credence to the idea. Too small a number or too short a time means you have less confidence in it.
for sure ...thats the reason for walkforward.. if done correctly it will catch the contracts cyclical (or other ) changes . I use 6 years of tick data (and percentages) to check the robustness of my ES parameters.
Careful... last trade time & sales tick data will lead you down a false path. Test against bid/ask/last trade data or better yet level II tick data if you plan to trade large lots.
I have a system I developed in 1993 and it still makes money. All universally qualified propositions, like yours, are proven false in the face of a counter-example.
I eventually settled on a different way to optimize. In my opinion the point is not to "optimize" but to "stabilize" and I therefore approach the problem as follows: I break down the data in e.g. 1 year blocks and optimize each of those blocks separately. I am looking for consistency across the blocks. If a parameter drifts, i look for a different way to express the relationship I'm optimizing so that I can get the stability I'm looking for. I have had very limited success with systems where the parameters needed frequent optimization (but that's just my experience).
Not true...you can account for slippage. You can not get (that I know) 6 years of bid/ask/last data. I trade large lots and monitor bid/ask sizes.