How would you define overoptimization and how do you know when you're overoptimized? Is any optimization of the "standard" parameters of an indicator too much? If so, then what's the point of backtesting? What do you think? Chuck
I believe overoptimization means optimizing a method to a short time length of historical data. I do not believe it is possible to overoptimize using 20 years of daily historical price data. If I can remove 10 or 20 percent of the price data from either start or end and the simulation results are little changed then I judge the method to be not overoptimized. It is a vague definition. Many things in speculation are messy. I look for parameters that are robust or hardy. By robust or hardy I mean there is a broad range of parameters that return profitable results. If the actual price behavior does not conform exactly to the optimum parameter set then the method still produces good results.
Hook, what I look for is a pattern in the results of changing variables, e.g. the best result comes from a value of 30, the next best 29, the next best 31 or something like that. If the pattern looks random or a small change makes a big impact, then I figure it's an anomoly. Does that make sense? If I'm looking at 5 minute bars, do you think 50,000 bars is enough for a valid test? I know that's only about six months, but would looking at 1 million bars be much more meaningful? I'm not sure where I could get that much intraday data anyway. ESignal backfill is only 6 months worth of data. Does anybody know if it's possible to get more from ESignal? Chuck
Even though you are a short term trader I am not sure that it makes sense to optimize over a time period of about six months. Even with 50,000 bars it might not make sense. I worry that a bull market can continue for a year then change to a bear market. Optimization conducted during the bull phase might be a poor fit to price data accumulated during the bear phase. Sometimes I observe an entire simulation - 100 test results - all showing profit. Tests of other securities show the spotty results that you mention, a more disordered distribution of profit. Some securities appear to me to be much better trading candidates than others.
You'll know when you are overoptimized when your results are inconsistent witht he opotimization period. In other words, tweaking a system for "top" results (assume that the way to evaluate a system has been decided) often results in poor results going forward. That's overoptimization. To avoid, take a long period of data (how long? depends on what period is being traded), divide into two, optimize the early half, check that results are similar using 2nd half data. Not too easy and you may decide that optimization is not the living end.
Since PriceTime Movement is irregular, optimization is averaging applicable only to that PTM tested whether it's 5 or 5 million PT bars. One might use 2 or 3 optimizations together to cover 'all' PTMs, or specific optimization/s for specific PTM/stock/index/future/etc condition, or devise a series of rules in order to use that optimization that way only or series of if/thens; there is no 'one size fits all'.
Easiest way for people to understand is: The strategy should be profitable(significant) before the optimization. .... In terms of optimizing an indicator for discretionary tool, you need to understand stand the purpose of the tool. If you use the MovAverage for trends, you'll optimize it based on it's Risk/Reward Ratio, If you use the Stochastics for entry timing, then you'll look at the efficiency of the signal... rather than the simple Net Profit... Like any trading... Clarity is the key to trading, and from clarity comes control...
What does "efficiency of the signal" mean? I think about developing a stochastics system sometimes and contemplate using growth of capital / drawdown as a selection criterion.