this is a question of long discovery for me... I'll anwser it like this: the point of switching markets is that it is an esay way to randomise the data for you. I would say that the robustness increases with the difference between the markets and the correlation (smaller is better). But let's say you are testing ES. Keep everything the same (with appropriate $ per point mapping), switch symbols to YM and see what happens. Performance plummets, then there is some particular aspect of ES that your system is capturing that doesn't exist in YM. The question then becomes what?!
Yes I think so. Without testing many parameter value combinations one won't know how sensitive the results are to parameter value changes. When you have found a system that is profitable over a wide range of parameter values, one that you can't disprove or invalidate with backtesting, out-of-sample testing etc. and you want to trade it you then need to decide on the parameter values used for actual trading. Suppose tests have shown stable and good performance of parameter A between 20 and 60 and parameter B between 100 and 200. I'd then favor center values, around 40 for A and 150 for B in this example. Because I do not know which combination will be best in the future center values are more trustworthy than extremes to me.
When I say combination, in this example, I'm referring to the combination of moving averages and the system buys when one moving average crosses above the other, and sells when it crosses below. Another system may only have one parameter (not a combination), like buy on a n day breakout (example... optimize to 21 day high breakout), in that case, the parameter is the window of time you are using to determine entry. Back to the moving average combination/crossing system. Suppose you ran many, many simulations, and one combination (say 50/100 cross) gave you 10,000 profit, but the nearby neighbors gave you decreasing profits (neighbors being, 40/100, 60/100, etc..), of only 5,000 or less. You would say, you optimized the combination of 50 and 100 (which peaked at 10,000), but it was not robust, as the curve (graph of profits vs. paramaters) looks like a hill. But maybe a part of the hill goes flat a long time (simulations range from 1day to 30day with all crosses), that portion is considered robust. If you operate on the flat portion of the curve (ideally the center), you sacrifice peak optimization for robustness and stability over a wide range of conditions. The goal is to optimize as robustly as possible. If the optimization is sensitive (i.e. the curve shows a hill with a sharp peak) it is not robust. The closer to flat the response, the more robust it is. As others mentioned, parameters (moving average combo) are just one aspect of the optimization process. There are many books on this. A basic explanation is in turtle traders, by curtis faith. Most modern trading systems books talk about this. Or maybe pick up trading systems and optimization from pardo. The basic idea is borrows from engineering design. If you designed a rocket that shows perfect low pressure and comfort orbiting around earth, it has been optimized. Maybe you take it to mars and it explodes, because the pressure was much higher than you expected; it was not robust.
Strigman's book is great, The Encyclopenia of Trading Strategies by Katz and McCorwick is even better. However, think GA and walk forward Optimization.
dima777, Here's a simple visual explanation: The solid line in the bottom left chart (d) has been over-optimized so it is not robust (i.e. its future performance is not reliable because it wiggles around to much). The solid line in (c) is robust although it is not perfecty optimized to the training set. We can basically tell that it is a better approximation of the data than the line in (d). The lines in (a) and (b) are neither robust nor optimized- they simply have not been run against enough training data. Regards, Max Dama
i don't believe the graphs posted above show any explanatory features of either robustness or optimization in a trading system... when a representation of system performance shows the bad effects of optimization what you see are a few isolated peaks of solid performance in a sea of mediocracy. when a representation of system performace shows good robustness you see an enduring plateau of solid performace that shows a clear maximum region and that gradually fades away from the genuine, as opposed to fake, optimal parameters.
Journeyman, You are correct, I am showing the opposition of robustness and over-optimization for a generalized approximation algorithm. I did not mean to indicate that the charts represented a trading system's performance. It is more of an abstract, academic demonstration. Thanks for pointing that out for others and I like your written explanation too. Regards, Max Dama maxdama.com
thank you very much for your thorough answers..can anyone please suggest any book other than Stridman's?
Great forum on systems: www.tradingblox.com/forum/ "Trading systems and methods" by Perry Kaufman. Have yet to read "Way of the turtle" by Curtis Faith, many rave about it. Michael Covel is a controversial author but he does explain trend following. "Winner take all" by William Gallacher for a different perspective. The turtle system rules could be downloaded somewhere for free. Bruce Babcock, Ed Seykota website, Robert Pardo, Chuck Lebeau, Futures Truth, Market Wizards and surely many more.