Trading Systems Robustness & Market Structure It is commonly agreed that the measures of a robust trading system are: 1. Common sense and valid underlying logic. 2. Identical parameters, rules for all markets 3. Consistent profitable performance across multiple markets in multiple market complexes 4. Simple rules However, I wonder in addition to the above, what role âMarket Structureâ as defined by: a. volatility, b. liquidity of specific markets, c. format of trading (pit vs electronic), d. continuity of trading (24 hr markets vs daytime only markets), e. type of market participants, f. number of market participants, g. market volume can affect a trading system and how this impacts on system robustness. So for example can one I have a system that is seemingly ârobustâ and profitable in one time zone and less profitable in another time zone due to market structure issues. So for example if a trading system were to show good results say between 2000 â date, but a mix of fair, good and poor results between say 1987 â 1999, should one deduce from this that : a. the system is fully robust and the less consistent performance in the period 1987 â 1999 (note I am not saying the performance was bad, just not as good) is due to a different market structure in the 1987 -1999 window. The 1987 â 1999 window had less markets available for trading, less liquid markets, fewer market participants, more discontinuous markets i.e. no 24 hour trading) or b. the system showed less inconsistent performance in the 1987 â 1999 window due to inherent system weaknesses; in which case one should expect that the period going forward could also show less consistent performance. Is there any way to even scientifically confirm which hypothesis from the two above is valid. I hope my thoughts will provide the basic for a hearty and intelligent dialogue on this important issue. Many thanks.
"Valid underlying logic" ? What oracle in the sky decides whether underlying logic is valid or invalid? Is valid underlying logic like pornography, impossible to define precisely, but "I know it when I see it"?
you rise interesing issue here. i believe the proper approach to it is the following: every system which enters and exits fixed size at single time (no scaling in/out) is designed to exploit some more or less specific type of market(s) inefficiency. that is especially true for "simple" systems. if for some period of time a given market(s) are running with very low amount of that type of inefficiency or it is completly gone and as a result your system's performance turns worse or much worse, then i'd say there's nothing wrong with the system itself. take a extreme example here. the market turns totally random at all timeframes for 2 years. in such conditions no system will make money (except ones which made few "lucky trades") and you just can't blame the system's internal design for bad performance in such case. however, if markets are running in favorable conditions for a given system, there're many inefficiences which it should exploit and performance turns to much worse, then the reason most probably lays in bad system's design. i'd also say the first case (changes in markets structure) are quite common and natural. quote from Mr. Simons: "Statistic predictor signals erode over the next several years; it can be five years or 10 years. You have to keep coming up with new things because the market is against us. If you don't keep getting better, you're going to do worse." Mr. Simons said that his models change weekly." well, i don't mean as frequent market changes as weekly (unless its very high freq trading, like in simons fund), but definitely you must expect that any single system can stop working due to market inefficiences which can be gone or severly diminished (while other inefficiences may be still present). and that is the reason for diversification over systems and markets. regarding systems design i think it's bad when systems results are sensitive to small changes in the markets and that's what we should avoid.
ok, but what to do with cases like: market "A" has 5% range over 2 years market "B" has many 10% DAILY ranges? more, you can be 99.9% sure that market "A" will never experience 10% daily range, ok maybe once in 50 years, and market "B" will never experience 5% range over 2 years. would you still require identical parameters and rules for both of these markets? sure, it can be done with some nice perf. on both markets. but is it efficient way of trading? is it a valid design limitation? for some high freq. systems the "identical parameters+good perf. on all markets=robust system" can be a nonsense. example: we've found some decent edge in EURUSD but it only works in 14:20-16:20 CET hours due to macro figures which shake the price strongly and it only nets a small profit per trade. and now as a "robustness confirmation" we want to test it on lean hogs... which don't have that specific reactions to macro figures and as a result no profit can be made with such system. also higher slippage and commiss. vs EURUSD make it impossible to see even a shadow of profitability on this market. of course, it was an extreme example, buy you get the idea.
Good points. I've actually thought of this conceptually...but at some point you have to be careful in making these distinctions. I would argue that in this example...you've developed a system based on a specifc premise for a specifc market...in which case this is a mono-maket system not a multiple-market system - so the single parameter concept would not apply in this unqiue case. In general though, how do you measure the robustness of systems.
Someone I respect told me recently that in assessing a trading system he pays most importance to the performace of recent years beacuse he believes that most recent performance is indicative of future perfrmance. I am curious, what do others think about this viewpoint?
yes, but it is only indicative. similar to markets themselves: recent activity is generally good predictor of what's coming next. i mean sort of there's a high vola now, there's a good chance that it will continue for some time. i think that one must be prepared that at any moment the sys performance can dramatically change and you have to know what to do in such situations, just like the trading system always knows how to respond to a given market action. that's what many managers are missing in my view - they trade the markets with quantitative methods, but theres not enough quantitative approach how to "trade"/handle systems themselves. regarding indication of future performance i'm convinced that a sharpe ratio=3 system has much higher chances for producing SR=3 going forward vs the system which historically traded SR=1. that's obvious. also, first system is at higher danger of poorer than historical perf. vs the 2nd sys. the reason - market inefficiences which enable high SR trading won't sustain for a very long time.
i agree these are different worlds and shouldn't be judged by the same merits. as a panacea to the problem of changing markets and differences between each market, there's an idea to detect the current, single market conditions/structure/character whatever you call it or to put it more scientific : the detailed, comprehensive, statistical properties of the market and generate optimal trading rules and parameters according to these properties. and of course update it every hour/day/week.