Understanding Why One System is More Profitable Than Another

In Chapter 3 of Weissman’s, Mechanical Trading Systems, he gives examples of basic trend following systems based on a diversified portfolio of assets back-tested on 10 years worth of data. For each system, he provides a table of metrics used to judge the system’s success. For brevity, I only include the Profit to Max Drawdown Ratio (P:MD) and Annualized Rate of Return (ROI):

1. Two Moving Average Crossover – P:MD = 4.24, ROI = 8.48%
2. Ichimoku Two Moving Average Crossover (whipsaw waiting period built in) – P:MD = 0.16, ROI = 1.26%
3. Three Moving Average Crossover – P:MD = 2.69, ROI = 6.92%
4. Ichimoku Three Moving Average Crossover (whipsaw waiting period built in) - P:MD = 3.01, ROI = NM

Weissman shows that Ichimoku 2-MA Crossover underperforms the basic 2-MA Crossover, yet the Ichimoku 3-MA Crossover outperforms the basic 3-MA Crossover; however, he does not explain why this is so?

I understand the benefit of back testing to truly test the profitability of a system, but this does not explain WHY one system is more profitable than another. I know these four systems are very basic, so I would like to get my head around how to explain why one system works better (ie, higher rate of return) than another before I keep reading and get into more and more complex systems.

I guess my question is essentially, how do traders explain WHY one system works better than another? I realize volatility affects each system differently, but I assume over a 10 year period this affect would be normalized.

I did want to also point out that Weissman does state that it is unfair to make generalizations based on a single study of each system.

Thanks in advance for your responses,

DJP

 

Hello Doronp

I will try to address some question about trend following systems even though I’m not familiar with Weissman’s book.

I have been developing custom code for my own ATS’s for about 12 years to trade stocks mostly in the Daily time interval (with trading models going back further). Many of my ATS systems trade trends.

One answer to why does one system trades trends better than another surprised me when I uncovered it. I will answer it with another question.

How good does your trend following system trade in congestion?

Congested areas can kill a good trend following system. Often the whipsaws and the volatility in these congested areas cause drawdowns that are way out of line with the profits made during the trend. The trend following system never recovers from the congestions affects. Congested areas do not need to be that big to suck up a big % of the profits made in a trend.

One early system I wrote in the late 1990s worked well until August of 2000 when I shut it off it in a large drawdown. I resurrected it in 2003. Upon investigation I found the system could not handle the bear market volatility of the year 2000. So I over hauled the system to recognize higher volatility and started trading again in late 2003. It made it through a good part of 2008 this time and recognized the changing conditions volatility much better. However, it still needs a small re-write to run continually.

Volatility does normalize itself over a longer period of time. But, a one time spike in volatility may be the affect that shuts down a good performing trend trading system and has the trader wondering why the trend following system failed when it has always worked so well.

Rabbitone

 

Don't forget that your book reported testing results of the COMBINATION

• (system + frictional costs + portfolio composition + begin/end dates)

Maybe different portfolios of assets, or different start-and-end dates, or different assumptions for slippage and commissions, would have given different results with different "winners of the competition".

I suppose you could repeat the testing and see for yourself.

 

Rabbitone,

Thanks for your post. So you define one trend-based system as better than another if it is better able to withstand whipsaws in congestion/consolidation/rollover periods and/or periods with high volatility. In other words, a system is better if it is not triggered by false breakouts as often.

As I have read, there are many ways to prevent the system from jumping the gun and executing a trade on a false breakout. One of them, as Weissman states, is waiting for the slower MA to turn direction to confirm the breakout (re: Ichimoku 3-MA and Ichimoku 2-MA).

However, waiting for this to occur is costly if the trigger is correct, as the the system does not capture the full price movement.

So there is a trade off between preventing false breakouts and capitalizing on the full price movement.

What I showed on my original post was that the Ichimoku 'filter' that is used to prevent executing trades on false breakouts lowered performance on the 2-MA system, yet it dramatically increased performance on the 3-MA system. Why? Is there an intuitive reason for this that I don't understand? Or am I foolish for even trying to figure it out?

To respond to MGJ's comments: Weissman standardizes the time period, frictional costs, asset allocation, and initial investment for all the tests. But I am suspicious that if Weissman had picked a different 10 year period, the rankings may have been different. With that said, is it futile to try and understand why, mechanically speaking, one system is better than another? Do traders always have to qualify the statement "This is the better/best system" with "for the period starting at time X and ending at time Y, on this asset class, etc" ?

 

You've got one datapoint. On portfolio P, with assumed costs C, for the time period T1 thru T2, using system goodness measurement metric M, system 1 happened to outperform system 2.

If you had MANY datapoints, trying the same two systems over many portfolios, many different cost assumptions, many different time periods, and many different ways of evaluating/measuring system goodness ...

and if, in those tests, system 1 outperformed system 2 in the vast majority of cases, then you might feel comfortable concluding that 1 "is better than" 2.

But all you've got now is one datapoint. A guy named Richard tested two systems; one had better measurements than the other. Whoopie.

Who's the better tennis player, Nadal or Federer? You had better ask "on which surface?" before answering. Get the idea?

 

Not necessarily if both systems do not have similar holding periods (avg number of bars in a trade) and similiar distributions of long and short trades over time.

Ron

 

Man MGJ, and I thought discretionary trading was subjective!

ronblack, it has become clear that there are just too many confounding variables to account for when judging these mechanical systems.

My next line of thought is how is it that traders develop and optimize their trading systems? Is it by pure trial and error?

I will continue to read up on the subject as I know it is addressed in the book I am currently reading. Can you guys recommend any Mechanical/Automated Trading books that you found most helpful in developing your own system?

Thanks for your help.

 

ronblack likes to recommend the books by Michael Harris. Do a search; he has recommended them dozens and dozens of times.

Me, I prefer (book 1) and (book 2) and (book 3). But to each his own. (I also think Weissman's pretty darn good, congratulations on finding it. If you love, Love, LOVE Weissman, you may also enjoy Lars Kestner. But they don't quite make it onto my personal Top Three list.)

 

All these books are great. Book 3 by author R. Vince is a bit hard to go through if you do not have a good math and finance background but it is excellent.

The first two books by M. Harris are for entry to intermediate level price pattern traders as their title says. The more recent book has some advanced concepts about profitability and automated trading system design.

Lars Kestner book has received many negative reviews at Amazon and this is one reason I didn't get it. Many 1-star reviews. I usually ignore reviews but there is a consensus about this one, it seems.

 

MGJ, I think I have recommended the books only a couple of times along with one or two other. Where did you see that "dozens and dozens of times"?

Maybe you got confused with the links I have posted to Harris' free articles on position sizing and price patterns.

Anyway, I like the Turtle story but I'm not disciplined enough for trend following. I haven't met anyone who is. I think a trader with a good execution platform, low commissions and a trading system with a fair winning bias can do better than a trend follower at much lower stress levels.

My opinion of course. Ron