How much is an ATS worth?

dom993 · Feb 13, 2013

... and yearly results

kut2k2 · Feb 14, 2013

Quote from dom993:

The first thing I look at for a system, is the ratio of its average year P&L to its max.DD ... but historical DD doesn't cut it for me, and the 2nd thing I do (pending having the distribution of trades outcome) is to run a MC simulation for 1-year worth of trades, and then use 5 std-dev away from the mean max.DD to re-evaluate the 1-year avg. P&L / max.DD ratio. I use that 5 std-dev away from the mean max.DD as my "system stop" - it is the capital that I am willing to put at risk if I start trading that system.

A 1:1 ratio (1-year avg. P&L to max.acceptable DD) provides a "risk free" system at initial position size in 1-year (2:1 in 6 months, 4:1 in 3 months, etc). IMO, the value of a system should take that ratio into consideration.

One aspect that has only be discussed indirectly here, is the expected lifespan of the system. In my view, positive & consistent backtest of the system over last 10 years (or more) is the best indication of the system's ability to survive market changes, but obviously no guarantee. And given that historical data is readily available, there is no excuse as a system designer (& seller) for not testing the system over as much historical data as can be purchased.

Just for the fun of it, I am submitting the attached performance report for "valuation"

This version of the system can (and do) take multiple concurrent trades. It has indeed 12,674 distinct entries in the course of the 6 years of backtest, but only ~3,300 exits.
More...

Serious effort here. So what would your ATS pricing factor look like?

TheMagican · Feb 14, 2013

Quote from kut2k2:

Here's an excerpt from the 2011 interview by Futures magazine of William Eckhardt:

FM: How do you ward off curve-fitting?

BE: What most people use to ward it off is the in-sample/out-of-sample technique where they keep half their data for optimization and half their data for testing. That is an industry standard. We donât do that; it wastes half of the data. We have our own proprietary techniques for over-fitting that we actually just improved on a year ago. It is important to test for over-fitting; if you donât have your own test use the in-sample/out-of-sample [technique].

I can talk a little more about over-fitting, if not my personal proprietary techniques. First of all I like the [term] over-fitting rather than curve-fitting because curve-fitting is a term from non-linear regression analysis. It is where you have a lot of data and you are fitting the data points to some curve. Well, you are not doing that with futures. Technically there is no curve-fitting here; the term does not apply. But what you can do is you can over-fit. The reason I like the term over-fit rather than curve-fit is that over-fit shows that you also can under-fit. The people who do not optimize are under-fitting.

Now the two numbers that most determine if you are over-fitting are the number of degrees of freedom in the system. Every time you need a number to define the system, like a certain number of days back, a certain distance in price, a certain threshold, anything like that is a degree of freedom. The more degrees of freedom that you have the more likely that you are to over-fit. Now the other side of it is the number of trades you have. The more trades you have, the less you tend to over-fit, so you can afford slightly more degrees of freedom. We donât allow more than 12 degrees of freedom in any system. If you put more bells and whistles on your system it is easy to get 40 degrees of freedom but we hold it to 12. On the other side of that, for us to make a trade we have to have a sample of at least 1,800; we wonât make a trade unless we have 1,800 examples. That is our absolute minimum. Typically we would have 15,000 trades of a certain kind before we would make an inference as to whether we want to do it.

The reason you need so many is the heavy tail phenomena. It is not only that heavy tails cause extreme events, which can mess up your life, the real problem with the heavy tails is that they can weaken your ability to make proper inferences. Normal distribution people say that large samples kick in around 35. In other words, if you have a normal distribution and you are trying to estimate a mean, if you have more than 35 youâve got a good estimate. [In] contrast, with the kind of distributions we have with futures trading you can have hundreds of samples and they could still be inadequate; that is why we go for 1,800 as a minimum. That is strictly a function of the fatness of tails of the distribution. You have to use robust statistical techniques and these robust statistical techniques are blunt instruments. [They] are data hogs, so both seem to be disadvantages but they have the advantages of tending to be correct.
More...

Interesting read,btw.Any additional reads/research on wide tails fenomena you could suggest?

dom993 · Feb 14, 2013

Quote from kut2k2:

So what would your ATS pricing factor look like?
More...

The generic formula would be:

ATSvalue = (ExpectedLifetimeP&LPeak - SystemStop - ATSprice)*DiscountFactor

- ExpectedLifetimeP&LPeak: calculated at minimum position size. This is basically the 1-year average P&L * anticipated number of years before system failure.

- SystemStop: 5 std-dev. from mean max.DD, calculated for 1-year worth of trades. For most system that I have developed, a fast approximation of that is twice the historical max.DD

Note that (ExpectedLifetimeP&LPeak - SystemStop - ATSprice) is the expected net result from purchasing then trading the system until its failure (defined by reaching a DD equal to SystemStop).

- DiscountFactor: this DiscountFactor could be a set value (independent of the ATS), to account for the general fact that the future is unknown, and on average only X% of all systems live through their promises. Lower boundary is obviously 0, upper boundary probably 50%, and a realistic value could be 20%.

Assuming the ATS is sold for a price equal to its value, the above formula simplifies into:

ATSvalue * (1 + DiscountFactor) = (ExpectedLifetimeP&LPeak - SystemStop) * DiscountFactor

hence:

ATSvalue = (ExpectedLifetimeP&LPeak - SystemStop) * DiscountFactor / (1+DiscountFactor)

This leaves as main variable the anticipated number of years before system failure. Two key elements that could play a role here are the # of trades in the backtest, and the time period covered by the backtest.

CT10Gov · Feb 14, 2013

Quote from dom993:

ATSvalue = (ExpectedLifetimeP&LPeak - SystemStop) * DiscountFactor / (1+DiscountFactor)

More...

Congrats. Your valuation formula depends on 3 variables that are basically impossible to forecast. (The ability to above any of the 3 is in itself a major source of alpha).

dom993 · Feb 14, 2013

Quote from CT10Gov:

Congrats. Your valuation formula depends on 3 variables that are basically impossible to forecast. (The ability to above any of the 3 is in itself a major source of alpha).
More...

For one thing, I disclosed how I do compute the SystemStop for any system - using both a detailed & precise formula, and a fast approximation of it.

If you took the time to read, you would also have noticed I suggested a fixed value for the DiscountFactor, which is system independent.

Try to make one useful contribution in this thread - it's Valentine day, we all need some love, and you'll be glad that you did.

CT10Gov · Feb 14, 2013

Quote from dom993:

For one thing, I disclosed how I do compute the SystemStop for any system - using both a detailed & precise formula, and a fast approximation of it.

If you took the time to read, you would also have noticed I suggested a fixed value for the DiscountFactor, which is system independent.

Try to make one useful contribution in this thread - it's Valentine day, we all need some love, and you'll be glad that you did.
More...

All the pseudo formulas in the world doesn't help you estimate things that are mostly unforecastable. You are just putting a false sense of precision where there is none.

kut2k2 · Feb 14, 2013

Quote from dom993:

The generic formula would be:

ATSvalue = (ExpectedLifetimeP&LPeak - SystemStop - ATSprice)*DiscountFactor

- ExpectedLifetimeP&LPeak: calculated at minimum position size. This is basically the 1-year average P&L * anticipated number of years before system failure.

- SystemStop: 5 std-dev. from mean max.DD, calculated for 1-year worth of trades. For most system that I have developed, a fast approximation of that is twice the historical max.DD

Note that (ExpectedLifetimeP&LPeak - SystemStop - ATSprice) is the expected net result from purchasing then trading the system until its failure (defined by reaching a DD equal to SystemStop).

- DiscountFactor: this DiscountFactor could be a set value (independent of the ATS), to account for the general fact that the future is unknown, and on average only X% of all systems live through their promises. Lower boundary is obviously 0, upper boundary probably 50%, and a realistic value could be 20%.

Assuming the ATS is sold for a price equal to its value, the above formula simplifies into:

ATSvalue * (1 + DiscountFactor) = (ExpectedLifetimeP&LPeak - SystemStop) * DiscountFactor

hence:

ATSvalue = (ExpectedLifetimeP&LPeak - SystemStop) * DiscountFactor / (1+DiscountFactor)

This leaves as main variable the anticipated number of years before system failure. Two key elements that could play a role here are the # of trades in the backtest, and the time period covered by the backtest.
More...

Wow, I wasn't expecting a full price formula, maybe just some factors based on the attributes you discussed.

Congrats!

kut2k2 · Feb 14, 2013

Quote from TheMagican:

Any additional reads/research on wide tails fenomena you could suggest?
More...

http://en.m.wikipedia.org/wiki/Heavy-tailed_distribution

Thread Closed At The Request of Original Poster - Admin Joe

Log in or Sign up

How much is an ATS worth?

dom993

df20x2v13.per year.png

kut2k2

TheMagican

dom993

CT10Gov

dom993

CT10Gov

kut2k2

kut2k2