This thread is in a way a continuation of my previous thread in this forum which has been closed to prevent the reappearance of trolls who posted during the new format. Thanks for the great timing, guys. That thread got me to thinking about what an ATS is really worth, but in nonmonetary terms. Thanks to inputs from both danielc1 and dom993, I was introduced to Van Tharp's SQN (System Quality Number) as well as its fatal flaw. As a result, I've been able to devise a much superior metric called SPS (System Performance Score) which is detailed below. I asked myself what is the single truly best way to evaluate an ATS. After a while, it hit me: how much are you willing to bet on it. The best answer to that question is unambiguous: the Kelly ratio (or Kelly fraction). As detailed in the first post of my Kelly thread (qv), there is an easy way to calculate a very good approximation of the Kelly fraction for a general case. I will hereafter refer to this as the NOBF (near-optimal betting fraction). As detailed in my previous ATS thread, we must establish a minimum acceptable number of trades in order to evaluate an ATS from backtesting data. We were introduced to both the Surf Criterion (1000 trades) and the Eckhardt Criterion (1800 trades). For now we will stick with the Surf Criterion. So calculation of SPS becomes straightforward: SPS = NOBF*min[1, N/1000] , where N is the number of trades in the backtest. So take the N trade returns from your backtest and sum them. If the answer is nonpositive, proceed no further: your system sucks. If the answer is positive, then divide that sum by the sum of the squared trade returns. This quotient is your NOBF. Now multiply the NOBF by the lesser of one and N/1000. This final number is the SPS of your ATS. Enjoy.
Something is missing here. Let's apply your formula to two systems, A and B: System A trades: [+10, +10, +10, +10] System B trades: [+20, +20, +20, +20] SPS for A: (10 + 10 + 10 + 10) / (10^2 + 10^2 + 10^2 + 10^2) = 0.1 SPS for B: (20 + 20 + 20 + 20) / (20^2 + 20^2 + 20^2 + 20^2) = 0.05 According to these results, system A is twice better than system B, whereas intuitively (and by common sense), it's the other way around.
What you've proven is that the formula doesn't work for holy grail systems. You should be so lucky. Remember what the NOTF is an approximation of. The exact Kelly fraction is the value of k that makes the following equation true. 0 = sum[ Ri/(1+ k*Ri) ]_i=1toN When all values of Ri are positive, the value of k that satisfies the Kelly equation is infinity. Go see the "Bad Kelly" thread to see other limitations on the NOTF.
I don't think it works for unholy systems, either. Here is another example: System A: [+10, -5, +10, -5] System B: [+20, -10, +20, -10] SPS(A) = 0.04 SPS(B) = 0.02 In this case, your formula says that system A is twice better than system B, but it's actually exactly the same system, scaled.
No it's not the exact same system. In system A you lose five times your bet, in system B you lose ten times your bet. The betting fractions reflect that significant difference.
There is nothing there that indicates the bet sizes. For all we know, system A bet 5% of the account, and system B bet 50% of the account on each trade. Or it could be the other way around.
I've concluded that nonlinear5 has pointed out a valid weakness in the original formula. So here is the revision : SPS = NOBF*R*min[1, N/1000] , where R is the average winning trade return (%).
Is this the correct Excel UDF calculation ? Just place it in any cell and point to the range with the returns: =UDFSysPerfScore(B2:B1220)
Now you are mixing the absolute P&L and the percentage P&L in the same formula. This will lead to confusing results. I think if you experiment long enough, you'll come down to this: SPS = sqrt(trades) * (P / S), where trades is the total number of trades P is the average profit per trade S is the standard deviation of all trades This closely relates to Sharpe's ratio and the SQN (System Quality Number). The above formula is what I use to rank and compare my trading systems.