 |
EricP
Registered: Dec 2001
Posts: 883 |
07-29-04 04:34 PM
Is my system profitable? At what level of confidence?
Assuming we are using 'future' data (whether this is backtested future data or actual forward tested data doesn't matter), then we can use statistical analysis to our data to determine the likelihood that our system is a profitable system over the long run as follows:
x = abs(Avg-Profit) * (Number of Trades)^0.5 / (Std Dev of Profits)
Once "x" is calculated, you can look up the confidence level on the chart below to see the likelihood that this system will be profitable over the long haul:
x ------------------ Confidence Level
0 ------------------ 50%
0.075 ------------ 53%
0.126 ------------ 55%
0.25 -------------- 60%
0.36 -------------- 64%
0.52 -------------- 70%
0.67 -------------- 75%
0.84 -------------- 80%
1.04 -------------- 85%
1.28 -------------- 90%
1.4 ---------------- 92%
1.645 ------------- 95%
2.05 --------------- 98%
3.0 ----------------- 99%
4.0 ----------------- 99.5%
5.0 ----------------- 99.7%
Note that if the average profit is negative, then this will give you the confidence level that this is a losing system, over the long run.
One quick example to illustrate this equation. Assume that we have 50 trades, yielding an average profit of $40 per trade, with a standard deviation of $150 per trade. Using the equation and table:
x = abs(Avg-Profit)*(Number of Trades)^0.5 / (Std Dev of Profits)
x = abs(40)*(50)^0.5/150
x = 1.88, from table, Confidence Level ~97%
-Eric
|
| |
|
Edit/Delete • Quote • Complain |
EricP
Registered: Dec 2001
Posts: 883 |
07-29-04 04:39 PM
Let me point out that these calculations were designed for series of data that have a Gaussian distribution (i.e. 'bell curve' distribution).
Is this true for our typical stock trade P&L's? Heck no, not even close!! 'Fat tails' from the occasional large stop loss, for example, will make our P&L distribution non-Gaussian.
Therefore, recognize that these equations and methods, while providing a good structured way to analyze your data, will not yield perfect results. The actual system may NOT be 97% likely to be profitable in the future, maybe it's only 93%... So what, big deal. The point is that a value of 97% is better than getting a value of 84%, and worse than getting a value of 99.6%. Greater 'confidence level' results should increase your confidence that you have a good system, while not necessarily being a statistically perfect answer for the true probability of future results. But, we are traders here, not statisticians. I don't mind if my predictions are off a few points, so long as I can make consistent money.
-Eric
|
| |
|
Edit/Delete • Quote • Complain |
EricP
Registered: Dec 2001
Posts: 883 |
07-29-04 04:51 PM
As abogdan pointed out, determining when to activate and deactivate a trading system really becomes a system within itself.
My choice has been to develop a very simple method that will shutdown a trading system when I 'lose confidence' in it (statistically), and activate a trading system when it proves itself to be worthy (statistically). I exclusively use the equation and table shown in the above posts for making these decisions.
Anyone can arbitrarily decide their activation/deactivation levels, as well as the number of trades to be used for calculating their confidence levels for a system. For me, I have arbitrarily decided upon using the last 120 trades and I deactivate a system if my confidence level drops below 90%, and I activate a system if my confidence level rises above 93%. Note that I will use as few as 50 trades for these calculation if that is all the data that I have.
Somewhat arbitrary? Yes. Statistically sort of invalid (non-Gaussian)? Sure. But, it provides a solid structure for making the activation/deactivation decisions in a consistent and reasonable manner. And, most importantly, it works for me.
I hope that this has been helpful for a few folks, and this is all that comes to mind on the topic at this time. Questions or comments are welcome, and I'll post additional thoughts if they come to me.
-Eric
|
| |
|
Edit/Delete • Quote • Complain |
abogdan
Registered: Dec 2003
Posts: 872 |
07-29-04 05:13 PM
Marvelous! Now that I had a chance to think about it, it makes a lot of sense! Thank you for your contribution, very nice!
|
| |
|
Edit/Delete • Quote • Complain |
shorty_mcshort
Registered: Apr 2003
Posts: 202 |
07-29-04 06:00 PM
Quote from EricP:
Is my system profitable? At what level of confidence?
Assuming we are using 'future' data (whether this is backtested future data or actual forward tested data doesn't matter), then we can use statistical analysis to our data to determine the likelihood that our system is a profitable system over the long run as follows:
x = abs(Avg-Profit) * (Number of Trades)^0.5 / (Std Dev of Profits)
Once "x" is calculated, you can look up the confidence level on the chart below to see the likelihood that this system will be profitable over the long haul:
x ------------------ Confidence Level
0 ------------------ 50%
0.075 ------------ 53%
0.126 ------------ 55%
0.25 -------------- 60%
0.36 -------------- 64%
0.52 -------------- 70%
0.67 -------------- 75%
0.84 -------------- 80%
1.04 -------------- 85%
1.28 -------------- 90%
1.4 ---------------- 92%
1.645 ------------- 95%
2.05 --------------- 98%
3.0 ----------------- 99%
4.0 ----------------- 99.5%
5.0 ----------------- 99.7%
Note that if the average profit is negative, then this will give you the confidence level that this is a losing system, over the long run.
One quick example to illustrate this equation. Assume that we have 50 trades, yielding an average profit of $40 per trade, with a standard deviation of $150 per trade. Using the equation and table:
x = abs(Avg-Profit)*(Number of Trades)^0.5 / (Std Dev of Profits)
x = abs(40)*(50)^0.5/150
x = 1.88, from table, Confidence Level ~97%
-Eric
Just a quick question. For example, if you were trading stocks and pyramid your profits. Would this test be valid or would you have to test it on the same # of shares or same dollar amounts on each trade?
|
| |
|
Edit/Delete • Quote • Complain |
| Receive
an email whenever a new post is added to this thread by subscribing
to it. |
|
|
|
|
