Hi, what do you think we should check when doing backtest ? Total PnL % Profit trades Largest Loss Largest Win Therotical % Profit what else ? here are two examples Both systems has - total PnL 150 % profit 87% Largest Loss -150 Largest Win 25 Therotetical % Profit 83% the SAME results, but i think you agree with me, that we will prefer the system #2 so i wonder, what other calculations should we include in the backtest, so we can compare them and see the "better" one ? Thanks, Shay
Also very important are these: - Final PnL% - Equity curve and/or PnL% curve - Max drawdown (MDD%) - StdDev of the equity and/or of the PnL% MDD% can be extracted from the equity curve or the PnL% curve, but that should be done by the test performer. Professional backtesting environments have much more metrics, like Sharpe ratio, Sortino ratio etc... You can check these metrics: http://www.amibroker.com/guide/h_report.html
Hey shay, I would suggest including average win- when the trade is profitable and average loss - when the trade loses. this kind of details may illustrate the feeling you about to experience during your trading time! further more, if your strategy includes stop losses- make sure you count them and compare them against a strategy without stop losses
I often get asked "if you had to narrow it down to one metric, which one would it be?" Would that even be doable? I guess for me I would need at least 3 to do anything MDD% %Profit Trades Avg Profit per Trade
Hmm. I think #1 has a better MDD than #2... Update; No, MDD seems to be about the same in both cases; w/o numbers it's not easy to calc the MDD... Ok, I overlooked the text where it says largest loss is in both cases -150... As said, I would need the StdDev as well...
Gimpyron - Avg Win Vs. Avg Loss - doesn't say a lot here, system#1 Avg Win = 25, Avg Loss = -125 system#2 Avg Win = 20.2, Avg Loss = -93.75 sysdevel99 - Avg Profit per trade is the same in both systems, because both systems have Avg profit of 5 (let me know if you wanted to calculate it differently) %Profit Trades are the same for both systems = 87% botpro - MDD in system #1 = -175 MDD in system #2 = -150 StdDev of PnL in system #1 = 53.09 StdDev of PnL in system #2 = 41.27 what do you mean by - Final PnL% - Equity curve and/or PnL% curve i thought about some parameters that show large difference - the slope of the linear regression of PnL and the R square of it. both should be high (high slope for profitable trend, and high R2 to know how good is our linear line)
- I meant that in the general case the equity curve should be made available to the verifying person. This can be done by providing the trade list, or in case of portfolio trading (ie. more than 1 instrument gets traded), the daily account values, so the verifying person can just load it into excel, mark the column, and click on graph... Of course providing the ready-made chart is a good idea as well. - Backtests should prefer presenting most results as percentages. So, you started in both cases at 25 and ended up at 150. Do you know how much that makes, ie. how much percent profit it makes? Sure you can calculate it in this simple case easily (it's 500% profit), but in the general case that information belongs into the backtest report. - Sharpe ratio for #1: (150 - 25) / 53.09 = 2.35 Sharpe ratio for #2: (150 - 25) / 41.27 = 3.03 --> so system #2 seems indeed to be better. If I didn't make a calc error then MDD% in #1 seems to be 140%... That's gigantic! A value >=100% means normally total loss of the investment (not neccessarily with options if they still have time till expiration)... MDD% of the investment should be kept under about 15%, IMO.... MDD% in #2 seems to be 335%. But I think in both MDD% cases I could have made some big calc errors. Anybody interessted should calc it him/herself... What you can learn is this: you need to apply "diversification", ie. trade many titles (at least 4 I would say, the more the better) at the same time, and make the calcs over your shared account value (ie. all trades done from the same account), then your MDD% of your investment, ie. your risk, will then be much lower...
botpro - it's not an actual system ! sorry if i misled you. i just entered some values in excel, so i should get the same PnL, same profit %, ... and i wanted to here what else should i check, for future back test so we will know better how to compare systems.
regarding Sharpe Ratio, looks nice, but - it has some disadvantages - http://www.futuresmag.com/2013/02/01/sortino-ratio-a-better-measure-of-risk