I did some backtesting of my strategy of few stocks for 18 months and also tried each 6months. backtesting was done in Ninja trader and Multicharts at for $1000/ stock trade my strategy is ATR based strategy 1st 6 months $3351 2nd 6 months -$4250 3rd 6months $1452 and I am testing this in sim now and have NO IDEA what would be for the next 6 months. it could be even -PnL so how do you guys use backtest results? without knowing it can reproduce a +PnL in the future. I dont expect a exact replica of backtest in real trades, so should be closer?
First off you shouldn't preselect stocks as that introduces bias. If your strategy only works on 20, 50 stocks then most likely it's just a curve fit. 18 months for me is nowhere near a reasonable period, we've had a massive bull market for longer than that.
ok, but if I don't pre select stocks, how would I even run backtest on what instruments? so 18months less, what period would you backtest, like 3 yrs or 5yrs? i doubt i have intraday data for that long
According to simple sampling theory (emphasis on simple... maybe too simple), for a strategy to be statistically significant with a confidence level of 99% and a margin of error of 5%... You would need to conduct at least 666 trades, and 363 of them would have to be profitable to reject the null hypothesis. If you wanted to reduce the margin or error to 1%, than you would need 16641 trades, with 8470 profitable to reject the null hypothesis.
good point. I never took period to test based on # of trades, always did 1yr to max 3 yrs but problem to get 666 trades is, I may not have so much historical datafeed example, I test a symbol called ACHN with 30min or 60 min chart approximate, it takes a trade or two /month so i get only 24 x 5 = 95 trades in 5 years (I have Barchart that gives this data) 666 is way too far from datafeed I can get
Well consider this: Lets say you can test 100 trades, and want to do 99% confidence. This would mean your margin of error is about 13%. To reject the null hypothesis, you would need to have 61/100 profitable trades. But there is a stochastic element to the market, and the backtest is only accurate to 61 +/- 13 trades, 1 Sigma. A one standard deviation error is not an unlikely event... It is reasonably likely that your 61% win rate is actually a 48% win rate. With only 100 trades, you really can't reject the null hypothesis reliably. Even with 666 trades, it is difficult. If you want to trade systematically, I would recommend shelling out a bit of cash for decent historical database. Otherwise, your backtests will not be statistically significant, when margin of error is factored in.
And that would mean if his average time in trade was 1 day and his system was in a position half the time, he would need to test for over 10 trading years: 666/0.5/250 In the second case, he could be dead before the testing is complete.
Depends on the nature of the strategy. If he is trading the same strategy, with the same parameters, on multiple assets, it could be significantly less. You could also invoke "Population Size" and say only one trade per day is possible... but that is cheating in my opinion. For trading, population size shoudl be infinite... but, if you only had 250 trading days of data.. you could say 182 trades is a large enough sample size.
Good idea. IMHO He would need to test with 10 different symbols to get the testing time down to one year.
I do use same strategy, with the same parameters, on multiple assets except 1 parameter, which is data series of the chart I might do 15min on few symbols, 30min on others and 60 on others. I have 3 data series so far