Previously, I asked a question about number of years of backtest required for results to be trusted. On second thoughts, I think that was the wrong question. I think the right question would be what is the minimum number of trades in a backtest, not number of years, for backtest results to be trusted. Number of years should vary according to time-frame while number of trades taken should not. Any guidelines for the the minimum number of trades in a backtest required from the elitetraders here? Time span over testing must include both bull and bear markets, otherwise any number of trades is invalid. Thank you.
I just finished a youtube series on algo trading where the presenter says he uses about 18k as the number of trades he likes to see on the backtest.
Sample size can be misleading. I can take thousands of temperature readings in Alaska during the summer and get an average, which will tell me little about the average temperature over the course of the year.
18k is hard to achieve if time frame is daily. Possible for minute timeframe but I doubt so for daily timeframe.
The presenter trades forex intraday. The point he was making is to do what it takes to get a statistical significant backtest. He recommends running the backtest over many symbols over at least 10 years to get this.
Here is the series I watched. I think much of what he says makes sense. I would like to hear what others more experienced think. https://www.youtube.com/playlist?list=PLv-cA-4O3y95J6xmwSaCILL4FlGJZO0PJ
lol - you'd never get a working strategy if you wait for 18k trades. There are standard ways of looking at statistical significance and they are mostly applicable to systematic trading. Obviously, there are issues to be mindful of, such as curve fits, family-wise errors etc, but that covers any case when you are working with data.
whenever i have an idea, i test it on 3 scenarios, bullish days, bearish days, and doji days. Since i daytrade, that means 2-4 setups a day. That usually result in about 30-40 trades a month. Take the 3 scenarios into account, it's little over 100 trades. If my ev is +, then i run with it live.
Lots of good answers but you probably want a practical examples from someone who actually does that. I personally wouldn't trade a strategy that has less than 1-5k trades on dailies per 5 year interval and survives multiple market types, to cover which you would need at least 4 of those. Normally a single strategy won't produce same return over different market conditions but you don't want risk to exceed what you're comfortable with in any scenario. That's assuming you're not using too many degrees of freedom or magic numbers. Robust strategies normally have risk reward peak around your chosen values, gradually degrading the further away you get from it. If you have only one single parameter value that works and otherwise expectancy is negative that is a sure sign of overfitting. Val