Limited backtest-data is one problem I have. That’s right. My H1-data feed only goes until 2014. But that’s not what this is here about. I'm curious if a backtested strategy is valid if it works only in some specific markets and in some timeframes. Trying to understand why the results are like they are is a good hint. I think there is a pattern in which markets the strategy works and in which not. Im already in a low-risk forward Test since beginning of November. I have 75% successful trades and a profit-risk-ratio of 1.45 at this moment. But I only did 8 trades. Backtest results are good, nearly my strategy was tested in various backtests by other traders. And it fits perfect to my trading possibilities. I give it a try. Get successful or die trying Thanks for all the answers.
Don't do this. A) You are using information you didn't have at the start of the backtest. Not sure if confirmation bias is the correct term, but it's definitely bad (I call this implicit fitting). B) It's unlikely the differential in performance is statistically significant or meaningful. Use all your instruments and allocate risk in a backward looking fashion over the backtest using a robust methodology GAT
Oops. Forgot to mention: this are *all* "mean reversion" concepts -- there is no independent reinforcement from using the three. Which puts a very big hole in things..... Sorry about my miss.
IMHO both answers (yes & no) are valid, because some strategies may work on a group of instruments and others may not. It would be great to use a strategy that works on many instruments but this may lower your chances or invalidate the strategy and then you’d have to either find a new strategy or rely on one that’s fit to specific instrument. Each approach will have drawbacks and benefits. One of algo traders and book writers, Ernie Chan, stated that using separate strategies for different instruments worked best for him. His blog may also be useful to you, for example where he discusses an H1 example specific to AUDCAD, but not all Forex pairs, and where at the same time he describes how to deal with and optimize strategies based on smaller data sets: http://epchan.blogspot.com/2017/11/optimizing-trading-strategies-without.html?m=1
Actually I just remembered when/where specifically Ernie Chan talked about using single strategy vs different strategies for different instruments (his primary focus is Forex but here he uses futures as examples):
I disagree. Treat the algorithm as a blackbox. Here is the long blackbox: EMA(15) > EMA(30) How many instruments will "pass" this in backtests? This is why I personally would want to evaluate why it is working on a certain set of instruments because then you would need another filter to apply to filter those out from the universe.
A black box shouldn’t matter. For example a strategy (whether black box or not) may detect corporate buy-back pattern that applies to one company but not another. But stocks are different and difficult, so I do trade vast number of stocks using few strategies. Though at the same time I have dozens of strategies per each stock/company, and may still add some of them to strategy pool. While with futures and forex there may be different trading patterns, for example depending on the weather, airlines buying crude, others hedging against gas, specific banks/countries buying currencies, etc. I’d expect those patterns to change over time, but nevertheless I get different results using different strategies. And I hate to refer to just one guy, but Ernie stated that results speak for themselves, even though he and everyone else assumes that using single strategy across multiple instruments sounds more logical. If your results show something different then this confirms that different strategies may work differently, so no one may even agree on what works best...
That is a mighty big IF. Most people never get to the point of trading profitably. Instead of spending years bouncing from one trading idea to another and losing lots of money looking for profitability, why not apply a bit of science to the trading hypothesis first? Conduct proper backtesting methodology first and see if results in positive expectancy before risking your bankroll. Why not do this? Because proper backtesting to avoid biases is hard and most people don't know how to do it.