Look for stocks in the strongest sectors. Sector rotation happens as the economy goes thru its cycles. There are times in the year that some stocks are better than others. Right now, the strongest sector is Energy. Everyone still needs to drive a car and oil is used in a lot of manufacturing processes. If oil supplies are disrupted by the Ukraine war or other geo political risks, oil supplies will go higher. Oil supplies are also, capped. OPEC will not produce more oil to drive oil prices lower? Why would they do that? Joe Biden's continued war on US oil companies, guarantees high oil prices. They are now talking about taxing oil profits? Oil companies will just raise the price of gasoline. They are not dumb. US consumers will pay higher oil prices as a result.
So, suppose I pick stocks with low correlation (say I have 15 stocks in my basket), then I optimize my model on every individual time series and trade them separately, or I should use one set of rules for all those 15 stocks provided if those rules give me better results? What I am thinking is that, if I were to fit a model to each time series individually, then it would be most likely overfitting? Especially if I don't get a good model in my first attempt on an individual time series, then suppose I refine it, then it would most likely be an overfit? In other words the question I am asking is that would you optimize over each individual time series, or look for rules that give profitable results over all the stocks?
Diversifying just for the sake of diversifying is a fool's errand. Why would that help you? When you spread yourself too thin, you are trading the worst sectors among your picks. What are the chances those sectors would do worst than most of the stockmarket? So, this so called diversification has actually, reduced your returns? With a limited amount of capital, you should focus on trading the strongest stocks in the strongest sectors to give you the best chance of making the most monies in the shortest period of time. Then, compound your monies to grow your capital. You got to change your way of thinking as most of your ideas are flawed so, you are just wasting precious time, achieving nothing.
Welcome to the market, where everything correlates. And when things get bad all correlations go to 1. It's tempting to use a Gaussian distribution and assume all events are independent of each other but that model is flawed as you have discovered. There's no easy answer to this.
I trade a trend following strategy myself, and these statistics seem reasonable. However, 10 years might be too short for stocks, since a large part of those years are bull market years except for a short bear market in H1 2020 and 2022 YTD. If you trade a long-only strategy, you would expect the performance to be great since the stock market did really well during this period. So from a trading system standpoint, are you going to assume the next 10 years will be as bullish as the last 10? If not, then you should expect a more moderate performance going forward. Performance may be worse during side-ways or choppy markets. Also, since stocks belong to one single asset class and are highly correlated, it is not surprising that you experience a clustering of trades. You are after all, trading the same market - equities. There are many reasons why trend followers trade multiple markets. Testing your strategy across multiple markets mean the results you get are statistically more meaningful - that your strategy has a real edge, rather than catching the market at the right time and getting lucky. Trading more markets also mean your performance would be more stable as you will still be able to catch a few trends here and there and not have to suffer through false signals in a single sideways market. I would encourage you to think more statistically about the robustness of your strategy and worry less about performance. Trend following strategies have very similar performance profiles. But I would say the good trend following strategies out there are the ones that are statistically robust and has a real edge that can be traded across many markets.