Your Algo? - MNQ daily candles, entry after 4 consecutive up bars, exit after 50 point pull back or 50 point take profit. Trend following win% is usually less than 50%. Underlying trend could be when the Fed was lowering interest rates, flooding the market with free money. Also rebounding from the 200 week moving average, supported by the mutual funds. That's four days to get in with one day to exit. There's five days in a week so that's tied to fundamental news. Instead of 4 up candles, enter on close crossing above the 5 day moving avg (H+L)/2 plus 50 points.
I used to use train station but I found that when I would back test the week and then forward walk the same week there's only about 88% fidelity match. So I kind of gave up and now I use one month of forward walk data where I capture all the values from individual trades then I stratify that data to project for the 5th week.
The truth is we are told that we cannot rely on past performance but past performance is all we have for without it we would have nothing no metric. However I believe what happened in last November does not teach you what's goning to happen during this August. I believe what happened in late June and July will teach what's going to happen in August and then in turn July and August will teach what to expect in September and so forth. So for me I use the last 8 to 10 weeks of data give me the historical basis to understand which days make the most known money which ones don't, which indices make the most amount of money which ones don't, where longs or shorts are best and the performance of target one and target 2. Then I have clear mandates I can and I can trade towards strength and away from weakness.
Yes, using 10 years of data is useless for back testing. It is actually the best recipe to over-fit your algo to the data. It is way better to use a a Geometric Brownian motion simulator to start with it: https://www.quantstart.com/articles/geometric-brownian-motion-simulation-with-python/ https://github.com/philipperemy/Quantitative-Market-Data-Generator And then do a series of forward tests with real data. That is how I got the best results IMHO
Market conditions change and in order to use ten years of data, you have to account for those conditions. I don't think that any of you, that claim to use years of data, do that. So your back-test is useless.
I had a different experience. At start I build a system that over time got improved many times. After a long time I realized that what I did during this period is start with a static system and switching it bit by bit in a dynamic system. So if now I test that system, it always gives positive results as the system adapts to the market behavior. So building a system on 10 years of data can be successful. All depends on how the system is build. I never did optimizations by running my system on past data. When I back tested, I watched where things went wrong and tried to find out why. Each change of the system had to pass the test of time without any additional optimizations. So I never optimized by changing series of parameters with values from X to Z to find the optimal return. I just made corrections that kept all parameters at the same value for the whole test. Switching parameter values is no solution for a system that should work in any market behavior.
Read my previous post. I did it. And it works. But the system was not build on ten years of data. It just works on any lenght of data. So the test is only useful to see how flexible the system is. Development does indeed not need ten years of data. At least not in daytrading. It also works on futures, forex, stocks... It works on anaything taht uses numbers as data. It is purely mathematical.