How do you develop/research good trading strategies for algorithms? I understand how to create one very well, but my issue is doing research efficiently enough to start an algorithm with the foundation of that research.
The only way I have found to validate a trading strategy is not an efficient approach - it is to forward test it for a significant period of time. Automation and other trading technologies needed are simply academic. So far I have only one day trading strategy that has tested over time to be consistently profitable and I have it automated using Autoview. This took building and testing about 50+ strategies and spending thousands of hours developing them.
I see. Again, I find it easy to implement a strategy but finding the idea behind it is definitely the most difficult part. Thanks for the tip.
There are a lot of ideas for strategies on TradingView (the platform that I used). I started with the leading indicators I liked to use for manually day trading, and then, over time, built them into a strategy in TradingView. The strategy I built has over 100 settings so it is highly configurable, then I kept testing various configurations of this strategy until I had something that back-tested well. Finally, I forward-tested the configurations until I had something I felt confident automating via Autoview.
The answer to this question is different than 10 years ago. Markets has become more efficient, it takes much more today to develop profitable strategies. Markets change behaviour more frequently, meaning you need to adjust strategies much more often, or even develop a new set of strategies. Today, it is all about the development platform and method used, programming has essentially become close to impossible. The first hurtle is passing the learning curve, but takes many years by programming, decades if you use scripting language, centuries if using the cloud for back-testing.
Forward testing is not an efficient approach as you have concluded, rather use a conceptual approach across the development, including conceptual strategies. Time to market is key.
A good place to start is to code a script that emulates the strategy with no posteriori knowledge. Most analysis i see that fails, fails in almost exactly the same way, which is it uses knowledge to make a decision that would not be available at the moment of decision making, look-ahead bias basically. If you take the open price of an MA crossover on 15 min bars, you'll be a billionaire. If you take the close price, you loose everything. That is not an exaggeration. Slippage and fees are simple to deal with, its ridding yourself of that blasted posteriori knowledge. Also, resist overfitting if you tune parameters, use the same concept as above in that you should only tune parameters on data that won't be used to test the parameters. In finance that basically means, tune on stuff earlier than when you use parameter.