Hi all. I am fairly new to this area and just joined the forum. I hope I can learn a lot from the people in this forum and also try to share my knowledge to newbies like myself. That being said, I have gone through a couple of finance and trading courses (autotrading) and am working on my own code for that. I was hoping people here could help me with my problem. I am using MLP (4 layer, 50 neuron each) for making prediction on forex data. I have been working for a long time on the model now with no luck. I totally understand that prediction accuracy in forex is fairly low, and I am not expecting much. But I am almost %1 above/below random which I think could be better. Now I can provide you with all the details but it will be a pretty long post. I will mention the ones that are essential and ask my question. I am using 20min period for prediction, EUR_USD. I work on Oanda, and use 180 days data to train my model on. I download 5min data and resample to 20min. Then I use some features (my main question will be about these) and use their lagged versions up to 5 lags (5*20 min=1H) but the features contain, for example, moving averages with window of 50 (so I am not limited to 1H). I train my MLP without any regularization and am able to overfit. But no matter what kind of regularizer I add, the end result is that my model cannot generalize. More precisely, the training and validation loss both decrease, the training accuracy increases but the validation accuracy is either almost same value (about %50) or goes down. Right now I have a comprehensive set of features but I am using MA, EMA, real price and volume as features for testing (any feature I have been adding actually makes everything worse). Now my question is, is MLP a good model for prediction in this case? What kind of regularization works better? and finally what features do you suggest I use?
You should use multiple data streams, so you can get intermarket strategies. Only when there is no spurious correlation among instruments you can get an out-of-sample true profits. But do not expect too much here. It is not easy. But if you only look on one symbol itself you will not find any meaningful, because there is no stable autocorrelation here.
Welcome & Happy Xmas! It's futile attempting to predict the markets I instead would recommend to do it for trading strategies only by using options, ie. apply your code on options & their underlying stuff you trade, not trying to attempt the future market price (stock or index price etc.), not even just the direction. IMO one does not need any ML for the financial markets (stocks, options, futures, currencies etc.), but I admit I don't have much experience in ML or AI, just the basics, but I'm an experienced professional senior C++ programmer / SW developer/ SW architect, as well an options expert, though not as a registered professional with license etc. Search for articles and literature for quants; it's IMO equally advanced and challenging like MLP and AI etc. See also: https://www.marketsmedia.com/machine-learning-in-finance-theory-and-applications/
It is difficult to feed any ML / AI machine/model when you have no idea of basic fundamentals. It is like garbage in then garbage out. You need to take care of interest rate differentials when forecasting Forex moves. These are important drivers. Of course you can also look at other currencies for any correlations here. But you definitely need to play around with interest rates and their differentials depending on the FX (country) pair.
Thanks for all the comments. I will be looking into all the resources and ideas you provided. Any other suggestions are also welcomed.
The first question you should ask yourself: Where does your profit/the market inefficiency come from?
Exactly. On the same note you see the same thing in the business world, happening over and over again. Once went to master class AI/ML and the number one reason why so many projects failed was that people never looked at the underlying problem. Let's just throw an ANN to the business because everyone else is doing it.