Machine Learning in Finance is a huge subject, even if you want to focus on NNs. There are many things that you will never learn from a book, for example in what way a transfer function actually affects an indicator. I have put in thousands of hours trying to understand how NNs work, their advantages and disadvantages. If anyone thinks he can read a book and start building successful Trading Systems he is in for a nice surprise. Like any mechanical Trading System, a balance is very difficult to achieve. NNs are just infinitely more complex. The best way to get started into Machine Learning is to get hold of a Genetic Algorithm and use it for rule extraction or optimization. At least in my humble opinion. The big drawback with all this is the need for cross validation or Walkforward testing as it is popularly called in the trading community. Without cross validation your results will be nothing else than curvefitting.
Whoops. This post would fall under the category of 'talking through one's...' I'll use the word 'hat' just to keep it friendly.
I also strongly concur with Murray and zentrader that first one should develop a rule based system that is robust testing on out of sample data using a walk forward technique, then piecing together the out of sample trades from the walk forward into an equity curve. This will give you a more realistic picture of what the system might do in real time. If you want to add NN to the mix do so only as a filter to try to eliminate bad trades. NNs tend to go haywire due to overtraining so they shouldn't be the core of the system as Murray has noted.
Could you please suggest literature/links? I know basics of NN. I use them in my strategy. But my results are far away from my expectations. Results of NN prediction are very similar to usual regression models. The main question is what should be enters for NN???
As OP you are sitting at the center of things. Several of your respondants are vectoring out in various directions, probably to never be heard from again. Let's hope you continue to be so fortunate. Going straight down the rabbit hole is, of course, rcanfield. Headed West we see Mark Brown. He let us know today that data varies from source to source so it is unwise to use any of it for any purpose. One exception, you can use the reverted value if and only if it is not a H, L, O or C. His father, on several occasions, ask him not to fool around with anything as well. Relatively speaking Murray, et al, are heading East. They say do the curve fitting last. The herd is generally headed South and they are all using compasses that have "for prediction only" stamped on them so hard the bearing and crystal are busted. You, yourself speak of "an ongoing development of a mechanical/neural system", roughly quoting you. Naturally you do not have the viewpoints of your respondants since you offer, now and then, a different point of view. I get it that you are not digging a hole, headed, West, East or South. Or even North as yet unmentioned but known to be other than your different view. Sounds like you simply want something to make money. Murray says he studio's stuff. He uses the word strategy and he mentions an indicator, MACD. Mark is "setting" on something. Predicting and making money do not mix as we all see from the corrolation in deep reports that have been referred to endlessly by people on the sidelines. I use totally mechanical systems, as is well known to all. They were modeled, and developed over the years. They perform unbelievably and are astonishing. You will not want to know about them since they are different than anything any respondant has suggested and further since you are different they will not be like what you do not have as yet. I determined how I got into the niche I am in and why others do not look at things the way I do. You are building a niche also. Congratulations. You and the others are new to trading so they have not had enough time to learn about things. My time has been divided equally into two parts, each 25 years long. At the end of the first part I wrote down what had hardened into place and I have used it more or less since the beginning in a totally totally mechanical manner. Recently with the advent of PC's and then software, I noticed that mostly everything a person can do when monitoring can be done automatically. Scripts accomplish that. By this I mean all the annotating and the converting of displayed information into high utility informative results. This sentence coming up is the crux of the matter. All of the above mentioned people and all of the other respondants, including you, have left out one aspect of making money: the nitty gritty of having timing that eliminates risk. 50 years ago I looked at things and decided to only trade with certainty in the space all of the time. It was a time when things were done manually to about an extent of 100% or maybe more. I could not do anything that was unnecessary so to speak. The reason was that unnecessary things would be consuming time that I did not have available to spend. Most all of your advisors are doing a lot of unnecessary stuff which they have a lot of time to be able to do. They all have computers and none of them are trading mechanically. I started out without a computer (except for 702's and 705's which I declined to use since there was no data feed nor instructions available) and began to trade mechanically just using as little time as possible. You think differently today and I have not changed my thinking and it is different than yours or mostly anyone elses. so you have announced the path you are now conscious of. you can use your skills to become mechanical. You can use a computer where I use just little old me. Write down the description of what you do to trade. Program it so it works on a computer. Use the computer. I built my mind 50 years ago. I am mentally programmed to make money. I look at a screen and simply use sports memory to trade all day long. What goes through my mind unconsciously is about four times what I could speak because I can only speak so fast and people can only listen so fast. There is no way to connect myself to others except by building their brains. This is fun to do and is a short process. For rcanfield, Murray, Nitro, "the predictors", et al, there seems to be a missing link for them to do what I suggested to you. That is, write down what is being done to make money trading. They are too young and inexperienced to be able to do that as yet. Age is not a problem, everyone gets older. Experience is gained through spending time. They are not doing the experience thing anymore as you found out. I was time starved when I was young in the late 50's. I had to focus my use of time. I cut out the superfulous stuff like reading books and all other secondary things. I could only do trading in the markets and make money. The exponent of the coumpound interest formula was the only thing I could deal with of importance. I used the width of the price fluctuations which the market gave to me as the interest to compound; that was a given. The market offering is a given. You extract all of it. This is the rough mechanics of trading. Write down what you do and make software to do it. Use your computer to extract the market's offerings. When I am trading and mentoring, it sounds like a broken record of a mechanical process. We sit there and take out all the market offers as the day proceeds. You use the word passion that applies to something or other. It definitely is not your education or training. Mechanical or NN is not a passion thing either, I believe. They are both just two kinds of browmie baking or tooling for some applications. I looked at my trading and tried to find a passionate aspect of it. It isn't money; I give that away a lot. It is our habit to bid the highest and longest on auctions at charity events. The auctioneer knows to know my name as well as my number. Could you let me know what you are passionate about? I was doing a conference call today for the open; attached are two trades. The gap inbetween was when we did some education. My comments on it are elsewhere. OOps you can't add attachments to edits. Too bad.
Could you clarify this? Is it the fact that a system is based on neural networks or merely that is automated that leads you to conclude that it will not work?
This is easily remedied through any number of means: early-stopping, limiting the number of hidden neurons, etc.
I don't understand why you believe that's the case: There are certainly ways to constrain neural network training to avoid this problem.