With backtester need pay attention to execution levels, so algos exec levels wont get much better than would be with incoming tick data and probably cant fully use last bar for simulation if on live it will be calculating on each incoming tick. I saw tremendous returns on first simple momentum based algo and automated parameter search implementation, then started to see it has to be too good to be true and it only found flaws, after making test conditions harder it got more realistic and most momenutm based algos tested have month of gaps on out of sample data or dont even break even.
Hey guys, pretty nice discussion so far. How do you get adapt the CNN's design to work well with time series data? I was under the impression that these nets are better suited for machine vision tasks, and other nets such as RNN's (LSTM in particular) and Markov Nets are better suited for time series tasks. Anyone has experience using Reinforcement Learning on market data? I've gotten bitter-sweet results using Heuristic search algorithms and would love to hear about other experiences with such approaches.
I'm not into CNN's, but I recall someone in the industry hinting that the data is first converted into a two-dimension image--at least that's how he was doing it. I assumed he was referring to a type of common stock chart image is fed into the CNN.
I hadn't considered colors...but I suppose so...if it adds relevant meaning. Train it with the input being a chart with some number of days/hours/minutes, output being that chart, but with so number of days/hours/minutes into the future??
Here is an interesting simple approach and basic R code. You can use your own attributes and the code to get you started after you download the R package. You do not need the DLPAL software attributes to play around. The code is well explained. Good point also about ML being GIGO. I think the simple classifier gml will give you an idea if there is high noise in your attributes.
Feeding algos with matrix or pixels with colors seems very inefficient idea. Would it be used for the purposes of finding ways for pattern recognition?
Im not sure if it is the right approach... if anything it'll allow to reuse techniques developed for machine vision (such as CNN's) and *maybe* avoid some of the pitfalls of working with timeseries data... Although RNN's do seem to be a more natural fit for the problem at hand. My initial impression is that LSTM's should be a good fit. Phased LSTM's also seem quite promising (they are LSTMs with a time gate). They show good results dealing with asynchronious streams, this may be a good fit when dealing with tick data as it is received from the market at irregular times... https://arxiv.org/abs/1610.09513
Btw. This paper seems interesting & relevant to this topic. https://arxiv.org/ftp/arxiv/papers/1603/1603.00751.pdf