Machine Learning Algo for Trading

Discussion in 'Automated Trading' started by stepseazy, Jun 13, 2016.

  1. conduit

    conduit

    Has anything come out of your 401k.you small see the second after you cash in the chips. Same with all other investments. You keep on asking for specific numbers. Are you looking to debunk the importance and usefulness of ai. Cause if yes we don't need to continue to exchange thoughts. I respect your opinion though I think you would be dead wrong.
     
    #51     Jun 18, 2016
  2. vicirek

    vicirek

    Since we figured out that machine learning does not work for trading anybody has any idea why?
     
    #52     Jun 18, 2016
  3. conduit

    conduit

    Who figured that out? I figured out that it does work. But as said many times before, I don't input raw time series and expect to derive buy or sell signals.

    But if it makes you feel better you could potentially just imagine that ML does not provide any value-added and nor do deep learning networks. Just do what you prefer and what works for you. But then its not that "since we figured out..." but that "you figured out that it does not work".

     
    #53     Jun 18, 2016
    Simples likes this.
  4. vicirek

    vicirek

    My machine learning algo cannot distinguish your writings from those written by guy named volpunter. Do you have any idea why? Are you guys related?


    Back to the topic. ML has been applied to markets decades ago and the consensus then was that essentially money cannot be made using it. Now ML has reemerged due to progress in science and technology with many successful applications working with static data in areas not related to finance. Market data are different. Yes, there are recognizable patterns but what follows after finding it is a coin toss. This is why everybody quotes some success rate with ML but then adds disclaimers and exceptions or says that this level of performance is combination of ML and other factors.

    Obviously you are not going to prove that ML works for trading. But assuming it is true it is still statistical error for this method. I am interested in answering question why is that because negative results sometimes carry very important information about underlying data and answering it may lead to choosing better method.
     
    #54     Jun 18, 2016
    eusdaiki likes this.
  5. conduit

    conduit

    You are of course entitled to your opinion, but with it you basically deny the huge investments most financial trading related firms pour into deep learning networks. It is not only non financial firms. And if anyone deems value in this particular area of Quant finance why would they feel compelled to share their details with you? Certainly there are no such obligations. Or have you shared in detail what works for you and have proven such to the rest of the world? This thread is for everyone to share. Some don't believe in it and others see value. Nobody is obliged to provide proof. But blanket statements such as the ones you made are simply untrue and are not validated by any sound data or research.

    You apparently have not even downloaded and installed nor used any of the free and most used deep learning network frameworks (the ones professionals use, such as Caffe, Tensor Flow, or Theano) yet cast judgements. It is quite difficult to follow your line of reasoning, the one of someone who purely shares feelings and emotions without any personal experience in this area at all.

     
    Last edited: Jun 19, 2016
    #55     Jun 19, 2016
  6. Simples

    Simples

    ML is naturally a part of TA, when done as analysis of "technical" price-volume data. Any more, and you start to include the fundamental realm of markets. This is simplified understanding and everything can be mixed together also, probably even favourably so. I think the consensus here is reasonable, although people tend to argue about differences in semantics rather than finding common grounds.

    Anyhow: I believe everyone agrees that given raw market data input, nobody here expects to plug in a generic ML/datamining algorithm or two, and then wait for a consistent trading plan and execution from the computer. You have to have some basic trading idea, data mine the possibilities (manually or semi-automatic) and then make a first attempt at a prototype. This is true for all technical trading, and you don't even need an ML algo. ML like everything else is just a tool. It's ass-backwards design to include it, if you don't really have a use for it yet. Instead, you include ML in your trading engine when you clearly see it can fit a specific need. Doing it the other way may provide much learning experience, but may waste alot of time towards a working implementation as well.

    What ML may provide is more dynamic non-linear analysis and rules based on deeper statistics. In that way, the same algo could provide many different trading setups, mined from the data itself. So you have more automation in the analysis part of your trading engine. Or it could just be used for one specific little need or anything a little more involved than if-then-else.

    This may sound great, but everyone who's had anything to do with software development, know from experience how hard it is to make something work in all circumstances. Such a general method as ML is even harder to create, monitor and maintain. Additionally, possibilities for analysis is endless, and ML doesn't really solve that problem of narrowing down the problem/questions in the grand scheme of things.

    If the trading engine learns from new data continuously, you'd need to quality check that process continuously as well, as all kinds of artifacts and biases may develop over time. A static ruleset is easier to maintain, but will of course lack the adaptivity you may get from ML.

    This is all very general, and may not be applicable to every implementation, but nobody is discussing specific implementations here yet.

    There's no reason you can't make money in the markets, with or without ML, other than lack of faith - which is the basis for most successes in life. Ie. There's always up-front cost and risk that needs to be paid. Of course, one must create a trading plan as one has to mitigate being wrong most of the time, which is natural in most trading except scalping. So to anyone planning to "trade with ML", learn trading first, as ML won't make anything "easier" (although it may perhaps simplify the trade engine parts it's replacing!).
     
    Last edited: Jun 19, 2016
    #56     Jun 19, 2016
    water7, 931, wolfcuring and 2 others like this.
  7. vicirek

    vicirek

    I did not expect any proof and I do not have any problem with it. I know that there is significant effort and money involved in this type of research but results are probably not that great.

    What I have programmed myself in one of my projects is not far from deep learning conceptually but the problem is that entry and exit points are not optimal. Funny enough the problem is it fires them too early.

    Thank you for pointing out some resources. I have already acquired some literature and programming resources on deep learning but i am hesitating to put more effort into it because it is a lot of work with potentially questionable results. It may not work any better than what I already have.
     
    #57     Jun 19, 2016
    conduit likes this.
  8. conduit

    conduit

    It may not get you better results than you currently have but it may also blow away your imagination. And you are absolutely right, it is a lot of work and requires a larger upfront investment in mathematics and statistics. Not everyone's favorite. I was absolutely taken aback abd fascinated when I saw and verified my first results from convolutional algorithm based deep learning networks.

     
    #58     Jun 19, 2016
    eusdaiki likes this.
  9. vicirek

    vicirek

    What are the input data for CNN? It is mainly used for image recognition. Are you feeding 2D charts into the network or parameters derived from time series?
     
    #59     Jun 20, 2016
  10. Hi,

    very interesting discussion.

    My understanding from the AlphaGo story is that the beauty of it was precisely that the computer had been fed next to nothing in terms of Go rules or strategies, but instead managed to learn from raw data (from real games as well as simulated games against itself).

    There is a consistent literature feeding known predictors (say RSI or MA) into ML algos and getting some results. In many ways you could just say this is merely normalising data / removing noise. But I wonder - is it just a constraint because of lack of data, particularly given changes in market regime? If we had a million years of EURUSD data tick by tick and the market behaviour was stable, couldn't we just feed raw data and get good results?
     
    #60     Jun 20, 2016
    water7 and Simples like this.