Is data mining for trading patterns impossible?

Discussion in 'Data Sets and Feeds' started by bulat, Mar 18, 2005.

  1. Since you are no longer using it do mind posting your set up? I would love to take a llok to see what you were doing. I am sure you can still find your set up at this link. http://www.trade-ideas.com/RecentWindows.html

    Thanks,
    D
     
    #31     Mar 23, 2005
  2. My experience has been the opposite. I have found greater success with data mined strategies than those designed by myself.

    This shows we can employ different methods, and we can each be profitable. To each their own.
     
    #32     Mar 23, 2005
  3. Good for you! That's what I thought would be the case at the time, but I just never got there. It's a never-ending evolving battle :)
     
    #33     Mar 23, 2005
  4. bulat

    bulat

    #34     Mar 27, 2005
  5. The paper described in non-layman's term what I said in my previous message :)

    So I will just summarize it here,

    The more explosive patterns (i.e. fits the tested data better) are chosen by most market participants as these patterns are more obvious to find (i.e. simple data mining) thus will not last (i.e. not working in the future).

    The key to correctly identify recurring patterns is consistencies in the patterns' behaviour in various type of market conditions. Consistency can be measured by means of stable percentage in their edge.

    As I mentioned before, one important area of data mining in financial data that is not really touched by both academics and actual market participants is the behaviour of smaller time frames (e.g. 15-sec, 5-sec, etc.) due to various reasons like not enough computing power, hard disk space, etc., so this area will be an area that gives an edge to the smaller players in the market.

    Notice that many successful independent daytraders are tape readers - the lowest time frame of all data, even though they use many other forms of technical analysis to assist them in their final decisions.


     
    #35     Mar 27, 2005
  6. Anseld

    Anseld

    hard to do? yes.

    impossible? no.

    one should not automatically accept results that are derived from data mining processes, but one should not automatically reject them neither. you need a very analytical approach to understand what is valid or not.

    as for taleb, he is right on some things, but he's also wrong by being unwilling to be more open-minded.

    his theories are worthwhile, but only to an extent, because he's the extremist. his vision is limited to black and white and nothing in between. people who worship him like god obviously have not had much success with any pattern recognition, and people who disregard him completely obviously have not experienced enough of disorderly events to understand that some of taleb's warnings have validity.
     
    #36     Mar 27, 2005
  7. bulat

    bulat

    I don't know where you are getting this from. Of all the successful traders that I know, most trade longer time frames (days to weeks). That's not to suggest that there are no successful daytraders, but I don't see them as the majority of successful non-professional traders.
     
    #37     Mar 27, 2005
  8. bulat

    bulat

    And what would the analytical approach that can separate the valid patterns from noise consist of?
     
    #38     Mar 27, 2005
  9. Hello:
    I am going off to have Easter dinner, and thought I would look in on this post.

    The reason that patterns do not have a significant edge, is that the underlying price series does not exhibit stationarity. Its interesting that with all the extensive background that you folks claim to have, you don't (none of you) mention this very simple statistical "fact of life".

    If you would learn to characterize the data series properly so that you know when price exhibits stationarity (or is likely to exhibit stationarity) you could then look for patterns with confidence that they are the result of non-random behavior.

    Basic examples of stationarity occur at or arround earnings reports, economic reports, end of month (also known as "window dressing") bond and note auctions, etc. This is quite basic stuff.

    As for how to do this (whether "by eye" or with a software approach) it is best to use both. A few posters have already published methods that work. Use search.

    Best to all,
    Lefty
     
    #39     Mar 27, 2005
  10. Anseld

    Anseld

    that's really beyond the scope of this thread because i don't know what your data was tested on and how it was tested. the only thing i can say is that one should try to cover and review *all* bases thoroughly when studying a system. investigate not just the core but every event. yes, it can be an excrutiatingly tedious process, and this is where human is stronger than machine. interrogate why things happen or why they don't. machines can crunch numbers but they don't have that intuitive judgement, so they can't perform that effectively. (maybe they can, but i haven't encountered any system like that yet.) but humans can.
     
    #40     Mar 27, 2005