My struggle - is it Data Quality or something bigger?

Discussion in 'Strategy Building' started by sysdevel99, Nov 30, 2015.

Why do most published strategies not work

Poll closed Dec 14, 2015.
  1. The system developer doesn't know what they're doing

    3 vote(s)
    42.9%
  2. The system developer is purposely misleading to sell more books or ads

    2 vote(s)
    28.6%
  3. I am too stupid to replicate their basic rules and mimic their results

    1 vote(s)
    14.3%
  4. They work off bad data (e.g. not dividend adjusted)

    2 vote(s)
    28.6%
  5. Something bigger that I don't understand

    4 vote(s)
    57.1%
Multiple votes are allowed.
  1. I searched for data quality, didn't get any hits, so my apologies if this is already covered -> in that case please point me to the links discussing it (that's my guess on the poll for this thread anyway).

    Disclaimer:This isn't my first rodeo, although duration doesn't mean competence, just because I've been trading for a while doesn't mean I know what I'm doing.

    I like to get ideas from various sources, from blogs to reverse engineering systems from collective2 to books to interviews ...
    Then I try to code those systems to see if there's anything there (I do like to write trading system backtests with the hope to find gold). Now here's the challenge, more often than not (19 our of 20) the systems described in these data sources don't work. Not only do they not work they are actually losing systems but not losing enough that you can just reverse them either. Now granted, if I've a winning system I basically don't share it either, but at least I don't "claim" that this is a winning system ...

    So what is the reason that these systems don't work (cause many go through a significant effort coding them, showing equity curves, nbr of trades, ....)
    1. Are the guys just full of it and they don't know what they're doing
    2. Are they purposely misleading to sell more books or ads?
    3. am I too stupid to replicate basic rules and mimic similar results (may be the most realistic option although I don't know how anyone could answer that)
    4. Are they working based on bad data (that's what kills me with TA all around, if I use thinkorswim for charting and use prophet or native chart for dividend adjusted stocks I'll get very different results in trendlines, retracements ... -> in system trading if they don't use dividend adjusted data maybe that's why their system results look so different)?
    5. Is there something else?
     
    stepandfetchit likes this.
  2. Xela

    Xela


    That was my initial experience, too.

    The figure did reduce to some extent, as I gradually acquired more experience and judgment at assessing - according to their sources - the probability of "there being anything in them", which is commonly very low.



    I basically don't, either. Partly because I want to be secretive about it, partly because I never have anything to sell/promote/self-promote at all and wouldn't want to, and partly because I have no interest in influencing how others trade. I'll offer more general observations from my more general trading experience though, especially if I think they can perhaps quickly/easily avoid someone being/becoming misguided.



    My guess (from what you've said there, though I may be mistaken and/or even maligning you, here!) is that you're being considerably less selective than I am, with what you look at. (I wouldn't dream of looking at most blogs or at Collective2, and I examine ideas only from sources I consider very well-qualified to be sources ... which, admittedly, is perhaps such a subjective and vague thing to say that it contributes almost nothing to the conversation at all ... :wtf:ops: ).
     
    777 likes this.
  3. sysdevel99
    I have had similar thoughts and experiences.
     
  4. Snake oil malfunction?
     
  5. Some of them for sure.
    Others however seem to want to do the right research but come up with completely different results. I wonder how much of that is based on the backtesting environment versus the data versus the actual logic [the logic seems pretty basic in many cases].
    e.g. ninjatrader versus custom app versus multicharts or tradestation ...
     
  6. Geez adunno if I had to consider all that... Makes me sick just thinking about it- - That's why I quit my career as an engineer. The best thing about discretionary trading, is, the simpler, the better. But that's just me.

    ...Good luck w/ that stuff
     
  7. dom993

    dom993

    Reason #1 : a lot of published backtest results come from unreliable backtest engines (Tradestation to be specific) ... when a LMT order is deemed filled on a price touch, the backtest results are simply garbage.

    Reason #2 : commissions and slippage are not accounted for.
     
    d08 likes this.
  8. Do you think they eventually figure it out or they've no intention of trading their system anyway?
     
  9. schizo

    schizo

    LOL. When I set out to find the holy grail, this was exactly what I had in mind. I wasn't looking for gold. Since there were so many bad ways to trade, I figured it would be better to simply write the most hideous system with the largest drawdown and merely reverse the code. Unfortunately, it didn't work. :mad::mad::D
     
    NoVoodooHere likes this.
  10. tommyho

    tommyho

    I like your point Voodoo. I'm a noob, trading only 2 yrs BUT what has worked for me is finding something simple, developing a plan that works for me and sticking to it. Too much info gets me way too confused. The less anxiety and questioning the better for me. Thanks Tom
     
    #10     Dec 1, 2015