Limitations in backtesting

Discussion in 'Strategy Building' started by oddiduro, Dec 5, 2003.

  1. Monday, put up a stochastics oscillator. Use any time frame you like. Note the numerical position of the osscilator 1-2 sec. before the end of the bar, and then note it at the beginning of the next bar. There will be dramatic displacement.

    This will adversely affect a backtest.

    Connie Brown discusses this in her book.

    Regards
    Oddi
     
    #11     Dec 6, 2003
  2. ...thanks for the explanation. Does that have to do with the famous end-of-bar follies, allegedly designed to fake out EOB traders (perhaps it was you who first noted that here)?
     
    #12     Dec 6, 2003
  3. maxpi

    maxpi

    Imposing an end of bar trade on a market is probably not going to represent reality, the market is pretty fluid and bar intervals are dogmatic and arbitrary. The dogma is bound to diverge from the reality all the time.

     
    #13     Dec 6, 2003
  4. ...the reason for decision making on the the end of the bar is that the transition from one bar to the next is the only time you can get a good handle on how your fill related to the decision to make the trade. In all sincerity, being a domschitz, I am all ears if you have a better way to relate reality to backtesting results.
     
    #14     Dec 6, 2003
  5. ...thanks. I forgot it because Harry posted, which sends me screaming the other way. Going back through my signals for the last 60 days looking for cases like this (I found plenty) suggests that the use of supporting but unrelated systems may help to solve the problem. If they agree, no problem. If they don't, abort the trade.
     
    #16     Dec 7, 2003
  6. Glad to learn it seems you've got something you want (although the something you found would be beyond my comprehension!). :p
     
    #17     Dec 7, 2003
  7. #18     Dec 7, 2003
  8. Sorry hypo, can't take credit.

    I actually found this out while trying to mechanize and optimize Jack Hershey's ES method. The system performs much better in real time than it does on the backtest because of the nature of the stochastics indicator and the way it behaves in real time vs. delayed time.

    It is a safe bet that the CCI oscillator and moving average crossovers have the same phenomenon.

    I am leaning towards discarding oscillators as not backtestable altogether, and focus strictly on price, time and volume.

    Regards
    Oddi
     
    #19     Dec 7, 2003
  9. Likely their timing would very much depend on how much they are misleading! :D

    :)
     
    #20     Dec 7, 2003