Backtesting Trading Strategies

Discussion in 'Strategy Building' started by Arthur Deco, Jan 7, 2010.

  1. Thanks, Organ. But there are multiple levels of jokes here. "Mikasa" is the uncountable reincarnation of a (im)poster who has been hanging here as long as I have. He is a confirmed old Random Walker (and bachelor), which world view he came to by stumbling from one lampost to the next after last call in his very cosmopolitan home town. The joke is that although he doesn't believe in determinism, he DOES follow the Brownian motion when it "randomly" occurs. Don't let him fool you. Let me.
     
    #21     Jan 7, 2010
  2. The Fourth Law is:

    Never include more than one "strong" parameter in your test system.

    In plain language, never attempt to optimize more than one parameter. The classic case is the crossover of two MAs. That has two "strong" parameters in the form of the two MA periods. You can spend a lifetime "optimizing" that and wondering why the "optimizations" always shift.

    What is a "strong" parameter? The day of the week. The blowout volume. The period of a single MA. The length of a blowout doji's tail.

    Contrariwise, you can have as many "weak" parameters as you want. Every system perforce has them. The start time. The stop time. The timeout time. The chart period. You know they are "weak" because when you vary them the results don't vary much.

    How do stops and profit targets fit into this framework? They are "strong" parameters. So if your system already has a "strong" parameter, you are not allowed to add a stop or a profit target. Only if your system has no "strong" parameter inherent in it can you optimize EITHER a stop or a profit target. What is an example of such a system? A good one is reversal on retrace to a pivot.

    Why does the Fourth Law exist? Because you, being a retail ET trader, have limited horsepower. Both computationally and intellectually.

    Is there a Fifth Law?
     
    #22     Jan 7, 2010
  3. Maybe not the Fifth Law but an important point: we know backtesting works because it's discouraged by "gurus" who don't want their "methods" objectively evaluated for obvious reasons.

    Example:

    Internet Guru (Jack Hershey): "backtesting is not a viable means for finding anything out"

    Actual Trader (Bill Eckhardt): "I know of no way to validate conjectures concerning technical trading without back testing"

     
    #23     Jan 7, 2010
  4. An astute observation, 6. Thanks. We have both been there, haha!

    The Fifth Law is:

    If you see it in a book, in a magazine or on the web, don't waste your time testing it. It doesn't work.

    Contrariwise, if you say to yourself, "Did I just see what I think I saw, and it's so absurd it can't possibly be?", code it and test it.
     
    #24     Jan 7, 2010
  5. Thanks to the respondents. Periodically I like to whack ET with a heavy ball peen hammer to see how it resonates. I got what I wanted.
     
    #25     Jan 7, 2010
  6. pwrtrdr

    pwrtrdr

    If you believe what you write on this board, would you also post what your BT system has performed over the last several years. If you dont have a system that IS working and you have all this "knowledge" regarding BT systems.... we'll I think you got your answer to "continue" to BT and worse post here about it..

    In other words you have honed a very lucrative system using your BT methods correct ? In fact you must have a couple "systems' that work, because there is no grail or one method... correct ?
     
    #26     Jan 22, 2010
  7. Does experience with a thousand failed systems count? Or only experience with one working one? From watching volume behavior on a one-second chart day-in-and-day-out for years, I conclude that there are in fact TWO Holy Grails, and every profitable trader knows them, but each trader knows them by different names. Thousands of blind men, but only two elephants.
     
    #27     Jan 22, 2010