backtest for 3 years, blow up in 3 days,

Discussion in 'Risk Management' started by jacksmith, Mar 23, 2009.

  1. May I just address the first one first...

    1. So one entry rule in the market long or short for 25 years (98.5% actually has a position). How would you curve fit that one rule if it is in the market all the time and almost always reverses on exit?

    Would a system in the market 98.5% of the time with 34 trades a year, be optinmized but a system in the market 10% of the time and 300 trades a year likely less so?

    So time in the market and # rules is irrelevant to over optimization? Appreciate examples to help me understand.
     
    #131     Oct 25, 2009
  2. iuykcif

    iuykcif


    It's true that "the market all the time and almost always reverses", if we talk in terms of event frequency.

    The problem is another. First, all long and short entries are often characterized by different
    reversion dynamics.

    Second, the frequency of reversals is not so important if most of them will lead to losses
    due to the fact that some drawdown are practically unsustainable for most accounts.
    (In presence of unlimited capital it's clearly always possible to devise strategies that will
    never lose.)

    In addition to that, most importantly, the amount of reversals in any case may be insufficient to cover the
    drawdown expenses which have occurred to maintain (or grow) the position until reversal. (And this, as
    matter of fact, is statistically more likely to happen with short entries.)

    Percentages are not much meaningful as measure of performances and can be deadly misleading.

    And yes, in general a higher avg number of daily entries will make a possible good result more reliable.


    If you have 2 strategies and P some "suitable" (net) performance Index:

    - One with positive performance P and 1 avg entry per day
    - Another with positive performance only P/10, but with 10 avg entries per day,

    then, without hesitation, choose the one with performance P/10 ! ;-)


    ____________________
    Tom
    My <a href="http://www.datatime.eu/public/gbot/2009Oct22/default.htm" target="_blank">autotrading</a> journal
     
    #132     Oct 25, 2009
  3. I am pretty simple and don't understand much of what you wrote. Perhaps you could clarify a bit more?

    I take it from your last statement that a system that cuts out the 90% of the action to be in the market 10% of the time on 10 trades, is less likely to be curve fit then to one that has 10% of the trades but is in the market 100% of the time?

    Here is the equity curve of this strategy. I do not trade with it by the way as drawdown can be a bit much though recovery is swift. More of a discussion on strategy development.
     
    #133     Oct 25, 2009
  4. iuykcif

    iuykcif

    >I take it from your last statement that
    a system that cuts out the 90% of the action to be in the market 10% of the time on 10 trades,
    is less likely to be curve fit then to one that has 10% of the trades but is in the market 100% of the time?

    i don't recognize exactly what I was saying in your statement above. I am just saying that
    the less the avg number of daily entries, more easily a strategy will be subject to a computer optimization
    which will reveal to be *specific of the past data* and therefore useless for forward (real or paper) trading.

    We all are able to produce fantastic equity curves, fitting past data. It's, then, forward testing that can offer
    you hints about possible issues.

    Along with backtesting results you need to show also forward testing results. If they are "aligned" then a more meaningful discussion can take place.


    ____________________
    Tom
    My <a href="http://www.datatime.eu/public/gbot/2009Oct22/default.htm" target="_blank">autotrading</a> journal
     
    #134     Oct 25, 2009
  5. The3 problem here is the fact that the public is always behind the form ande out of phase with a cycle. Something will work, until it stops and nobody will know when the cycle is going to change.
     
    #135     Oct 25, 2009
  6. This particular system was finished in February 2007. So that is the walk forward.
     
    #136     Oct 25, 2009
  7. RedRat

    RedRat


    Hi Alex,

    I am also interesting in NN. Could you tell what were your enters and exits? What do you predict with NN?

    Thanks
     
    #137     Dec 17, 2009
  8. Muskoka, dont suppose you would care to elaborate any more on the strategy behind the equity curve?
     
    #138     Dec 17, 2009
  9. What accounts for this is market manipulation, you can apply optimization and backtesting, you can fit the best model but at the end of the day market makers and banks have the enfluence over price and they will "blow you up" as you say...guess I would call it ...shake you down.
     
    #139     Dec 17, 2009
  10. The last time anyone here posted usable information was back in 1997.
     
    #140     Dec 20, 2009