anatomy of a backtest report

Discussion in 'Strategy Building' started by Gordon Gekko, Dec 4, 2002.

  1. I have been reading about allot of systems for sale that show very steep smooth equity curves that would make the system vendor so rich that he could retire in 4 years and live a life of luxury, so why are they selling them instead of trading them.

    Oh and i haven't been around for i while because i went down, BUT i am back in the game trading e-mini's.

    MERRY CHRISTMAS:)
     
    #31     Dec 9, 2002
  2. Appreciated appreciation. :)
     
    #32     Dec 9, 2002
  3. acrary

    acrary

    Then don't build a system where it has an expectation of 9 losses in a row. If you're only comfortable with 4 losses in a row, build a system with profit targets and go for the high win %.

    The formula for estimating the maximum losses in a row is:

    S = ln(1/T)/ln(L) where:
    L = % losers
    S = Streak
    T = # trades you plan to make

    Ex.
    T = 10000 trades in your trading career
    L = .1 or 10% losers

    S = ln(1/10000)/ln(.1)
    S = -9.210/-2.303
    S = 4 expected max. losing streak per 10,000 trades

    So, you'd need to build a system with 90% winners to be comfortable with 4 losses in a row.
     
    #33     Dec 9, 2002
  4. Yeah, I've never seen one for sale that didn't have an equity curve that arc'd smoothly upward at a 45* angle. Hard to believe the idiots that buy those systems can't make an absolute fortune with them. Is there anyone out there who has bought a system and had the actual results be anywhere near as good as the advertised historical results?
     
    #34     Dec 9, 2002
  5. Agreed, and don't buy one with that expectation either. If it is in the expectation of the system traded it will be handled mentally.
     
    #35     Dec 9, 2002
  6. One other factor to consider is data integrity. It may seem obvious, however, a system I've been using flagged a trade recently for NXTL at 11.15 ... All of the quote sources said NXTL hit a LOD of 11.00 (12/9) ... but it really never did (and never hit 11.15 either). I brought this point up in a previous thread and a few folks chimed in mentioning that they've seen this before on occassion. While it may be a rare exception, this might be your "biggest winner" on your newly backtested trading system.

    This shook my confidence a bit.

    I still wouldn't trade without a tested system .. but as you see, there are a lot more things to consider than P/L. Put your numbers to the test and sticking with a system is very difficult when you experience that drawdown in real life.
     
    #36     Dec 18, 2002
  7. ges

    ges

    ...that worries me about my systems... all the data I have access to has survivorship bias. All those stinker companies that have gone bankrupt or been merged or delisted are unaccounted for in my database (QuotesPlus). So, the tests are not really accurate.

    Does anyone know of a source of data that includes all the stocks that have traded, say over the last 10 years or so?

    I would feel much better about my systems if I knew they were being tested on a complete dataset. Survivorship bias can be a real problem with certain kinds of systems.

    gs
     
    #37     Dec 19, 2002
  8. Besides a long only system, as clearly a company going bankrupt will lose money for shareholders, what systems did you have in mind?
     
    #38     Dec 19, 2002
  9. ges

    ges

    It is a long only system and it buys extreme oversold stocks. Looks very good in testing, but I don't trust it because I can't account for what it might look like if all the crap was still in there...Enron, WCOM, etc.

    gs
     
    #39     Dec 19, 2002
  10. My guess would be that your system would for the most part still work even with companies going broke. Especially if it was a situation with the entire market oversold, less so if it is company specific. Let me know if you need a couple of years of Enron ascii.
     
    #40     Dec 19, 2002