Backtesting vs. Realtime results

Discussion in 'Automated Trading' started by jonnyman, Oct 19, 2005.

  1. man

    man

    are you sure you are talking about annualised sharpe? annualised return divided by annualised std? if not, multiply your figure by 3.4, then it makes more sense.
    annualised 0.4 would imply unbearable vola at your return level. so i assume you actually trade a mod sharpe of 1.3, which is quite okay for the time we are in.
     
    #11     Oct 20, 2005
  2. kotika

    kotika

    This statement is incorrect. There is no way to annualize a sharpe ratio without knowing the average number of trades per year. Roughly speaking when small sharpe ratios are annualized it might increase with the number of trades per year. But if you want to be more precise this is no good.

    This brings us to E.Thorpe on two counts. First, his book "Beat the Dealer" in which he revealed the system for blackjack: yes thats right he is the inventor of card counting. He is also inventor of quantitative convertible arbitrage in his 1968 book "Beat the Market". His hedge fund was also the first to put to use pure statistical arb around 1980.

    Anyway, in the book there is plenty of time-tested wisdom about mathematical expectation versus the real results. If you cannot get the book, or dont trust me that blackjack has anything to do with what you're trying to do, try to learn about the Kelly criterion which Thorpe advocates using in the market in his paper (can be found at bjmath.com).
    http://www.bjmath.com/bjmath/feature.htm

    Below i try to summarie the known facts about the Kelly criterion.

    The important thing to always to remember is that what you should really care about is the log (the mathematical logarithm function) of your money. Given a sharpe ratio S, the log of your wealth W will grow like:

    ln(W) = S^2/2 * N

    after N trades. Your money grows exponentially! So greater sharpe ratio increases the best possible growth rate quadratically. But there is a tradeoff, if your optimization reduces the number of available trades in order to increase the sharpe ratio, it may not be worth it.

    The above growth rate is only possible if you use leverage L in such a way that given the average profit per trade m and average standard deviation per trade s

    L = m/s^2

    Unfortunately, as you will surely find this formula asks you to leverage way too much, so you need to use some common sense. In gambling pros use half to a third as much leverage as the above, but in the markets, esp futures markets i prefer to set limit on the maximum acceptable loss instead.

    Still , as a practical recommendation optimize the growth rate
    g= S^2 * N

    this will not harm you, and actually do you good even if you dont use Kelly criterion to optimize leverage.

    So, dont overleverage, and good luck,
    K

    PS unfortunately these formulas are a bit tricky to use, but its well worth it.
     
    #12     Oct 20, 2005
  3. Im a bit confused.

    Say I have 40 trades a year

    .4 ln 2/2*40 =
    .4 ln 2/80 =
    .4 ln .025 =
    = -1.475

    what did I do wrong? How is it possible to have a negative sharpe ratio?

    Thanks,

    Eric
     
    #13     Oct 20, 2005
  4. mindfull

    mindfull

    How does the survivorship bias apply in backtesting?
     
    #14     Oct 20, 2005
  5. DoctorJ

    DoctorJ

    Depends what you're trading ofcourse. If your system is set up to trade stocks, options or warrents your testing may have introduced survivorship bias through limiting the universe of stocks you trade - for example a margin list or index components.
     
    #15     Oct 20, 2005
  6. Thats not untrue, but has anyone calculated the effect of survivorship bias?

    Id like to see what this invisible edge is. If there is a bias, then there is someone taking advantage of it.
    And if no one is taking advantage of it, then it likely isnt a bias, its slippage more or less.

    The Markets have an invisible edge, they go up or they go down. I guess that too is a bias. Can we calculate the Markets bias? Thats what a system does. But you cant call anything anything unless you can measure it. Unless your zen.
     
    #16     Oct 20, 2005


  7. I swing trade pullbacks. Over the past few years of trading a mechanized system (but not automated), I've found that I was unable to take about 15% of the trades generated by the system. Two main reasons. (1) Trades are sometimes generated by a very quick spike in data, or in a single trade that spiked a bar, but which I would not have been able to follow up on. (2) There are sometimes market conditions, which have been frequent this month, when overall weakness presents more trades than can be taken, even in a large trading account. While the system logged these trades, they weren't practical in reality.

    As has been mentioned, system vs actual performance will depend much on timeframe and strategy.
     
    #17     Oct 20, 2005
  8. mindfull

    mindfull

    Are your trades manually keyed in or system generated? It sounds like they are manually keyed in. But if it was automated what kind of orders are they?
     
    #18     Oct 20, 2005
  9. man

    man

    kotika

    referring to your post:

    "This statement is incorrect. There is no way to annualize a sharpe ratio without knowing the average number of trades per year. Roughly speaking when small sharpe ratios are annualized it might increase with the number of trades per year. But if you want to be more precise this is no good."


    you might realise that the alternative investment industry standard for calculating sharpe ratios is to annualise daily returns and divide them by annualised daily standard deviation. where this is not possible, like hedge funds, people use monthly data. same process. this has no reference to the number of trades. now, if people do not annualise their sharpe ratio and divide return by std (always on days), then multiplying monthly figures by about 3.5 leads to correct results since you annualise average log yield return by multiplying with 12 and you annualise std by multiplying with the square root of 12. i am little vague in that because especially the return side annualisation finally depends on how people calc their return (log or percentage).
     
    #19     Oct 21, 2005
  10. kut2k2

    kut2k2

    kotika: "Given a sharpe ratio S, the log of your wealth W will grow like:

    ln(W) = S^2/2 * N

    after N trades."

    S = sqrt[2*N*ln[W]] = sqrt[2*40*ln[1.3]] = 4.58
     
    #20     Oct 21, 2005