Statistical tests to ascertain if the real time strategy is inline with the backtest

Discussion in 'Trading' started by woodja1, Aug 25, 2010.

  1. woodja1


    Hello everyone. I'm new to ET, I trade mean reversion on US stocks, also the VIX. I was wondering whether anybody could give me some advice on how do I know if my strategy is working i.e. if the real / walk-forward daily returns are from the same distribution as the returns in the backtest.

    I understand this is a grey area, and there are no simple rules, more judgement based. But I'm looking for something more scientific than rules of thumb like "if the drawdown > 2 * worst backtest drawdown then stop trading".

    I understand there are statistical tests that can be used, and that this is what algo traders in investment banks / highly scientific hedge funds use.

    E.g., simplistically, we have an empirical distribution formed by the backtest's daily % returns (say 250 results), we have a smaller set of daily returns from the walk-forward test, say for 21 results only (1 month). Each month we need to show the small population is still not inconsistent with the backtest population to some degree of confidence. No idea how to best do this.

    Any ideas would be much appreciated. Thanks, James