How reliable is the backtesting result?

Discussion in 'Automated Trading' started by TAM, Apr 21, 2011.

  1. TAM

    TAM

    To intradaybill:
    Good point too. Holding positions for 3 days is not my intention, it is just the optimized result for AUDJPY. If you check the GBPJPY result I posted, the average holding period is about 1 day. For USDJPY, it is about 1.5 days. To me, this is a medium-frequency instead of intra-day strategy. I use 15min because it should generate more trades and more accurate back testing result than longer interval(30min or 60min), is that right?

    Regarding the win rate, my thought is that 50% win ratio may not be too bad, since the average winning trade (1.99%) is more than twice of average losing trade (-0.84%). Overall, the average trade makes 0.68%, or about 55 pips. Even assuming 4 pip slippage, this should still be profitable based on back testing.

    My back testing period covers 2006 to 2011, which I think comprises of most typical market conditions (smooth, crash time). If you check the attached return curve, the best period is in 2008 turmoil. This point (perform better in market disruption) was also proved in real-time test last month when Japan earthquake happened.

    The above are my thoughts. I welcome your further comments. Thanks
     
    #11     Apr 22, 2011
  2. Murray Ruggiero

    Murray Ruggiero Sponsor

    What happens if you optimize across the 3 pairs, or even add other spreads and find the best set of parameters on the basket ?.
     
    #12     Apr 22, 2011
  3. TAM

    TAM

    To Murray:
    That should be a good way to test. I haven't tried that. To my limited knowledge, Ninja platform allows me to test on a group of pairs but show results for each pair separately. I may give a try when I have time.... Thanks
     
    #13     Apr 22, 2011
  4. Murray Ruggiero

    Murray Ruggiero Sponsor

    That how I designed TradersStudio because this is the way I develop systems.

    http://tradersstudio.com/Overview.aspx?PageContentID=20

    Here is the link to the overview page of TradersStudio
     
    #14     Apr 22, 2011
  5. TAM I do not believe this will be profitable in real life. The win percentage is too small, and no better than counting cards. Needs to be at least 65% for meaningful edge. Too many trades taken, and not enough profit. The losses are skewed left.
     
    #15     Apr 23, 2011
  6. I could help checking it trough that there are no bugs :)
     
    #16     Apr 23, 2011
  7. +1
     
    #17     Apr 23, 2011
  8. TAM

    TAM

    I posted the real life return in previous post already. Want to post again here.
    Win ratio is one aspect, don't you think average winning trading is also important?
    Eventually
    Net Profit = Average Winning Trade X Number of Trades

    Is this correct understanding?
     
    #18     Apr 23, 2011
  9. problem with back testing, is that it doesnt take into account the actually liquidity available to execute your trades, so what usually happens is that you miss out on the entries and get smoked on the outs..
    stop trying to cut corners, and do it the old fashioned way. the way it has always worked


    Happy trading :)
     
    #19     Apr 23, 2011
  10. TAM

    TAM


    If you read my previous post carefully, you should notice I also have real-time trading results. The problem is that my real time result has only a few months, that's why I come here seeking more experienced advice.
    I did compare back testing and real time results, for most trades the difference (or slippage) is about 3 pips/per trade. Given the average trade wins 55 pips, is this acceptable? As a matter of fact during last month's earthquake period the real time result way outperformed back testing result.
    The reason I care a lot about average winning trade is to leave enough room for illiquidity that you mentioned. In addition, I purposely choose 15min as time unit, and my average holding period is usually at least one day, so iliquidity impact should be limited since this is not high-frequency trading.

    I don't quite understand what you mean. Can you elaborate more about old fashioned way?
     
    #20     Apr 23, 2011