results: backtesting vs market replay vs live

Discussion in 'Strategy Building' started by calhawk01, Jun 20, 2013.

  1. dom993

    dom993

    Congrats!
     
    #21     Jun 23, 2013
  2. Bob111

    Bob111

    #22     Jun 23, 2013
  3. Pipflow

    Pipflow

    First on demo and then on live, it should be the same result if you have kept all the parameters the same.
     
    #23     Jun 26, 2013
  4. Sergio77

    Sergio77

    #24     Jun 27, 2013
  5. Sergio77

    Sergio77

    = trading system paradise:)
     
    #25     Jun 27, 2013
  6. Hi. If I have indicator X and use it on a backtest with an entryprice calculated from previous bar ex. entryprice= MvgAvg(10 Bar, Close). Let's say the buy trigger is

    Buy = low < entryprice and X> 10 then buy at entryprice.

    The test comes out 70%W/ L, good profit faactor , excellent Sharpe Ratio. It seems to good to be true since in real life trading, the last bar on the right is still being formed so does the fact that I did not offset x mean I am "cheating" in the backtest? I changed the buy trigger to

    Buy = low < entryprice and X(1 bar back) > 10 then buy at entryprice and the backtest goes to 55%W/L . Does that imply that I looked" into the future in the first buy statement? (due to the incomplete nature of the right most bar? )
     
    #26     Feb 28, 2014
  7. There is no way of knowing the answer to that without getting into the nuts and bolts of what is happening. Maybe the right-most bar once traded at your buy price but by the time X >10 that trade would never trigger because price is on an upward tear, all within that same bar that is forming. But the simulator logic might be set up so if the buy price ever occurs within a bar then a simulated trade is triggered even though that price is ancient history (in "bar years"). Hence the over optimistic results.
     
    #27     Feb 28, 2014
  8. great point Mike. That scenario you painted is precisely why I tried to set back the x>10 trigger 1 bar back since that value does not change as the "close" value of the last bar is still being built. Using 1barback got me the 55% which is a far cry from 70%. To try to get more accurate results I guess the bar replay is a better tool than the backtest system where is reads ohlc statically. or... trade for real in small amounts.
     
    #28     Feb 28, 2014
  9. Read SEC rule 611(b)(8)

    Basically says you can be filled at worst bid or offer of preceding second.

    Understand the execution venues can match 3 million trades per second with laser precision. Quants are paid big $ to take advantage of this one second flickering quotation exemption to maximize profits when brokering trades.

    Read up on Hide and not slide order types only available to certain market participants. Understand Queue position is not necessarily FIFO.

    That being said:

    Build and test your algo's using worst bid and offer of preceding second for your fills will provide most realistic results.

    Assume Stop orders are only released after a trade prints through the stop price.

    Assume Limit Order are filled only when quoted through.

    Using Second Bars with the above simple rules will provide an accurate and realistic representation of real world results.

    Even If you test with tick and trade data you MUST assume being filled at worse bid/offer of preceding second for market orders and order types converting to market orders for execution as that is the relief the execution side has been granted by our regulators.
     
    #29     Feb 28, 2014
  10. pocketchange, do these rules apply tightly to the spot Fx where quotes are fragmented? I purposely moved timeframe at a much longer level in order to avoid these backtesting "fantasy pnl" traps. My backtests use 30m for timing entries .
     
    #30     Feb 28, 2014