Questioning backtest results

Discussion in 'Automated Trading' started by ctrlbrk, Oct 20, 2008.

  1. ctrlbrk

    ctrlbrk

    Thanks for the advice. The indicators I am using are mainly built on the typical of last bar, not on last trade.

    The limit orders I am using are mainly calculated on the close price of last bar.

    Yes I can technically do market orders on entry signals, but my success percentage decreases as some of my signals call for me to enter at a few ticks +/- the last close for highest percentage.

    That last part concerns me as well, and is why I am looking forward to live testing as well w/paper account. But the way I understand Ninja, it only considers a market order filled if the price is penetrated, not just touched.

    Thank you.

    Mike
     
    #11     Oct 21, 2008
  2. If you are using 1 minute bars then your results are not valid. In writing a trading system you cannot buy and sell on the same bar (of any time frequency) because you do not know the order of the high and low for that bar. Rewrite the trading system logic to only allow a buy or a sell on a single bar.

    If you were to trade your existing system against real time data the flaws would be quite visible.
     
    #12     Oct 21, 2008
  3. ctrlbrk

    ctrlbrk

    I am not sure I am following, can you explain more please?

    If you look at screen shot #2 it lists entry/exit times and the # of bars before the trade closed. The average is only 1 bar, but it is not the same bar it was entered on.

    If I am still in trouble, please help me understand.

    Thank you for the valuable info.

    Mike
     
    #13     Oct 21, 2008
  4. ctrlbrk

    ctrlbrk

    Ok I spoke to soon, I looked closer and I was wrong, it does show the same minute for enter/exit on some trades.

    So, are you saying that say 12:01pm it decided to buy @ 901, and the tick data showed it hit 903 which was my profit target 30 seconds later, it is not valid if I sell @ 903?

    Again ninja would only consider it a successful trade if the limit order penetrated the price, not just touched it.

    I also am including 1 tick slippage on all calculations on the backtest, just in case.

    If I am still missing this, I look forward to your advice to help me better understand why this scenario won't work.

    Thank you.

    Mike
     
    #14     Oct 21, 2008
  5. How are you ensuring that your entry/exit targets are actually hit? Being inside the OHLC is not enough.
     
    #15     Oct 21, 2008
  6. ctrlbrk

    ctrlbrk

    Alright, so thinking about what you said... not knowing what order the low/high came in on that bar...

    But wouldn't Ninja know, since I am providing it tick level data? Even though calculations are on 1 minute bars, it has the tick data for the backtests.

    Thank you.

    Mike
     
    #16     Oct 21, 2008
  7. ctrlbrk

    ctrlbrk

    I guess that is the unknown, or at least I don't know how to find out except in real-time vs backtest.

    I was basing it on the Ninja rules which say if you place a buy limit order @ 900, even if 900 is touched you will not get a fill until 899.75 comes along.

    I do grasp what you guys are saying... OHLC ordering... but isn't Ninja supposed to use the tick data it has to figure that out?

    Thank you.

    Mike
     
    #17     Oct 21, 2008
  8. Trust me, there isn't any better expert to ask than me. Your backtest is not long enough, and I suspect curve fitting just on that data. No, you absolutely should not expect similar results. Stretch it out past 8 months and I'm sure you'll think otherwise, guaranteed.
     
    #18     Oct 21, 2008
  9. If you are testing against tick data then you have prices in time order as they occurred and it is obvious that you don't buy and sell the same tick.

    If you are testing against 1 minute (or any time period) bars you only have prices in time order as they occurred from one bar to the next bar. Thus you cannot buy and sell on the same bar and you must avoid doing so for trading system design.

    You have to be certain that in real time the entry price occurred before the exit price and not vice versa. From what you have posted so far it is not clear to me exactly what you have done.
     
    #19     Oct 21, 2008
  10. I'm not a Ninja expert but I do not believe that it looks inside the bar.

    I test only on tick data and I build the bars in an array. That way there just never are ambiguities and bar interval can be a testable variable.
     
    #20     Oct 21, 2008