Questioning backtest results

Discussion in 'Automated Trading' started by ctrlbrk, Oct 20, 2008.

  1. ctrlbrk



    I am new to ET, and would like some expert advice to help interpret my backtesting results.

    Let's take one script in particular that I wrote. I like to be in front of my computer when it is running, so the time frame is 12:00pm to 3:59pm (market time). I allow multiple entries per direction, have defined 1 tick slippage, and the quantity is 1 contract per order.

    When you run it against ES for Oct 1-Oct 20 (present day), I get the following results (see image attachments for complete details on this period, sorry for image quality, I had to compress a lot to get it under the forum file size limits):

    39 trades, 92% profitable, max drawdown $(-217.50). Profit factor of 10.35, average time in market 0.8min.

    When you run it against ES for Sep 1-Sep 30, I get the following results:
    12 trades, 100% profitable, profit factor 99.00, average time in market 2.3min.

    Now, this seems too good to be true. I will spend quite a bit of time in the coming days to continue running this against new market data. I only looked at about 3 days of charts to code my strategy script, and I know markets fluctuate dramatically over time. I already realize no strategy will work for extended periods of time, etc etc.

    Finally, here is my question -- asking for advice from experts -- what should I make of these backtest results? What I am really asking is whether or not I can 'believe' the results as I see them, and assuming yes, would it not be realistic to expect similar results from the real market in the _very short term_ period (matter of days)?

    Thanks for your input.


    <img src=>
  2. ctrlbrk


    image #2
  3. ctrlbrk


    last, image #3
  4. What is your data source?

    What is the data timeframe? Tick data or 1 second bar or DOM or ?

    Do you buy the ask and sell the bid?

    Since the average trade only last 0.8 minute why are there less than 3 trades a day?
  5. ctrlbrk


    IQFeed real-time is the source and the interval is 1 minute bars.

    The entry orders are limit orders, the stop losses are market.

    The average trade is 0.8min, but there are only enough 'signals' generated for approximately 2-3 trades a day.

    Thanks for your input.

  6. You have a very high probability setup with larger losses than gains, nothing wrong with that. In your testing did you inadvertently use any information that was not available at the time of entry? This can happen in a test if you use an indicator based on the end of the bar but entries are intrabar and you test on bars. One way to never have that happen is to test only on ticks and build your bars as you go.

    I would say the next step is to run the strategy live against a simulation account and see how it does.
  7. Looks promising... relatively low risk... Turn her on and let her rip.

    If your one foot in and one foot out... forward test in a sim/demo account. Have you tried your algo intrabar?
  8. ctrlbrk


    Thank you for the advice. I wrote it in ninja, and am by no means an expert with NT yet. The code looks for signals 'onbarupdate' where bar=1 minute intervals. If you look at screen shot #2, all entries occur on the minute, so afaik, no entries are occurring based on future data.

    I agree on running it against the live market. I will run it 'live' in NT thru my paper trading IB account and see what happens.

    Thank you.

    (edited to fix screen shot #)
  9. ctrlbrk


    Great, I hope it works real time! I have tried it on 75 tick intervals, it does not work well. It was designed around indicators that were built using 1 min data.

    Thank you.

  10. Do not use last trade data... Your real world results will be skewed. Can you enter trades using market orders exclsuively?
    #10     Oct 21, 2008