Backtesting and Strategy Validation - How Much is Enough?

Discussion in 'Trading' started by bhause, Aug 13, 2011.

  1. 1 ) Slippage was misrepresented

    - Let's take this a step further. When you trade 5-10x a day and your estimation of slippage is off by one tick I can almost guarantee you will be a net loser and see the same upside down equity curve I did. All it takes is to be off a half of a tick. Doesn't sound like much, but it will ruin you.

    2 ) Wrong hypothesis by backtesting engine for entries and exits

    - I think IDBill may have covered this. In the live market I came to find out that it did not get into trades like it did in the simulation and it also SKIPPED trades that it did in the simulation. Which were correct? Hell, I don't know, but I assumed the backtesting engine was correct with its hypothesis. It wasn't.

    3 ) Ego.

    - Check your ego at the door. It has no place in this environment. Perhaps I was too proud of myself at the time to realize what a slippery slope I was really on.

    There are probably a million more. I could write a book on it, but I rather forget about all of it to be honest and just try to look ahead. I'll be done with school soon so hopefully I can get out of here.

    Have a good one
     
    #51     Aug 14, 2011
  2. I know exactly what i need.And to create my own i have to have some programming skills.What,you will show me ho to become a programmer?I have some algos i want to be automated but have no skills to do that and doubt you can show me how to do that unless you are a programmer.
     
    #52     Aug 14, 2011
  3. and your conclusions? backtesting is useless or daytrading is impossible?

    'a million more' doesn`t count,only the first two!
     
    #53     Aug 14, 2011
  4. bhause

    bhause

    Wow - I go away for 2 days and 9 pages worth of replies - you guys are great.

    Thanks for the advice - its encouraging to hear some people say backtesting is reliable and credible, and good to hear the admonishments of others who have had bad experiences with it. Given what I've read, I think I am going to keep working on my model, get more data, and then decide if I want to actually trade it. A few notes based on your comments-

    1) I am not using any program for the backtesting. I am doing it manually based on intraday charts and tracking everything in an Excel sheet. While this is extremely time consuming, I do find it beneficial as I get to study price behavior within the charts in addition to simply getting the results of the backtest. I am early in my trading career and price charts still fascinate me.

    2) Regarding slippage - my system works on 5 minute bars with the assumption that I will enter at the immediate beginning of the next bar after the signal. That being said, for the backtest I am assuming the worst price of the bar in order to make the test as hard on the system as possible. E.g. If the signal is to short and the bar opens at 95, but goes as low as 94.8, in reality I would be shooting to get in at 95, but for the sake of the test I am inputing 95 as the entry. I think giving myself the worst possible execution within a 5 minute time frame should account well for slippage.

    Again - thanks for the help, I will be collecting more data and continuing to paper trade before this goes live for real.

    Thanks!
     
    #54     Aug 15, 2011
  5. SnakeEYE, there are a number of platforms out there that use a psudo code for implimenting strategies. Does require a bit of work but then you have something you know intimately. I'm no programmer but create algos in easylanguage (multicharts) and RTL )MarketDelta). I can't get to first base with Ninja as its a real computer language but when I need Ninja code I go to a couple of guys who implement MY ideas.

    I believe that the trading logic is what is key. Lots of it can be worked out just using your charts.The piano playing of coding up the algo is a much lower level skill and can be bought easily if you don't want to do that part. But then using a psudo language is not that hard.
     
    #55     Aug 15, 2011
  6. 95% of backtesting is probably useless IMO. That is unless you are doing something very unique and you probably will not get to that stage until after years of work. If you are using "time interval" charts you might as well give up now.

    Most of you are really marginalizing this effort. It isn't simple and the time and money you must invest to actually be legit is too much for the majority to handle.

    Best advice would be not to even waste your time. I'm not going to go through every pitfall. Do a google search, you will find plenty.

    But please, feel free to prove me wrong. It isn't impossible. Best of luck.
     
    #56     Aug 15, 2011
  7. No,man i don`t want to prove you wrong at all.I`ve never backtested anything myself.I opened a small account and tried with real money what worked and what was not.
     
    #57     Aug 15, 2011
  8. I again will say that Kid is correct. Most people shouldn't even attempt to backtest anything. Too many pitfalls and the whole idea is ill-conceived at its root:

    http://www.elitetrader.com/vb/showthread.php?s=&postid=3271049#post3271049

    Trying to find a system via backtesting is like trying to go from London to New York swimming accross the ocean. Some may make it, I agree.
     
    #58     Aug 15, 2011
  9. Argent

    Argent

    This whole thread reminds me of a friend who once told me "I tried for years to learn to play guitar and could never get any good at it." One is amazed therefore that there are virtuoso guitarists.
     
    #59     Aug 15, 2011
  10. spindr0

    spindr0

    I'm going to ditto the content of the 1st reply that you got. I trade intraday based on a very simple system. In '08 and '09, its return was nothing anyone here would turn their back on. With the exception of the past week, now it provides just a bit more than pin money.

    Did I mention that the market dropped 50% in '08 and '09 ? :D :eek:
     
    #60     Aug 15, 2011