The 'Ol Backtesting Dilemma

Discussion in 'Automated Trading' started by pdmoney, Jan 7, 2009.

  1. pdmoney


    I got started using Amritrade's Strategy Desk. I have read that their backtesting leaves much to be desired and I can see why. After creating a few strategies and testing them they work almost nothing like they do in real life as they do in the backtests. My question is, is there ANY software that somewhat comes close to getting the same results LIVE as they do in backtests? Not looking for the Magic Bullet, just looking for what works best for a novice just getting started in Automated/Strategy trading. Any help would be greatly appreciated.
  2. As a beginner I think Tradestation will offer a good start. You have a long way to go but start there, read forums, etc. It's probably not the software though its the user that screwed up, try to find some information on creating a good test and be prepared for lots and lots of testing work.
  3. PDM, IMO the sad fact is that profitable strategies are too complex to be testable because they have two many variables and conditions. Consequently successful startegies verge on being intuitive. They are complex deductions about market conditions based multiple simultaneous observations. Were it not so, every backtesting fool would be making money, and we can't have that.
  4. pdmoney


    Thanks for the quick input. I know the ideas on backtesting and if it worked everyone would be rich etc. etc...however, it seems as though there should be something out there that has some merit to it. I really dont want to be a programmer. How would you compare NinjaTrader to Tradestation as far as backtesting results? Ninja sure seems buggy no? I am only starting out trying some basic Stochastic crossover, RSI crosses and Moving Average crossover backtesting and trial trading with encouraging, yet disappointing results. It seems to me you can make a strategy as complicated as you want. Has anyone had good results with simple strategies??
  5. It's just as likely - more likely in fact - that the problem is with your strategy or manner of testing rather than the testing software itself.

    If you test an over-optimized curve-fitted system and don't factor in commissions, slippage, or the spread then your backtesting results will be garbage. GIGO as they say.

    Eventually you gain some realizations about systems trading. It's hard to imagine that a system with extremely complicated rulesets or multiple different inputs, indicators, filters etc. could have totally random performance, but 99% do. It's also hard to imagine that, by picking the right variables, you can turn a useless losing system into an amazingly, unbelievably profitable one over the period of your backtesting, but thanks to optimization you can do so easily. Unfortunately, these systems always break down in reality.
  6. If you insist, I use E-Signal when I'm not backtesting. It has a very nice autocode feature and has more powerful features for people who want more than Publum backtesting.
  7. Every single member on this board, profitable or not, has backtested MA crossovers, Stoch crosses etc. etc. These strategies do not work. Every single trader goes through basically the same process and tries the same 500 different "systems" to start. By the time you get to system 501 you might at least be thinking in the right direction.

    In the meantime you should read the "AHG: profitable strategy for struggling traders" thread, and/or the "day trading 2.0 for small traders" thread.
  8. Succinctly and well put, Specterx! A little dose of Hershey wouldn't hurt for a well-rounded ET education.
  9. pdmoney


    Thanks, I will look at those threads. Like I said, I am not expecting someone to just give me a strategy so I can go play golf and drink Margaritas on the beach (unless anyone is offering of course ;-) ) but you guys offering the threads to look at and what direction to take in getting started is just what I was looking for. Will try to let you know how things are going on here.
  10. lol i did this....

    i remember the good ole times throwing stuff yelling at the computer that it worked when i backtested it blowing accounts...

    ahhh good times
    #10     Jan 7, 2009