No, seriously... My dad and I got our automated trading system set up about a month ago using his "rules" and after tinkering a little bit the results are incredible. In fact, too incredible. The backtesting of our system makes it appear more successful than believable for him or myself... Using TradeStation with EasyLanguage for futures trading we currently dont have it setup to take into account commision, exchange fees, or slippage. What are some other common "screw-ups" made during the backtesting? Looking to avoid going live and losing bundles of money. The skeptic within me simply refuses to believe it can be this simple to become infinitely rich.
Well, sorry to be pesimistic but you most likely (> 99.5%) have an error in the code, it's never that easy... Check that you don't look 'forward' in your backtests, that's the first error that comes to mind when an 'unbelievable' results shows up.
If you use limit orders then you need to setup "look inside the bar testing" and test on the smallest time frame. Limit orders can yield unrealistic results. I don't know what contract you are trading. Every contract varies. You can't look ahead in Tradestation 8. After you do all the basic sanity checks, you need to check to make sure it is doing what you think and then go trade by trade and evaluate the results. If you don't use a stop or loss limit then the equity curve will always shoot straight up... you don't see the open drawdowns without deeper inspection. There isn't a simple answer for this though. It requires a lot of work. You can start to run it in a paper account and see if things still work.
Test it forward in sim/paper mode using a live data feed. Next, if possible, test it on a low value instrument to lessen any risk of major disaster. Back testing is not representative of real world results.
Believe the skeptic within you. If it appears to be too good to be true, then it probably is. A good way to check is simply to reference individual backtested trades with the appropriate charts. Do the charts verify the trades precisely? And could the trades have been made with the systems you have in place? And the fact that you are not including slippage and commissions could kill your results, depending on your trade frequency.
Using the built-in reserved words and functions increases the probability of a flawed backtest. Custom written logic is always preferrable.
I once found myself in a similar situation. As it turns out, on just one particular line of code, I had forgotten that arrays start at zero instead of at one. My system was basically granted the ability to see a day into the future when dealing with historic data, so of course it worked extremely well. Once a "-1" was added to the appropriate line of code the whole thing collapsed.
The 2 most common mistakes when backtesting (and I speak from experience) is "slippage" and "accidental forward testing". I remember when I started backtesting with WealthLab years ago, I came up with "unbeatable" trading strategies, only to find out that I was actually forward testing by accident. Another and less common issue is that a particular strategy that is deemed to be extremely profitable, seldom if ever actually trades. For those systems, the actual sample set was way too small to be considered viable. If you are new to all of this, and it seems to be too good to be true, then it is. If you have discovered something that you think works well, make sure you backtest it over the last 11 years or so. You will probably find every kind of market that ever existed in that timespan. Also, if you are trading very liquid instruments, be wary of the last few years where HFT's have dominated these markets.