my ATS is pretty simple.. a crossover system w/ ADX filter. it looks viable... but is obviously curve-fitted. NQ - 5$/tick using 6$/transaction commisson 10$/transaction slippage (1 contract) the drawbacks: limited data for testing. currently have ~ 3 months of tick/volume data for NQ. (i use volume bar charts, 800v-2400.) should i purchase data to test properly? would purchased data test w/ similar results as my current data. i use IB data. would the purchased data give skewed/different results? (and be a waste of time/$$). i was thinking i would hook it up to IB's simulated TWS and fwd test the system on varying charts w/ slightly altered parameters to see which tested the most efficient. for any suggestions you have, i am grateful. jim my setup is: IB data metaserverRT tradestation 2000i traderAssistant
You say the system is obviously curve fitted, do you have any data to support that, or just assuming it has to be curve fitted because it looks viable? How many trades did your system perform in the given test period? What happens if you backtest with your 3 months of data but with different parameters for your indicators? Does even a small change to your parameter values cause huge changes in your P/L? Is your system still profitable if you add or subtract around 25% to each parameter. For instance if you were using a 50 period MA, is it still profitable if you use 37 period MA instead, or a 63 period MA? Concerning purchasing data.. I would not waste my money on it until you have a better idea how robust your system is. Use Ib's backfill feature and get some data on es, ym, and er2... run your same system on those. How does your P/L hold up?
You can get free historical tick data for the NQ from www.opentick.com (I'm not affiliated). You just need to register with them. I don't know if you're trading with a US broker, but your $6.00 per R/T commission is high. A figure like $4.50 is more in-line with normal retail rates (it can be considerably less if you do significant volume).
i assume its curve - fitted. can't prove it thus far. it will trade ~ 200 +/- ( would be much higher w/o adx filter. but still profitable) depending on which chart i have up. exits i use, etc. it does remain profitable w/ minor parameter changes. assuming changes are in the ballpark so to speak. i use IB broker, so yes.. i'm a bit high on commissions, but what the heck.. i like overkill. do you think fwd testing ~ 1-2 months is decent testing period? thanks very much
thanks Jango for the opentick site. looks like a handy source. only thing, i havn't figured out how to use the data they have w/ my platform. may not be possible...
if some parameters "close" to the optimum produce negative returns then you should RUN AWAY test all paramaters across a gradient -> randomly fluctuating returns = system is CRAP run on slightly different time frames (in your case different vol bars) -> randomly fluctuating returns = system is CRAP run with different start/end dates -> randomly fluctuating returns = system is CRAP TRIPLE check the entry/exit prices used in the calculation change the system so that all logic runs of -1 bar index and only use open-0 to enter -> if it is now unprofitable then the system WILL fail real time if the system is ONLY profitable on one instrument then the system WILL fail You need to backtest with a TICK simulator, not a back tester, AND test as close to 120 days of TICKS as you can Foward testing in real time is absolutely essential
Hey journeyman, great info, thanks. I am new to automated trading and backtesting and would like to ask if you can elaborate on this comment please? Cheers.
3 months of data is not enough to determine anything. Get at least 5 years and backtest that before using real money
journeyman, thank you for the suggestions. i have applied a few of the techniques you mentioned. varied time frames, data periods. ad nauseum it seems. but, i expect traderdragon2 is correct, my data sample is way to small. the tick simulator was an interesting idea. i had never heard of such. i'll see if i can find the program. i'm afraid i didn't quite grasp the statement gpzany referenced either. "change the system so that all logic runs of -1 bar index and only use open-0 to enter -> if it is now unprofitable then the system WILL fail real time " .. was it to do with the way the entry signals are sent? thanks again for all the feedback!
most backtesting is total fantasy simply because a bar with four values (open, high, low, close) is smoothed with respect to time. In realtime the OHLC relationship is never determined until the bar actually closes (will the close really be ABOVE the OPEN, is that current LOW going to be the actual LOW etc...). The only value that stays constant is the OPEN. The OPEN will always be the OPEN, even on the first TICK of the new bar. So -> if your system really is a system and not simply optimised random CRAP then it should be able to generate a trade entry when the current bar closes and the new bar opens. Then the trade entry signal really comes off the -1 bar and the entry is open(0). This situation reflects reality with a high degree of accuracy, your only slippage being whether or not you actually get filled at open(0). In my experience any system that cannot produce a profitable back test with these changes has NO, I mean zero, chance of working real time. Please note the following qualification: all my sytems testing and design run off sub-minute, ie TICK bars and the longer your time frame the more likely you can ignore my rules. But I wouldn't bet on it. And for sub-minute time frames 5 years of backtesting is unnecessary and you are wasting your time. Forward testing is far more important.