I totally agree. I don't think I ever tried to argue against this point. My broker is IB, but the data feed I use for charting and backtesting after 2002 is IQFeed, which most would say is quite complete in its delivery of unfiltered tick data. For data prior to 2003, I used data purchased from ANFUTURES.COM.
I do not mean to be general here, but I want to put in my opinion without spoon feeding. If I am obvious you would not listen, so read between the lines and it will be far better for you to discover for YOURSELF what will work for YOU. Backtesting in my humble opinion can help you identify the dead spots and can be quite useful in the further development of your system. The fact that you entered commish in your backtest results was a good thing you did. Try to get as much wiggle room you can get in the long backtest you did, which is a good thing(you will need it during real trading). Your post here to ET took a lot of courage, and I understand that you posted the backtest to find out if your system was robust enough for further study/development/trading. Your short term(near scalping) system has held up over this long backtest period which is not easy to do. Stick with your plan. Just use 1 tick slip on each side for evaluation. The point that you enter/exit in a liquid fast moving scene will help your fill evaluation. Your on the right track. Try to backtest again and see how it effects your entries with the above tick per side allowance ok? Does it reduce the # of entries? Another question, Does your system see profit in a trending way? losses in a trending way? You hope for this actually as then it is something you can see clearly! Try to discover a filter, as I am convinced this is all you need. Without me attempting to rewrite your system, try adding a filter to reduce your # of trades. I think your on to a consistent winner here with some fine tuning. This is a case where you can use your backtest results to discover what is happening during your losing trades. Reverse engineer to eliminate 50% of those trades by altering your signal generation portion of your system. No need to curve fit or over optimize just get rid of some trades, ok? Please report back. Michael B. P.S. Perhaps less greed, fewer trades and more contracts is the key here. With proper "MONEY MANAGEMENT" (not trade management) you can get a smoother "REAL TRADING" equity curve, but you need room to wiggle. P.S.S. I am guessing because of the nature of the momentum type of scene that you are entering and exiting in, might be more effective with NQ. P.S.S.S, This is an after-thought. Since you have so much data from ANFutures. Try to break your backtest up into segments of time and attach a range value to each segment. This way you can alter your signal generation based on Range Rank.
Not impossible but not easy either. Essentially when you trade one of these types of systems you become a scalper. Yes you may have the ability to automate your system. However if you are not manually making sure all is going according to plan then I can't see how a system with such a small positive average trade (if the avg trade is in fact positive) can make out. You will have the missed entries, and you will also have the missed exits. Missed entries are easier to deal with. You simply have no trade. Missed entries will reduce your system's profits and further reduce your system's average profit which is already vapor thin. Missed exits require a specific set of rules to be implemented. Essentially it is a money management routine, and as such warrants backtesting to see if implementing it degrades the system's performance. As the ES is a noisy instrument it isn't unusual for it to hit your exit price, retrace back through your entry price by a certain amount, and then finally trade through your exit price (all on the same 1 minute bar). So how does ESignal's backtesting engine score that one? Is that a small profit (assuming you could get filled), breakeven, a loss or a profit at your profit target? Your choice of MM rules will determine which is correct, and a backtest of those rules will show what they do to overall system performance. The bad news is that you won't be able to do this type of testing on 1 minute bars. As Mr. Sub has suggested, you will need to test on T&S data. And as has been pointed out by others, your system might not even be profitable because the assumptions used in testing might be overyly optimisitc. So that vapor thin $4.67 avg trade might turn into -$5.0. So likely this particular system is just a whole lot of churning, and not much earning. Don't feel bad, I've been there and done that countless times over. I can't speak for others but the "secret" is really in keeping loss sizes small and having a particular trade setup edge that allows you to enter when the market has a high probability of moving in your direction (at least a little bit), and having superior trade management skills so that you protect your profits, minimize your losses and squeeze every ounce of profit out of your winners. In other words it comes down to the same thing that makes traders who trade on longer timeframes successful. You must find a way to turn the risk/reward and win% picture in your favor. And as some have stated in other threads, you don't necessarily have to have a high win % in order to scalp profitably. A 2 to 1 reward/risk ratio with 50% wins is a very good system regardless of whether it is a scalp system or a long term system. But these systems don't grow on trees. Play around with the equity curve simulator on the following site - there are many roads to profits: http://www.hquotes.com/tradehard/simulator.html And one final bit of editorial. The average trade statistic is one of my favorite system statistics. I routinely discount backtests that do not give an average profit of $25. Once you take care of commissions and slippage and no fills, etc., you might have a profitable system if it starts out with at least that amount. There are just too many things that can go wrong with a marginally profitable system in the actual execution to bother with them.
Fine. Then rests the slipage see Mr Subli above I agree with him also as for the number of trades. Too many trades is equivalent to too much friction or leverage. And then before commiting real $ you can still simulate trades with real market to further confirm that your theo hypothesis were realistic. And then you can play with risk of ruin calculation because even with an edge especially when it is small you can be in a drawdown easily. Also there are unexpected drawdown due to accident that is out of normal statistical forecast so you must always put a supplement margin error especially for the begining of the equity curve.
Another point, I agree with your automation goal. As it is difficult to trade day in and day out without losing concentration and trade errors. Believe me I know what I am talking about. Automation will reduce the slippage in your system, if you look at automation with the correct attitude. It is far better to monitor than to actually press the buttons. BUT and a big BUT...You must WRITE out all the scenarios that you are monitoring for...Lot's of "WHAT IF?" questions. You must be able to react in a near automated way to the DEFINATE automation execution errors. Someone once quoted to me the percentage of "actionable" monitoring instances. They kept a log, even though its a very small percentage its still thee and you WILL need to trade manually, eventually. Michael B.
When I first started out a few years ago I was hell bent on backtesting everything. Then I realized that computer backtesting is useful only for narrowing down some theoretical methods. The only way to backtest is manually. Step through each signal and manual record, entry etc. John
mrpace - Couple quick thoughts which may already have been mentioned: 1. 64% win/lose trade ratio with only a 65% win/lose $ ratio is a thin starting point for purely similated results 2. With a narrow margin of error, your simulation assumptions need to be seriously boundary tested - e.g., assuming either slippage or a spectrum of failure to execute rates (if you have tick data you could do this with varying assumptions of how long your order takes to execute and then insure that your limit price is available within a certain traded volume - you can't just flatly assume that your order will execute at the closing price of some price bar). Even if you're using limit order to enter, you need to factor in the impact of slippage and see how sensitive your results are to different slippage rates. 3. Take a more granular look at your results. Since you're using 1999 to present (and also have a whole in your data), you should examine (under a variety of boundary tests) where you're supposedly making your profits. If you show disproportionate distribution of your simulated profit accumulations (i.e., if you're not generating profits at a more or less consistent rate over the whole time period - be warned, even simple moving average systems will show profits during high trend period but get seriously destroyed during choppy periods) and/or a disproportionate % of the overall profit is generated by a subset of the "winning" trades - then you could have an illusion of profitability which would disappear if the conditions (as noted in #2 above) were changed and/or you hit less consistent market conditions (the period of time you're using contains a lot of very trendy periods - assuming that will persist over the long haul is a potentially dangerous assumption). Good luck.
I scalp similar like you. If you can perfect this method, you can trade any market all time. Most people here in ET are not really elite trader, they are system traders. When their methods breakdown and it will, they are in trouble. But your style 'go with the flow' with average profit of $30 and stay in the mkt for around 8 minutes is the right way to trade something noisy mkt like es. Finally, I found a trader.