Hi. I am new to backtesting strategies. I have a question regarding the system I am testing. Currently, my buy / ss signals are generated off of simple Reversal / trendline / s/r breaks. It uses ADX indicator, and ATR. This is all fine. Entries are ok for me. Now exits, I am just starting to work with simple stop loss and profit target. (this is after months of coding my own platform in C++ which does this. Almost finished. Whew!). It seems that, over a course of 3 months data, taking 45 trades (round trips) in the Eminis, if I make my stop loss 3.5 and my target 7, my winning percent is up over 300 percent, with 60 percent winners to 40 losers, and a very positive expectancy. As I move away from this target, like I risk less or more (on stops and targets), the returns get worse (lower winner to losers, percent gain is lower, etc). Now, how did i decide on this stop loss and target? Just mistake (i had the parameters in there as place holders and was suprised by the result). Of course, I will need to increase my testing time (need to "find" more intraday data, while I am compiling my own - my system is working on minute bars), but in the mean time, I think I know the answer to this, am I optimizing my stops? Is this healthly for systems development? I mean, if I move away from these stop "optimized" amounts, my profitablitlity decreases. But I also know, that these numbers are pretty good. I am scared though, that I am confining this system to figures which are just "fitted" to the time frame I am testing on. I know I will know more as I get more testing days, but in the meantime, Any suggestions or advice from the men and women of this board? Thanks.
I dunno, but just to be more certain, I'd call the traders in the pit, MM's or specialists for the securities you're trading and ask them where to put your stops. Who'd know better than them?
lol. If thats the answer, then I guess I should accept my (albeit a small test sample) results to be successful. I acutally like that answer - its what I wanted to hear.
I am by NO means an expert on systems development, but I like to see smoothness when I adjust parameters. eg. if a 3pt STP has a 50% sucess rate and 30% ROA, then a 2.5pt STP should have, say a 40% sucess rate and a 25% ROA. In theory, you should see gradiations across your results as you adjust the parameters. That way you know you have identified a systemic phenomenon that has a margin of safety around which it is profitable, as opposed to trading "magic" indicators.
It suggests that one or more of the logged backtested trades is skewing your results. You need to look through them to see. it's going to be something like one or so particular trades made a great profit or less loss when you used X as a target/stop, or that your system didn't permit you to enter into a new trade because you were "loaded" whereas when you changed the parameters your capital was freed up permitting you to enter another trade. Whenever you have results that have a gap like that simply by tweaking a parmeter slightly, that's usually the cause. Run the same system on a few different markets and see if results are similar.
...let me give you some friendly advice. NEVER reveal your methods or your exact parameter values. The question you pose is very intelligent, and you are headed in a direction to make a ton of money. DON'T give it away. All it takes is one fat boy to think that paper is wise to him to change one number in his automatic trading code and your system goes down the drain. You are proceeding correctly with your optimization if you optimize stops with no takes, takes with no stops, and then spiral around the combination of the two optimized values with small parametric variations. If you are finding broad areas of optimality with no spikes, you have arrived. If you are having mental orgasms with no physical discharges, don't worry. That's a symptom of incipient success. Get on with it with one contract, but watch like a hawk the experienced drawdown and the number of consecutive losing days compared to the backtest result. Also, reoptimize every weekend. If you are using one minute data over three months, you probably have so many degrees of freedom that you cannot possibly be overoptimized. For example, if you trade the first hour you start out with 60 minutes times 60 days which equals 3600 degrees of freedom, which is a lot to burn. To get a hint if the results are random, look at the Z-Score your backtesting computes and learn how to interpret it. If you haven't read the books on backtesting by Babcock, Katz and McCormick, and Pardo, in that order, I strongly recommend that you do so. That will answer a lot of your questions and help you to avoid giving away what you are doing by asking here. Of course, you can tell ME in PMs! I am a very trustworthy individual, and I would never dream of stealing your ideas.
All good advice till now and there's more here : http://www.elitetrader.com/vb/showthread.php?threadid=24127 and in many other threads in Strategy Trading in particular, and EliteTrader in general. My magic numbers after backtesting a year's ES tickdata and generating 1 trade per day were a 7.5 point target and a 2.25 point stop loss. Unfortunately, live forward testing not only invalidated these parameters but made a complete mockery of the entire method. Had I started this project 6 months earlier and been satisfied with backtesting only 6 months of data, would I have enjoyed 6 months of idyllic profits that, with my aggressive Kelly bet-sizing, would have catapulted me into Larry Williams territory? And what are the implications for you, having only used 3 months of data to date?
...stupid (newbie quality) question. To what do you attribute the failure of that beautiful system? Just a random result? Overfitted? You revealed it publicly? Market changed? Other? I had a similar experience three months ago with a system that was so good it made my tongue so hard I couldn't talk. You can guess how it traded. I lost a grand before I wised up. But the lessons learned were worth the cost. Re TheStudent's thread, maybe we should just start a thread called Bad Backtesting Advice for the Credulous Beginner from Disagreeable Irascible Insufferably-Opinionated System Traders. Thanks. - Mike
Thanks for all the replies. I have looked back, nothing skewing my results. I also understand that systems can and will break down any time they please. This is the nature of the market, and it does a good job of keeping us humble. And to you, Hypo, I will take you up on the offer of a PM - but in the future, as I know then I will have better formed questions. I have been in the trading area for close to 10 years, but a systemized approach is new to me, so many questions exist. As for the friendly advice, about revealing my parameters, they are wise words - But I believe in that posting I did not reveal too much (out of your kindness, let me know if I did!) and I will be sure to watch myself in the future. My system does have many idiosyncricies (ie. many parameters! Lots of coding it took me), and I dont think i gave away the "Holy Grail". Thanks to all again. The help is really appreciated.
There is no "HOLY GRAIL"... Best wishes...great systems are a dime a dozen.....great traders are few and far between....I am certainly not one of them......too much anxiety during the trade.