jimmy, I do look forward to it everyday when I turn it on.... its just a few bad days will leave a sour taste.. as soon as it has a good day again i'll be excited..... Everyday I run tons of backtests on differnet variations of my core strategy, hoping one day I find one that is slightly more consitent.. but I never find one.... I shoudl just be happy knowing that I have a strategy that WILL make money in the long run and a lot of money at that... So I should just stop complaining and let the bot take care of itself...
As I get prepared for monday, I am backtesting as usual and have come up with a question. I am looking for opions on the merits of what this would mean. Here is the scenario: You have your core method, over the long run its profitable.. everything good.. Secondly, you have a method based entirely on the core method with a few filters. If you look at the MOST recent history, say past 4 months, the backtests of this strategy DESTROYS the origional method. The total profit nealry double, expectancy per trade is 2 and 1/2 times the origional.. and the equity curve is a thing of beauty.. nearly a 45 degree angle... perfect slope upward with very small drawdowns... the # of trades in this period is 250. Here is the kicker, long term just like the origional its profitable.. however if you exclude the recent 4 months and look at the prior 3 years before, then the origonal method outperforms this strategy.. both are profitable but the origoinal has a much nicer equity curve and just flat out performs the new method... So here is the question.. do you trade this new method? My thoughts thus far on this is that perhaps the new method is more in sync with current market conditions and thus is the better method to trade RIGHT now.... my second thought on this is the new method is just curve fitted...... The number of trades is around 250, so the sample size is still realitivey small but its not like its only 20 trades.. which makes the method seems semi legit. So the question is what are your thoughts on these results? I am hoping to generate long term equity charts on both methods tomorrow to try and get a better picture of what I am looking at.
OK Sample size is something that you should have resolved up front, not after 17 pages of posting and going live with your system. There a several methods of determining statistically significant sample size depending on your requirements for "confidence" and "power analysis". Since you have only 250 data points, it is possible that your test results are invalid. I am just wondering why you waited till now to deal with this. Here is an internet link that might help http://www.statsoft.com/textbook/stpowan.html#power_doe3 Steve
The few filters you added after a losing week - they are probably tailored for the recent market action to get rid of a few disturbing losers. Hence, the better performance over the past few months but not over the 3-year testing period. However, the observed performance on the 3-year period is more reliable as a predictor of future performance. Maybe changing the characteristics of the system would make you feel more comfortable - e.g. trade less but with a higher winners %, a bigger account, a smaller contract than ER2, etc. ++
Remember. Beware over fitting to your sample set, and the larger the sample set the greater probability it simulates the problem correctly. You can always tweak values of systems to take advantage of very unique trade opportunities which have happened over the past week. There is no reason to believe that the tweaked values will perform just as well over a longer time frame.
Steve: Not everyone has tons of experience, and/or targeted training, and/or read every book, good or bad, on trading system design, statistics, etc., etc. If you can get through the twerps and twits who occasionally jump into a thread, ET is a great learning resource. I think Frosty is doing very well - in this thread he is demonstrating his system and getting help in refining/revising it, in another thread he offers his expertise, software development and coding. He is doing fine! Jack
The market is a changing system, a system which performs well three years ago will not always perform well next year. There has been an explosion with the number of automated systems you are trading against recently, Id imagine that this may fundamentally change how the market works. Looking to see how the equidty curve of backtesting has changed over the years will give you a good idea how valid the system still is recently.
Steve46, Perhaps you did not read the whole thread.... but what I was stating was if I test ONLY the past 4 months and not all of my data then the sample size is only 250 trades.... thats only for the 4 month period not the nearly 4 years of data I have................ the system I am trading has been profitable over the 4 years... I was simply stating that I have a system that performs MUCH better recently that the current system but is still profitable over the 4 years
I think I understood you clearly. I come from a commercial background. I wouldn't be able to make any assumptions based on a 250 data point set. Nor would I trade a system based on 4 years of data unless the frequency were extremely high. Simply not enough data to obtain a significant confidence level. Good luck Steve
Unfortunately Jack, the truth remains after your comment. Either you have enough data or you don't Even though Frosty is "doing very well" his system may fail if he bases it on a sample size that is too small. If at some point he gets it right, but has no capital left to trade with what is your comment worth then? This is why system design details like sample size are best resolved in advance of live trading.