2 years is not nearly enough. I use 5 years for brief analysis as it includes various market conditions, this is crucial (2008 was a blessing for testing). After that I use 10 years of data for verification and optimizing. It should also work for the 20 year sample but I'm more forgiving as some things in the market have changed and when it comes to equities, I don't have the delisted ones.
You don't need anymore history than 2 years of intraday data. Backtesting more than that is irrelevant. In my case, I use 2 years of intraday, tick and 5 minute, and that's plenty of data. Anymore data than that and the optimization borders on irrelevance.
Thanks again to all for your suggestions. 10Kguy I actually tried that last year in trading the cash ES. I decided to try and make 2pts per day and did that for about 6 months. Discovered 2 big problems ...the 1st was that all your doing when you set a profit target is limiting your upside. You don't limit your downside. IOW I would make my 2 pts then not trade wile the market ripped up or down another 10. If I jumped in and out trying to make that 2 pts I also ate a lot of commish. The other problem and this is more personal is that "life" got in the way. I could not sit at the computer and trade un-interruped. We travel quite a bit and most travel days are not good trading days. I threw in the towel in the summer with that idea because I began to hate it. I'm just not cut out to sit and daytrade ES. With options its problematic to set profit targets because you need to take what the market gives you. With higher volatility sometimes you make your profits (losses) quickly..low volatility it may take longer. However I do believe you are definitely right to pre-determine (then have the discipline to follow thru) how and what you do with your profits. I know they always say worry about your losses and profits will take care of themselves. I'm not sure I buy that anymore I think regular withdrawals help keep it all in perspective.
I don't know if you are deliberately misleading people or what the case here is. Last 2 years have been very calm in comparison to 2008 and the beginning of 2009, when most of the normality rules were broken. Most systems working great in the last 2 years would fail in that data range. I'll take a wild guess and say that you have only data for 2 years and/or tested on the last 2 year data, now trying to justify your position and keep it relevant.
Two things might help: 1. Kelly betting is proportional. As you account bal increases, your betting size goes up with it. You can do as previous poster suggested, adjust size every 10-20K increase in equity, and keep size constant before the change. 2. Your model/system should have a vol or vol of vol component. When market gets volatile, it does not take a lot of position to have the same exposure. Remember Kelly rule says you want to bet a size proportional to mean divided by variance, or sharpe/stdev. So sharpe is your edge, while stdev is your exposure. njrookie
If your model is intraday and your holding period is measured in minutes or hours, two year of data is not too bad. There were plenty of mini-crashes or mini-huge breakouts in the last two year. While more is better, two years are not too bad, especially if your model has a dynamic/adaptive dimension to its parameterization. njrookie
I have 4 years of data through late 2007, and optimize monthly on the most recent 2 years. This keeps my models robust, and it isn't a case of having limited datasets. Certainly the 2008-2009 period wasn't good, but figuring that the most recent 2 years is all that needs to be analyzed, indeed there is plenty of data for a 5 minute intraday system.
It's good that you have interest in trading different products/strategies. Just make sure you size them correctly. On any new strategies or trade ideas, I tend to put on the minimal amount and follow the trade for a few month until i understand the risks and see positive pnl. If it doesn't work, then just cut it with minimal damage. Don't expect hitting a jackpot in your first try!
I agree on targets - however you can make a system with profit targets that will limit the downside, and you can adjust the difference in returns with leverage When I said targets, I meant something you can beat (some days beat easily) and feel comfortable for the rest of the quarter that you're on track. Likewise when you fall down to below quota, you know something needs changing. This system is used by me and allot of traders that I know of..... Also in regards to allocating capital, the way I do it is I rate which instruments I like the best (usually based on profit factor, sharpe and liquidity, but sometimes other points), then calculate how many contract(s) to allocate based on the estimated monthly returns with one contract in backtesting or replaying data. For example, silver is extremely volatile, so even if you wanted just as much silver as crude in your intraday portfolio, you'd probably really only want 1 silver contract and 3 cl contracts. If your system only work with a few instruments, you need a serious look over. It doesn't mean that it's curve fitted, but could mean it's incomplete or not very good, or that your system works in trend or inside periods only because some contracts inside all the time while others trend all the time. The ES is kind of crappy to trade imo.....look at NQ, TF, DAX, YM indexes instead. The spread is much better and volatility makes easier trading. The ES is the whole market, while NQ TF DAX and YM are blue chips only. Again, opinions of a relatively new guy.......