in the mid-term there is no difference between creating your own and buying one off the shelf. it's all been done before and it's all being done now, creating your own will just duplicate what others have done/are doing. still a worthwhile exercise, to be sure, but there is zero reason to expect it to outperform something you can readily buy.
Thanks! Yes, I'm always interested in hearing about other peoples' methods for monitoring their strategies also. Drawdowns are always painful, but it helps to have a plan for when and if you'd stop trading a strategy rather than just fretting about whether it'll come back or not.
Aaron, I've gotta a question... Why does it seem like drawdowns in real life trading is larger in amplitude than the ones done in hypothetical tests... Yes, I do Monte Carlo via Excel but it just gives a relatively equally distributed results than extreme results in real life...
A couple things that come to mind... First, even if the real drawdowns are about the same as the hypothetical ones, they are going to seem larger just because actually losing money is more painful than hypothetically losing money. Secondly, be sure you are basing your Monte Carlo simulation on purely out of sample backtesting results. If you use in-sample data (the data you developed and optimized your strategy on) results, you will have an overly optimistic estimate for your strategy and it will probably underperform in real life. Finally, it's not impossible for the markets to change and strategies to stop working and actual results to be much worse than an accurate Monte Carlo simulation would suggest. This is when the Monte Carlo simulation work pays off -- it tells you when to stop trading a broken strategy.
What is the correct (or more statistically correct) way to perform Monte Carlo simulations... 1) randomly resample list of trade outcomes 2) resample the daily equity changes I'm trying to understand the best way to run Monte Carlo simulations on out of sample data to get a range of potential outcomes similar to the data points Aaron mentioned in his CBOT presentation. Excellent presentation, by the way.
Possibly you have already done this...Use the mean and standard deviation from your in sample data and generate the simulations based solely on those two parameters. You then will have simulations on totally unseen data. Reason I am offering this alternative is I've read some papers recently questioning the bootstrapping method you suggest...resampling has apparently fallen into disfavor among academeics for a few technical reasons.
By using the mean and standard deviation, do you mean for each data point that I'm interested in? For instance, if I know the mean and standard deviation for monthly returns from my sample data, I would just run the simulations off of these 2 components. And repeat for other data points, like average length of drawdowns most consecutive negative months... Am I understanding this correctly? Ultimately, I'm trying to better understand the systems I'm testing in case I begin trading them. To have a range of possibilities prior to trading to assess the "normal" behavior of the system would go a long way in increasing confidence in the system during inevitable drawdowns.