Not exactly market making and you wouldn't cancel orders; the point is if you think the fair price based on some analysis is 100 and you'd happily buy at 90 and sell at 110 then you can put a limit buy at 90 and limit sell at 110, and just leave them there. GAT
I have a couple of questions about bootstrapping weights: The faster variations of EWMAC (2:8, 4:16) seem to get positive weights, despite these consistently losing money over the long run when operated on their own. My prior for these would be 0. Is there an issue with my bootstrapping process? I have removed these variations manually, but I see this as meddling. My system accounts for costs, and trades some expensive instruments (oats, coffee etc). I have implemented the breakout algo from the blog. Again, I removed the shortest lookbacks (so I have 40, 80, 160, 320). My expected Sharpe ratio came out lower with these in (0.59 vs 0.68 before). I bootstrapped the uncertainty on the Sharpe, that's 0.15. Should I just ditch the breakout algos? My approach doesn't seem very scientific to me.
The bootstrap will only give something a zero weight if there is overwhelming statistical evidence that the rule loses money. This is a much sterner test than you glancing at an account curve and deciding that the thing is crap. Having said that I remove anything from my backtest which has a cost, measured in sharpe ratio units, of 0.13 or higher. This includes EWMAC2:8 on everything I trade, and would include EWMAC 4:16 on most other things I trade. This isn't cheating since we can measure costs just by measuring the turnover of a rule and ignoring whether it makes money before or after costs. Finally of course I tend to use methods which upweight the importance of costs versus pre-cost returns since we know the former with more accuracy. If the reason the fast rules are crap is because they have higher costs, then it is reasonable to downweight them. You may want to (re)read this post. You've answered your own question! The Sharpe ratio on each backtest isn't statistically distinguishable from each other. So it's worth adding these rules if (speaking like a Bayesian) your prior opinion is that they will improve your performance through diversification (correlation less than 1, so yes), and you have no reason to think the breakout rules will have worse performance such as higher costs (I certainly can't think of any reason why); the data isn't strong enough to move you away from that prior. Nor to me GAT
Hi GAT, can you tell us what is the name of the test? I believe you mentioned it in one of your talks before but I couldn't catch it properly. Thanks!
This isn't really a formal test I'm talking about since the bootstrapping process doesn't actually do such a test. However you can do a t-test to see if the distribution of returns is likely to be significantly negative. The correct way of doing this, to account for weirdness of returns, is to bootstrap the estimated mean of returns over the history length of the backtest. GAT
So just to be clear, you reached the same conclusion yourself (that adding breakout offered no statistically significant advantage), although stuck with it on the basis that there may be some diversification benefit?
Yes. It's worth quoting myself from the original post: "Ideally I'd now run the breakout rules together with the existing chapter 15 rules and see what the result is. However I already know from the correlation matrix and the account curves above that the answer is going to be a small improvement in performance from the ewmac+carry version, though probably not a significant one (I'm also in the middle of optimising pysystemtrade which runs far too slowly right now, to make such comparisons quicker). Notice the difference in approach here. Traditionally a researcher would jump straight to the final backtest having come up with a rule to see if it works (and I confess, I've done that plenty of times in the past). They might then experiment to see which variation of breakout does the best. This is a shortcut on the road to over fitting. Here I'm not even that interested in the final backtest. I know the rule behaves as I'd expect, and that's what's important. I'd be surprised if it didn't make money in the past; given it's designed to pick up on trends, and judging by it's correlation with ewmac does so very well. I have no idea whether there is a variation of breakout out of my set that is the best, or even if there is another combination of smooth and lookback that does even better." GAT