Thanks for the generosity with your time and thoughts as always. To add some context to my motivations, I run an automated high/mid frequency equities business focused on the entire US listed stock/etf universe. It's over 10k transactions/day, so I'm wary of adding any additional data management headaches/chores. As you'd expect, capacity is a problem so I'm hoping to incorporate the AFTS trend following ideas as part of a longer term portfolio with stocks/bonds/RE. I only share that to frame some of my views on ETFs vs futures data handling. To some of your points: "Well in fact to trade ETFs properly you would have about as much pain as you have to keep track of dividends unless you stuck to accumulated only, but if you are only going to use indicies then I suppose that's not a problem, although the logistics of shorting and borrowing are MUCH worse." There's no doubt that I'm biased based on already having systems in place for ETFs. I actually didn't mind the stitching together of the futures contract series and back-adjusting as it's essentially a one time programming project, but the amount of data quality issues I ran into for mature contracts using a mature data provider (IQFeed) surprised me. Also, the lack of symbol mapping between broker/data provider/exchange seemed medieval. Lastly, and this is strictly personal, I had no confidence that my lack of experience with some of the contracts in a large universe wouldn't result in my doing something immensely stupid and potentially catastrophic (miss a roll, take delivery, bungle negative prices, or some other unknown thing). --Incidentally, I do have historical borrow availability and rates (at IBKR) back to mid-2013 for US stocks and ETFs...I'd be more than happy to share the files if the need ever arises while you're writing book #5 or conducting other research. "not sure I follow here, ETFs have management costs, and except for a few cases the risk adjusted trading cost is usually higher than in the futures. You would also probably pay more to get leverage than in futures. I do a full comparison in book #3 LT. Lower costs is a poor reason to" I agree about the expense of portfolio margin (although there are ways to mitigate it), but my findings were a bit different on the other costs. Free commissions and executing in the closing auction make execution costs near-zero (still minor slippage possible in the auction if size is enough to tip the imbalance). Also, I would think management costs are largely a wash in long/short strategies. The exception would be in assets where you're primarily long (e.g. equities), but those are typically where expense ratios are de minimis. Anyhow, I'm not trying to make a case or be argumentative; just hoping to add to the discussion. I have two of your books but LT isn't one of them - I'll get a copy and see what you've written on the topic. "EDIT: I just saw your later post; OK if you want to replicate some index that can be replicated with a perfect regression on some ETFs, then do that regression! Don't do it implicitally by plunging into an optimisation, that's batshit crazy and your eignevalues will be bonkers." Ha! We're getting to the point where I should be typing less and thinking more...but I wanted to clarify something if you don't mind. So I start with my fantasy universe of 1000 independent indices that has a trend following SR >5. Each night, I calculate the optimal exposures for the indices the next day. Now I want to replicate those exposures using some subset of ETFs. Are you suggesting that I regress the ETFs against that night's optimal set of exposures in the indices? Or that I have a replicating basket for each of the indices in my fantasy universe (based on regressions) and then run them through the optimization process? I am aiming not to drop the optimization process because I ultimately need to make trades in the real world and it neatly deals with various constraints like expenses, borrow availability, etc.. Again, I appreciate the willingness to indulge me on this. I get that it's likely garbage in = garbage out, but I have a nagging feeling that I'll learn something useful.
I think this is the key point. I'm biased in favour of futures, and have had the bad experience of trying to shoehorn ETFs (and individual stocks) into a legacy system designed originally for futures. I'd say given your experience then that it is perfectly reasonable to trade ETFs instead of futures, so let's move on to the more academically interesting point. I think the crux is you are doing optimisation for diferent reason and a multistep optimisation might make sense. Eg first decide on your optimal ETF positions, sans constraints and costs (that might not even require an 'optimisation' per se). Then run a second optimisation to work out the best trades to do, taking into account constraints and costs. You probably already have one of these ready to go. Of course then you will end up with a slightly suboptimal portfolio than if you did it in one go. But it will be a lot more robust. Ha! I have fantasies like that as well. Good luck with getting that SR... The latter approach again would probably be more robust (better to do 1000 regressions with a relatively small number of factors on the RHS in the latter case, than to have 1000 indices on the RHS of your regression using the first approach) and has the advantage that you can very closely target the right technique. For example you can use loess or pruning to reduce the number of ETFs required to target a given index, and ensure you don't have multicolinearity because some of your ETFs are too similar. Once you have your ETF exposures implied for each index, then you can just add them up. There is no optimisation required for this first stage (regressions don't count). (again bias; I've had bad experiences with doing kitchen sink one step optimisations) Rob
All good stuff and a sensible way forward. Thanks for all the input! Should this fun house mirror approach turn up anything interesting, I'll share the results/code.
Anyone on here have KOSPI futures in their system? (I do.) It looks like the CFTC has revoked their certification (https://www.cftc.gov/IndustryOversight/IndustryFilings/ForeignOrganizationProducts) and Americans are blocked from trading it (generally, there's an exception to close out positions, but last time something like this happened I had to call IB to get a position closed because I couldn't do it through TWS). Fortunately, I have no open positions in these and I can easily disable trading of an instrument in my system (but leaving it there in case it becomes available again in the future).
I finally tried the 70\30 volatility estimation in my system (only took me a couple years ) These are 'Australian Dollar Futures' and 'US 10y', orange is the new way of estimating with 30% of average vol over 10y +70% of the recent vol, the blue line is 100% of the recent vol as before. So this looks quite nice to me - the new way clearly produces less extreme estimations., incidentally it also improved my backtest performance by ~8%, which I should probably not even look at, but it does make it easier to incorporate a new change.. Btw, the Skew rule is actually reducing my backtest performance slightly, even when I give it just 30% of the carry weight. I'll probably add it anyway..
I have an open position in the KOSPI mini contract, which is also on that CFTC list. My trading account is with IB US but I'm not an American citizen. I have not received any communication from IB regarding this position or whether I'm (dis)allowed to trade it.
Last time this happened, I didn't receive any notification from IB. My system tried to trade it, but the order was repeatedly rejected. I had to stop my system and call IB to find out what was happening. Perhaps you will be allowed to trade it since you aren't an American citizen. Perhaps IB has improved their system and you'll be able to close the position through TWS and not have to call them.
I will keep an eye on it. My latest trade (it opened a position) was on Thursday Feb 15 Korean time. This might have been just prior to the notification by CFTC due to the time difference between Korea and the US.
Hello Rob I recently read your another book Leveraged Trading. It is really great book and enjoyed it a lot!! Wish I knew this book before I found out that the forecast scalars from AFTS and Leveraged Trading are different. I just know (from your book) that you calibrated it to have absolute value of 10. Could you please elaborate more on how to calculate Forecast scalar? ps. Never mind. I saw the post from your blog!!