Update time. Performance wise, nothing to write home about eithier way. For the calendar year I'm up 4.4%, so a smidgen on my last update. For the financial year I'm up 6.3%. For context, gere's some extracted bullet points from my last couple of updates: Right now I'm working on adding lots of markets, as per the recent blog post.... I'm continuing to add markets but clearly this is going to take a while. The other thing I'm doing, also flagged up in the post, is to substitute several of my existing markets for the relevant micro or mini contract. I decided to add the micro/mini instruments as additional markets rather than replacing existing ones. I'm going to look into my idea of modifying the system to make better use of limited capital. I have a bunch of other changes and potential trading signals I want to look at putting into the core model. A lot of this is stuff I've blogged about over the last couple of years but not implemented... . I added some 'fast' cross sectional momentum (I already had slow cross sectional mean reversion within asset classes, so this was just a case of reversing the sign and shortening the window). I added the skew and kurtosis rules. This also means I can take out the 'short vol bias' rule. A 'feature' that isn't strictly a new signal is the idea of attenuating momentum and carry style signals. The next step will be to 'build' the new system as a traditional model. So that means generating forecast scalars, and then fitting forecast weights. I have a method for fitting forecast weights that I will probably blog about. It involves fitting on each instrument individually, then clustering and fitting clusters; finally fitting across the whole set of instruments. The final forecast weights will be a blend of the weights from each of the three methods, the blend depending on the amount of data an instrument or cluster. Then it's down to calculate the FDM, and fit instrument weights (something I'll have to do regularly as I add new instruments). Once the system is working in the traditional model, it's a case of adding the additional optimisation stage that accounts for discrete positions. As part of the optimisation I will be including my idea for dynamically targeting risk according to average forecast strength (this is why an IDM is no longer needed). At this stage I also want to include my new blended vol estimate that incorporates mean reverting vol. Another thing I will have is a 'don't trade' list of instruments that is updated automatically according to my criteria for adding new markets (in practice we can trade, but it won't allow new positions to be opened); in particular volume and cost constraints (the 'too large to trade' will be dealt with by the opimisation). I will make some simplifications. I can remove the hacky forecast mapping that I use right now for larger contract size. And the exogenous risk overlay will also be redundant. Finally buffering will no longer be required, since the optimisation will include a cost penalty. Some completely new systems: fast mean reversion, intermarket RV eg spreads, intramarket RV spreads (calendar spreads). The next book... New markets As of now there are 77 markets with price data in my database, with another four for which I've downloaded data but have not yet set up, plus one that I'm currently in the middle of downloading data for. One unforseen consequence of doing this is that my system process timing got out of whack, because it was taking much longer to get all the prices, which meant my other processes weren't running until after midnight (when I start trading again - I was operating with a 'closed period' between 8pm and midnight to do all my daily stuff, and that wasn't long enough). As a quick fix I've moved the start time for daily price collection earlier (to 3pm, which means I won't be getting closing prices for that day), but I have a better fix in mind which involves only collecting prices for currently trading instruments on a daily basis, and getting a full update at weekends. Replacement markets After some dilly dallying (see this thread for discussion), I replace the following markets with their micro or mini relations: - CRUDE_W - GAS_US - GOLD - KOSPI - NASDAQ - SP500 I'm not 100% sure about the KOSPI mini, since I've had a couple of bad fills when trading it which make it look expensive, but for now I'm going to monitor the situation. "New" system So today I implemented my 'new' system. I say, 'new' but it's very similar to the old strategy. Here are all the changes I've made, most of which are discussed above: Adding new trading rules as detailed above. I now have 41 trading rule variations in the system, covering twelve different trading rules. Removing the 'short bias' rule for vol (effectively replaced by skew) Adding attenuation for momentum and carry signals Replacing the vol estimate with one that uses both recent and a long run historical average Removed thresholding and the exogenous risk overlay Forecast weights are fitted using an average of instrument specific, asset class pooled, and globally pooled data. All instruments with data are included in the system, but only ~30 will have non zero instrument weights. This means that cross sectional signals within asset classes will be more accurate. Instruments are selected from the possible universe using the static optimisation technique I found to be the best in my series of posts on this topic. Notice that there are quite a few ideas from above that got binned, basically because I decided not to implement the complex dynamic optimisation methodology for dealing with small capital constraints. I've tried to avoid making the system more complex; sure I've added a few things but also removed things, so overall it's not that different from before. Instrument selection I've got a more rigorous approach to instrument selection with some better diagnostics. First of all there are the instruments I don't consider at all or calculate forecasts for. These are the large contracts for which I currently trade small contracts, listed above. Including them in the system would result in double counting for cross sectional forecasts and messed up correlation matrices. I continue to sample prices however, in case I decide to revert back to the larger contracts for some reason. There are also some instruments I can't or don't want to trade (but which I can calculate forecasts for, so they will enter into cross sectional calculations). Right now this includes US real estate (rule 871), and the European sector futures (which I'm reserving for a future cross sectional system). That brings us to instruments which I could trade, but which are too expensive or illiquid. I've added some new functionality to help here. Firstly, I've added a new method which samples bid/ask prices every hour for all instruments which are currently open. There are a few possible uses for this data, eg monitoring liquidity throughout the day and deciding when it's best to trade, but it's current use is in a new costs report. This report compares the median bid/ask spread from 3 sources: The spread captured just before an order is created The actual spread paid for an order (twice the mid versus fill price) The spread from the daily sampling process I take the most conservative of these, and this is then compared to the bid/ask spread in the configuration file, that I use in backtesting, and which thus determines the speed I trade a given instrument at. So if it's massively out of line, I need to consider changing that configuration value. There is clearly some judgement here; for example if I've only done a couple of trades with especially high slippage I might have had bad luck and be better off using the sampled bid/ask, at least for now. (I can also use 1 vs 2 to evaluate the success of my execution algo but that's for another day) Once I'm happy with the configured spread, the next part of the costs report shows me the current calculation of costs in SR terms per trade. I use this to exclude any instruments with a cost of greater than 0.01 SR units per trade. The other new report I have is a liquidity report, that shows liquidity sorted by contracts per day (<100 is a problem for me), and daily trades as units of annualised risk (I'm unhappy with less than $1.5 million). So now I've got a list of instruments that I will exclude from consideration (but I will still calculate forecasts for): Can't trade Won't trade Too expensive to trade Too illiquid to trade I now run the selection methodology, after which I will have a much longer set of instruments I'm not trading. Finally I generate instrument weights in the normal way. I will also do one last check to make sure that I can currently take at least one contract in every instrument at maximum forecast. Here's the new list of instrument weights (zeros are for excluded markets). I've added a * to new instruments which I haven't been trading, but for which I've now got a positive instrument weight. Code: AEX: 0.0 AUD: 0.015453375106911324 BBCOMM: 0.0 BITCOIN: 0.022023075020260716 BOBL: 0.022316032042555146 BONO: 0.0 BRE: 0.0194706222681167 * BTP: 0.0 BUND: 0.0 CAC: 0.03631698944675614 CAD: 0.014391724118600534 * CHF: 0.0 COPPER: 0.0 CORN: 0.0 CRUDE_W_mini: 0.040003011151262376 EDOLLAR: 0.05045629150461343 EU-AUTO: 0.0 EU-BASIC: 0.0 EU-DIV30: 0.06683805046890984 * EU-FOOD: 0.0 EU-HEALTH: 0.0 EU-OIL: 0.0 EU-RETAIL: 0.0 EU-TECH: 0.0 EU-TRAVEL: 0.0 EU-UTILS: 0.0 EUR: 0.0 EUROSTX: 0.03631697826517361 * FEEDCOW: 0.0 GASOILINE: 0.040003011151262376 * GAS_US_mini: 0.0 GBP: 0.013266567131044219 * GOLD_micro: 0.01821250887431691 HEATOIL: 0.0 JPY: 0.03066560714009915 KOSPI_mini: 0.024131621339922613 KR10: 0.0 KR3: 0.052541406749157 LEANHOG: 0.034485508922167 LIVECOW: 0.034485508922167 MILK: 0.0 MXP: 0.0194706222681167 NASDAQ_micro: 0.04099135200960326 NOK: 0.0 NZD: 0.015453375106911324 * OAT: 0.02231603171026421 OATIES: 0.0 PALLAD: 0.0 PLAT: 0.0 REDWHEAT: 0.0 RICE: 0.0 RUR: 0.0 SEK: 0.0 SHATZ: 0.0 SILVER: 0.01821250887431691 * SMI: 0.0 SOYBEAN: 0.0 SOYMEAL: 0.0 SOYOIL: 0.0 SP500_micro: 0.04099135200960326 US-REALESTATE: 0.0 US10: 0.02707276334871268 US2: 0.0 US20: 0.0 US5: 0.02707276334871268 USIRS10: 0.0 USIRS5: 0.09227765188468574 * V2X: 0.0 VIX: 0.0735667522716838 WHEAT: 0.05106532368159405 Some of these instrument weights are surprising; they are certainly a little different from what 'manual' handcrafting would have come up with. In particular there is the inclusion of both 5 year bonds and 5 year swaps. For now I decided to let them ride, since I'm planning to do RV models with swaps and I wanted to get some trading history for them. One question is how often I go through this exercise. For example, what if an instrument I'm currently trading suddenly becomes too expensive or illiquid? I'm currently leaning towards doing this annually, with the obvious time being at fiscal year end when I evaluate my performance anyway. By year end I'll certainly have a few more instruments I can trade with prices, and there could be instruments I want to trade with spread models that I don't want to also trade here (more later). Some thoughts about implementing system changes There are two dangers with system changes. The first is that you'll screw something up. The second is that they cost money in trading costs. The first defense against screwing up is having good diagnostics and staring at them a lot. So I checked and rechecked every calculated position, and in most cases they're not that different from what I currently have on. I'm reasonably confident I haven't screwed something up. However there are still steps you can take; rather than just flipping a switch and updating the configuration files, and saying 'let's go!': Turning off the automatic trading (what I call the 'stack handler') Adding especially tight position or trade limits for new instruments Manually controlling the trades especially in new instruments As for trading costs, well you can try and implement the system over a number of days, you can look to tactically replace instruments when they get to zero position... there are a number of possibilities. I'm not trading big size so none of these things will change my expected trading costs (I only trade one contract at a time regardless), only the risk that I will implement these changes at exactly the wrong time and suffer regret. But that isn't a risk I'm bothered about, so in the end I (very carefully!) did the changeover in one go, over the last hour or so. Apart from occasional changes to instruments traded, I have made no significant changes to my system since 2014 (the implementation of pysystemtrade was designed to be a like for like replacement). I don't believe in making regular changes to a system, except where there is good reason (and changes in instrument costs and liquidity come into this category); why put yourself through unneccessary stress and pay extra trading costs? I also feel I've rather flogged to death this particular part of the trading ecosystem (I've done a lot of research, and mostly concluded that it works just fine!), which means from now on I'll most probably be adding entirely new strategies rather than modifying my workhorse carry/trend model. Which brings me on to... What's next: New systems, New book So that's where I am today and good timing, as I plan to go on holiday shortly, having rashly decided to implement my changes a few days in advance - fingers crossed. I'm quite mentally exhausted, firstly with the rabbit hole of small system optimisation, then with the less cerebral but mind numbing effort to build up and test the revised trading strategy. Hopefully after a rest I'll be refreshed and ready to attack my next couple of projects: new trading systems and a new book. I've talked at length about the new trading systems, which fall into two main categories: a faster univariate system (principal tech challenge: using conditional limit orders and netting with my existing strategy), and a series of relative value systems (principal tech challenge: trading eithier baskets of instruments or calendar spreads rather than just outrights as I do now). As for the book, of the half dozen or so ideas I have permanently bubbling around my brain the lowest risk one is a book specifically about trading futures, and with much more on actual trading rules than I usually include (since cynically that's what people want! And there's only so many times you can explain what position and risk management people should be doing). I haven't yet decided what I will start on first; to an extent the book is dependent on the systems since I plan to write about my new systems in the book. However I can do them in parallel to an extent, since most of the content will relate to stuff I already know. It might make sense for me to get a book plan and deadline in place before I do anything else; it's always better to have tangible goals, plus I feel a bit 'coded out' and it would be nice to do a bit of writing in English rather than Python for a bit. Hope everyone enjoys their summer... GAT
I've always had the impression that RV spreads (calendar, ...) are an integral part to a robust system. Good to know that you are implementing it! Will it change your data timeframe? How come you haven't touched these kind of strategies before?
Very briefly (I'm on holiday, on and off, for a few more weeks) I've traded these professionally in the past (fixed income space), but they require a fair amount of implementation effort. GAT
Rob, I’ve seen mentioned a few times now in papers about absolute momentum signals, where the most recent month is excluded for a cleaner signal and/or to try to avoid a mean-reversion effect that occurs on shorter term horizons. Have you ever tested t-30 / lookback vs the standard t0 / lookback? Any opinion if there’s merit to this idea? I’m inclined to think it shouldn’t matter much but I have seen several mentions of this idea now and was curious about your thoughts.
@globalarbtrader Hi Rob, was hoping to ask a question on your carry signal from your book which is the normalised price difference between the 1st and 2nd contract. This usually works well, however there seems to be issues in cases where the 1st and 2nd contract may refer to different underlying. For instance in bond futures where the CTD is different or Nat gas where there is seasonality involved so that the difference between the price of contracts is not only reflecting carry. Was wondering how you dealt with those.
I've seen these used as signals (eg in AHL from ~2008 to 2011 for a start): they're beloved of people who do regressions to fit signals, since it is a way of creating a non orthogonal set of RHS variables. But I can't get my head around the intuition of something that does something like look at the price change from t-6 to t-1 month, but ignore the final month (plus yet another parameter to overfit!). It seems to make more sense to add a mean reversion system or signal on top of momentum. GAT
I shrug and move on.... I've discussed how relaxed I am about carry being 'wrong' already in the thread. GAT
So I'm back. Was going well then bit of a mare when stocks sold off, so now back to a ~4% drawdown. IB gateway giving me a weird message about SSL connections, hope that won't break IBC. System mostly stable whilst I was away (okay it was down for ~10 days because of internet issues, but then has been okay). Working my way through a pile of non trading related admin, overdue gardening, pull requests, emails and issues. Once all that crap is out of the way I plan to start writing the proposal for my new book (on futures trading strategies). Anything you gentlemen (and ladies? And other??) would like to see in book IV? GAT
Welcome back Rob! Hope the vacation was nice I also had these ssl-errors with IB-Gateway in the last several weeks, simple restart was usually fixing it, but yesterday it gave me some other error about my version being too old, so I decided to upgrade to the last Gateway version, which turned out to be incompatible with my old IBController, so I had to upgrade that too to "IBC" and change some jobs and settings (apparently the new Gateway version now must be restarted every day, you can only choose the time, they probably finally accumulated enough memory leaks in even that small application ). Trading-wise, on my end almost nothing happened for the last 2-3 months., a bit up, then a bit down, almost like the whole world also decided to go on vacation.. As for the new book - just like it's name - more trading strategies! , with futures (or any other things for that matter, but probably futures first, because we already have a lot of infrastructure for them).