Feels like time for a bit of an update. Hit a HWM again on Monday, but I guess now like most people I'm a little off that, down about 0.9%. YTD will be somewhere between 33% and 38% depending on whether you use non compounded or compounded returns (I prefer the former, but for comparision the latter makes me look better). For the financial year I'm probably up like 6%. The teaching year has finished, and most of my time recently has been spent book writing. I'm 99% of the way into finishing my first draft. Right now I'm waiting on my industry experts to review what I've written so far. I also have to write the technical appendix, which for some reason I've got a bit of a writers block on, even though it's only a couple of pages. I probably ought to be proofreading, but I have also reached the point of basically being sick of looking at the bloody document every day (it's nearly 500 pages - about the same length as 'Smart Portfolios', and I know every page by heart). So I'm taking a bit of a break from writing whilst waiting for my expert feedback to come in. With my break I'm catching up on some longstanding DIY jobs, refurbishing an old bike I've picked up from somewhere, trying to get a bit more exercise, baiting crypto people on twitter, and working my way through the list of issues and enhancements on pysystemtrade. Pysystemtrade will of course never be finished, there will always be code that needs refactoring, documentation that needs doing, the ocassional bug to deal with... so it's a project I can keep going back to. When the feedback comes back I'll incorporate that and finally attack the appendix. I then will have the official first draft, hopefully before the deadline which is mid July. Once I've handed it in I will effectively take the next 6 weeks off. Then in September it will be a case of writing up code and spreadsheets to accompany the book, and doing several rounds of proofreading ping pong with my editor. At the beginning of the year I said "In Q3 I also want to implement the new strategies I will have developed for the book." Related to this, on the wall behind my desk I have a piece of paper with 'Stage 6' on it. Many moons ago, I put together a six stage plan to get pysystemtrade into production and then do various other things with it. This is the earliest reference I can find: https://www.elitetrader.com/et/threads/fully-automated-futures-trading.289589/page-216#post-5125364 from almost exactly 2 years ago: "I'm now making good process on stage 2, the goal of which is to be able to do basic auto trading. .... Stage 3 will get me to the same functionality that the old system had, whilst stage 4 will be a full bells and whistles version (at this point I'll crown it with version 1.0 and do the documentation that's really fallen behind). Stages 5 and 6 will expand the system so I can use it to test my new trading strategies (TBC)." [Stage 5 was dynamic optimisation by the way] On the list for stage 6 were things like - intra and inter market spreads - intraday mean reversion - order netting (required for multiple strategies) - conditional orders (required for intraday mean reversion) - fast daily momentum You can see here the overlap between 'new strategies from the book' and stage 6. Anyway, the list now looks like this: - intraday mean reversion - conditional orders (required for intraday mean reversion) WTF? I hear you gasp. Well basically one upshot of the book is that I've realised that: - I couldn't get fast daily momentum to work - it's not even in the book. - spread trading is too risky and capital intensive; I will get a better SR leaving the money in dynamic optimisation - intraday mean reversion works, brilliantly, but I am better running it on seperate instruments without the need for the considerable complexity of order netting. So I'm basically left with only implementing a directional intraday mean reversion system (in practice the holding period will average a few days, but it's backtested on hourly data and will run in live continously). This will run on a subset of instruments, which I will then mark as don't trade in the dynamic optimisation setup. This will hopefully be quite an easy win to more SR, but I will be doing it gradually, one instrument at a time. I'm not sure what the final balance of instrument and capital allocation looks like between the two strategies I will be running; but ideally I'd like to run at least one instrument in each asset class if I can with mean reversion. In minor news, I'll probably return to my 'adding new markets' project, since I want to get the most value out of my barchart subscription before it runs out at the end of 2023. That should keep me pretty busy for the rest of the year and perhaps a bit longer. Then I'll probably get itchy writing fingers again. I keep adding, but not deleting, to my list of ideas for books... Rob
Pity that didn't work, was that supposed to theoretically, or it's actually the opposite to what's happening in reality on such time-scales? I was hesitant to add all these stock-sector futures to the trend system because they can't be that diversifying after having other major stock indexes already., but they probably would be good candidates for the short-term mean reversion, if it's better to do on separate products. On the other hand, they can remain in the DO as well with maximum number of contracts 0 but still contribute through correlations.. Spreads and order netting sounded cool and sophisticated (therefore must no doubt make a lot of money ), I guess will read about the details in the book, can't wait
Well there has been empirical work showing it would work at least pre-cost (so you'd have to run it netting with another strategy to make profits) but I couldn't reproduce the results. Mind you, I only have hourly data, and some of this stuff can't really be tested without at least 5 minute bars. I will probably be making that decision based on costs and contract size; not sure how good they would be in that respect. Yes that's the beauty of DO Rob
Looking for sensible suggestions, https://www.elitetrader.com/et/threads/suggestions-for-futures-to-trade.367767/. Rob
A small update on my ICE-futures trading with 15-min delayed data, I started doing it in PROD with "SNAP MID" order type (only Sugar so far but adding coffee and cocoa starting next week). But I also switched to this order type all my paper trades temporarily and I'm running a report that checks if any orders were filled outside of the Bid-Ask spread, it's widest limits between the order placement and filling times. (I have to do it daily before I purge some of the collected quotes, I only keep 1 per minute), and so far there were no such fills, so this order type seems to be working as expected. When I was checking it before I realized that I was doing after purging some of the collected quotes so it seemed as some of my orders were filled outside the spread.. Btw, Rob, when you add new instruments, how do you choose weights, do you rerun a full backtest in pysystemtrade and use it's latest instrument-weights? I did it once recently, but the weights came out somewhat weird, so I adjusted them by groups and now when I'm adding a new instrument I'm simply adding it with the average weight of it's group and then bring other weights from that group down to normalize everything to 1. Is it better to just take what pysystemtrade optimization outputs, or just assign weights 'manually' somehow? My current weights:
I do an old school hand turned handcrafting in a spreadsheet for my production system (I use the automated version in my backtest FWIW). So I'd do exactly what you do when adding a new instrument (although, to be fair, I normally add them in huge batches so in practice I'd probably re-do the spreadsheet). Rob PS Where are you getting the delayed ICE data from? (It's such a pain that you can't get delayed data for non subbed markets through the IB API).
But you certainly can get it from the IB API., before subscribing to the RT ticks I make this call "ClientSocket.reqMarketDataType(3);" which tells the API to send me delayed data as well for things I don't have subscriptions., then I subscribe to RT ticks the usual way with "reqMktData" call, and then the delayed ticks start coming under the the special delayed tick IDs: 66://Delayed Bid, 67://Delayed Ask, 68://Delayed last, 75://Delayed prev day's close, 74://Delayed Volume 69://Delayed Bid Size 70://Delayed Ask Size so essentially every normal tick has a corresponding delayed tick which I'm processing the same as the non-delayed tick. Another problem is that they don't give you historical EOD data for the non-subscribed instruments as well, but for that I'm running the daily price collection job which is using tick 68 as a temporary today's close price and 75 as the true yesterday's close overriding what I got for it yesterday from tick 68. So I incrementally collect EOD prices daily. here it is in the IBs docks: https://interactivebrokers.github.io/tws-api/market_data_type.html https://interactivebrokers.github.io/tws-api/tick_types.html
I see, do you have only one level of hierarchy when you do handcrafting with futures, i.e. just these 8 groups equal weights between them and then equal weights to each instrument within every group, or you use the grouping similar from the appendix of your first book, like: Top level: RatesBonds, Equities, FX, Commodities, Volatility then for RatesBonds: Bonds, STIR BondDuration: Duration2, Duration5, Duration10, Duration20, Duration30 for Commodity: Agricultural, Metal, Energy Agricultural: Agricultural Energy: CrudeAndProducts, NaturalGas Metal: Precious, Base For Region in stocks: Emerging, Developed Then by country