I think there's also a problem with mini-crude: it has a good liquidity but not in the Dec contract, all volume is concentrated in the 2 front months. I guess it's still possible to trade it but not with Dec contract.. How critical is avoiding seasonality in this case by trading only December ? We do trade GAS_US monthly even though it should also probably have seasonality.. Btw, in this file "https://github.com/robcarver17/pysystemtrade/blob/master/sysinit/futures/config/rollconfig.csv" X and V months in the roll and hold parameters are flipped for GAS, does it have any special meaning or it doesn't matter? Also, regarding mini-GAS, not sure if there's enough volume in the second contract if we use front as carry and trade second., e.g. we would probably need to roll into the Jul contract around April 20th to April 27th (when May, which was Carry for Jun expired), and Jul's volume only started reaching 100 contracts on April 26.. Sorry for too many annoying questions
And another thing MINI-KOSPI seems to be monthly, not quaterly: http://global.krx.co.kr/contents/GLB/02/0201/0201040204/GLB0201040204.jsp "The quaterly month : The settlement price of KOSPI200 futures The non-quaterly month : The current settlement price methodology is applied" I was just looking at the volume of the HMUZ months and there's a volume gap, i.e. if I tried to roll from Mar to Jun I would have to do it before Mar 11th (expiry day) and the Jun contract has almost zero volume around Mar 11, so people are clearly rolling into April and May first..
It's not critical and there is probably more value to be gained from greater granularity than from having the seasonal match. Action: change mini crude to monthly rolls. This will involve rebuilding the multiple and adjusted price series, and needs to be done before the next roll occurs. No the order is irrelevant, pretty sure they are sorted when used. Action: change mini gas carry offset to +1. This will involve rebuilding the multiple and adjusted price series, and needs to be done before the next roll occurs. This is a bit trickier. I can't just rebuild the multiple and adjusted price series, because I don't have monthly contracts for KOSPI in the past, only quarterly (and I don't have them for barchart). At the moment I'd be looking to buy June, which would be fine right now anyway. But then the forward contract in the multiple adjusted price series is shown as September. It's probably possible to hack this, but it strikes me that a lot can go wrong. What I think I will do is delete the cloned prices, see how much recent historic data I can get from IB, and effectively start again with micro KOSPI as a new contract. I'll lose some data history, but that is a lot simpler than torturing my code and possible introducing bugs. Action (for now): override positions on KOSPI-mini Action: Get prices from scratch for KOSPI-mini No problem! In fact I should really have put these changes up on here before implementation as a sense check. Stupid to waste the free resource of people looking at your stuff for you. GAT
What is IB's symbol code for this KOSPI-mini? I am using K200 as KOSPI and have daily OHLC price files since about middle of 2018.
I only know of K200 being a quarterly contract using March, June, September and December. I am not aware of any other expiry dates. IB only shows me these when I use reqContractDetails(). Edit: added a screen shot from IB's contract search tool: Only the K200M has monthly contracts:
I'm glad to help Onboarding new instruments is very annoying and it's always good to have multiple eyeballs on the details., and I'm naturally encountering these issues as I'm trying to onboard these contracts into my system, especially because I have automatic rolls I'm forced to check all these things when I generate new contracts and roll-schedules.. Yeah, I think the full KOSPI has only quaterly contracts (http://global.krx.co.kr/contents/GLB/02/0201/0201040201/GLB0201040201.jsp) so no way to use full KOSPI prices for all monthly contracts of mini-KOSPI..
Micro WTI futures are coming soon. If it's like the other micros, it should have good liquidity, but only in the front month. https://www.cmegroup.com/media-room...e_group_to_launchmicrowtifuturesonjuly12.html
Progress has been quite swift over the last couple of weeks. I'm continuing to add markets but clearly this is going to take a while. I've put all the markets for both the first and second batches into my configuration file and there is a sobering total of 227 rows; of which 40 or so were my legacy markets, and where I've added another 16 of the remaining 187 so far (including the new micro and mini markets). This is going to be a long haul! For some markets where there isn't Barchart data I created some code that will 'seed' the database with as much IB data as it can get (again thanks to @HobbyTrading for the heads up about expired contracts). I decided to add the micro/mini instruments as additional markets rather than replacing existing ones. That means things like risk and position reports in the past will still make sense, and overall just seemed a less risky approach. That also means for the time being I'm still sampling the prices of the original markets; given the number I'm adding to my system another half dozen is neither here or there. I did some profiling work and was able to get some quick wins quite easily. Broadly speaking I did this by removing use of pandas apply (on rows) using my own functions, and instead did the relevant operation on the entire dataframe in situ. A particular time sink was the calculation of the carry signal; in particular working out what the distance between two expiries is in a fraction of a year. Now I could assume the distance is always what it is specified in the roll configuration, but to be safe I'd rather not do that. I was doing this with apply and also working with the string "20060300" converting it into a datetime, and working out the exact number of days. This is crazy since it assumes the expiry is on the first of the month, so we're doing a massive approximation, so it doesn't really matter if (for example) we account for the fact that February is 28 days long. Instead I changed the contract series to floats so the operation could be done on the entire dataframe, assuming each month was exactly 1/12 of a year. There were also little things like "".join(["x","y"]) being faster than "x"+"y", introducing a new 'simple' contractDate object where both inputs are strings (since the full object is very flexible and therefore slow). Then there is this little beauty, run every time a contractDate is initialised (since I specify spread contracts as eg '20200100_20200200': Code: MISSING_STR = -1 def list_of_items_separated_by_underscores(this_str, result = ()): find_underscore = this_str.find("_") if find_underscore is MISSING_STR: result_as_list = list(result) result_as_list.append(this_str) return result_as_list partial_str = this_str[:find_underscore] result = result + tuple([partial_str]) remaining_str = this_str[find_underscore+1:] return list_of_items_separated_by_underscores(remaining_str, result=result) Which I replaced with the surprisingly quick: Code: date_str_as_list = date_str.split("_") WTF was I thinking?! Anyway, moving on, nothing to see here... In a change from the plan above, I decided to circle back to potential trading signals and changes to the core model before I went ahead with anything else. When I sat down and went through the list, there weren't actually that many things that I intended to add in terms of new signals (which wouldn't be more suited to a newer seperate strategy, such as spread trading). I added some 'fast' cross sectional momentum (I already had slow cross sectional mean reversion within asset classes, so this was just a case of reversing the sign and shortening the window). I added the skew and kurtosis rules. This also means I can take out the 'short vol bias' rule. That's basically it. A 'feature' that isn't strictly a new signal is the idea of attenuating momentum and carry style signals. The next step will be to 'build' the new system as a traditional model. So that means generating forecast scalars, and then fitting forecast weights. I have a method for fitting forecast weights that I will probably blog about. It involves fitting on each instrument individually, then clustering and fitting clusters; finally fitting across the whole set of instruments. The final forecast weights will be a blend of the weights from each of the three methods, the blend depending on the amount of data an instrument or cluster. Quite slow to do, I'll probably do it as a single in sample fit once I've tested it, and quite heuristic, but I would like to introduce some more instrument specific weightings where that is justified. I expect this will be in my next blog post. Then it's down to calculate the FDM, and fit instrument weights (something I'll have to do regularly as I add new instruments). One consequence of the new methodology, by the way, is that no IDM is needed (but I'll probably set it to the maximum 2.5 just for old times sake). Once the system is working in the traditional model, it's a case of adding the additional optimisation stage that accounts for discrete positions. As part of the optimisation I will be including my idea for dynamically targeting risk according to average forecast strength (this is why an IDM is no longer needed). At this stage I also want to include my new blended vol estimate that incorporates mean reverting vol. Another thing I will have is a 'don't trade' list of instruments that is updated automatically according to my criteria for adding new markets (in practice we can trade, but it won't allow new positions to be opened); in particular volume and cost constraints (the 'too large to trade' will be dealt with by the opimisation). I will make some simplifications. I can remove the hacky forecast mapping that I use right now for larger contract size. And the exogenous risk overlay will also be redundant. Finally buffering will no longer be required, since the optimisation will include a cost penalty. GAT