Todays trades Code: code contractid filled_datetime filledtrade filledprice 2765 AUSSTIR 201606 2015-02-13 08:07:32 1 97.86 2766 SP500 201503 2015-02-13 14:18:10 1 2087.50 Slippage in GBP, for entire trade code gbpt_slippage_process gbpt_slippage_bidask gbpt_slippage_execution gbpt_slippage_all_trading gbpt_slippage_total 2766 SP500 -16.36 4.09 0.00 4.09 -12.27 2765 AUSSTIR NaN 12.19 -24.37 -12.19 NaN The reason there are is no slippage for processing is because I'm rolling into the next contract, for which I don't yet have a sample price (until I've completely rolled out I calculate my signals from the old contract). Might be a good time to talk to you about rolling. This is the kind of report I get 3 times a day. Code: code msg near_expiry_days position price_contract relative_volume rollstatus suggest 25 V2X Roll positions ongoing 4 -1 201503 0.974434 ALLOW CONTINUE 28 VIX Roll positions ongoing 4 -1 201503 0.218656 ALLOW CONTINUE 40 GAS_US Roll positions ongoing 11 -1 201504 0.282547 ALLOW CONTINUE 41 AUSSTIR Roll positions ongoing 16 1 201603 0.672121 ALLOW CONTINUE 42 EDOLLAR Roll positions ongoing 3 2 201806 1.008589 ALLOW CONTINUE 20 AEX Roll near on price, not liquid. 6 1 201502 0.003839 NOT START ROLL...NOTLIQUID 21 CAC Roll near on price, not liquid. 6 2 201502 0.011452 NOT START ROLL...NOTLIQUID msg: Current rolling status. Roll positions ongoing means I will do closing trades in the near contract, and opening trades in the far one (a natural one). If that doesn't work out and I still have positions when the time comes I will do a forced roll near_expiry_days: Number of days to when I'd ideally get out. The ideal contract depends on the market. So for example for STIR to avoid the very low volatility at the front end I like to stay about 3.5 years out for Eurodollar and about a year for Aussie STIR. price_contract relative_volume: 1.0 means the next and current contract have the same price suggest: What the report thinks I should do. For the first 5 instruments, nothing. For the last two, nothing yet because there is no liquidity yet (and there won't be until Wednesday or Thursday). PROFIT: £2295 It's quite dull reading this thread this isn't it? Automated trading is dull, if done correctly. Tens of thousands of lines of code to make sure things stay as boring as possible.
Got a question: Based on the picture you posted on your first post, how much was the beginning Account Balance? Was it 500k?
OK makes sense: Another question. You use many different instruments to trade. What is the advantage for you to trade many instruments instead of focusing on just one single instrument?
Hello globalarbtrader, first of all thanks for your reply concerning the order placement approach. After reading your post about roll trades I have some passages that I did not understand. So as far as I understood the rolling dates for you contracts are not fixed but they depend on the relative values of prices and volume for front and back month. However I did not understand clearly this passage: What do you mean exactly by saying "I like to stay about 3.5 years out for Eurodollar". Furthermore, for a correct calculation of the roll dates, information about exchange holidays should be known. Is it there a clean way you might suggest to get the exchange holiday calendar. I know that they are available on the exchange website, but that would require scrapping the values. Thank you again for your posts, they are really helpful for a beginner like me.
Diversification. The back-tested Sharpe Ratio on a single instrument probably averages about 0.5. On a system of 45 instruments, if they are properly diversified, the back-test comes out closer to 2.0. "I like to stay about 3.5 years out for Eurodollar". What this means right now I'd like to be trading the September 2018 contract, i.e. the 14th quarterly contract in the future (ignoring the monthly serials in the first year). In a years time I'd like to be trading September 2019. The nearer contracts don't have enough volatility. The later ones don't have enough liquidity. IB provides a contact object which includes the expiry date of the contract. The caveat is that in some contracts, like bond futures, you actually want to be out by the first notice date which is a couple of weeks before. I use this expiry calendar https://www.interactivebrokers.com/calendar/2015/ I don't see why I need the full exchange holiday calendar to calculate roll dates - can you explain why you think this is required?
I thought that the holiday calendar should be needed in order to be able to know how many "active trading days" are left before the expiration date or first notice date.
I guess I would need that if the decision to go from 'passively rolling' (trade contracts such that we're increasing in the new contract, and reducing in the old one) to 'force rolling' was automatic. But it's not, it's manual. The decision is quite complicated, market specific, and involves a lot of knowledge that the system doesn't have, so it's not worth coding it up for the few minutes a month it would save me.
Trades from the last couple of days: Code: Trades take 1 code contractid filled_datetime filledtrade filledprice 2770 GAS_US 201504 2015-02-16 16:47:23 1 2.886 2771 AUSSTIR 201606 2015-02-17 02:37:15 1 97.85 2774 BUND 201503 2015-02-17 08:41:21 1 159.16 2772 V2X 201503 2015-02-17 08:37:06 1 23.90 2773 V2X 201504 2015-02-17 08:37:06 -1 23.25 code gbpt_slippage_process gbpt_slippage_bidask gbpt_slippage_execution gbpt_slippage_all_trading gbpt_slippage_total 2770 GAS_US 111.25 6.54 -6.54 0 111.25 2774 BUND -66.64 3.70 7.4 11.11 -55.53 2772 V2X 53.68 1.85 0.0 1.85 55.53 2771 AUSSTIR NaN 6.08 0.0 6.08 NaN 2773 V2X NaN -1.85 3.7 1.85 NaN You can see that I did a spread trade to roll V2X, which I managed to do at a good level. Yesterdays LOSS: £192 Todays LOSS £3,511
I've been away for a few days, but naturally my system has continued to trade. I'm confident enough in the checks and balances I have in place to do this, even without email access. If I'm going away for more than a week then I might have to do some work in advance, for example to avoid missing a roll. If my risk looks quite high I might even prevent the system from opening new positions whilst I am away. First P&L. Three good days. I should go on holiday more often. 18th Feb: £6,214 19th Feb: £5,592 20th Feb: £816 My expected risk is £6,277 per day, so none of these are large days. One more day like Wednesday or Thursday and I will be up to my high water mark. Here are my trades: Code: 2783 AEX 201503 2015-02-18 10:42:32 1 466.65 2784 AEX 201503 2015-02-18 10:47:19 1 466.75 2785 AEX 201502 2015-02-18 10:51:47 -1 466.35 2777 ASX 201503 2015-02-18 02:31:48 1 5857.00 2778 CAC 201502 2015-02-18 08:01:38 -2 4780.00 2779 CAC 201503 2015-02-18 08:01:38 2 4781.00 2786 LEANHOG 201506 2015-02-18 14:34:07 -1 78.05 2780 VIX 201503 2015-02-18 10:04:53 1 18.20 2782 VIX 201504 2015-02-18 12:55:34 -1 18.80 2787 AUSSTIR 201606 2015-02-19 02:00:30 1 97.91 2788 AUSSTIR 201606 2015-02-20 05:17:01 1 97.91 2789 CAC 201503 2015-02-20 11:30:30 1 4825.00 You can see there are quite a few roll trades there. And the slippage: Code: 2778 CAC -547.95 3.70 -14.81 -11.11 -559.06 2786 LEANHOG -19.51 13.01 -0.00 13.01 -6.50 2780 VIX 0.00 16.26 0.00 16.26 16.26 2784 AEX NaN 7.40 0.00 7.40 NaN 2785 AEX 77.75 3.70 -0.00 3.70 81.45 2777 ASX -259.69 6.33 430.70 437.04 177.35 2779 CAC NaN -3.70 14.81 11.11 NaN 2782 VIX NaN 16.26 -32.52 -16.26 NaN 2783 AEX NaN 7.40 -14.81 -7.40 NaN 2787 AUSSTIR NaN 6.12 0 6.12 NaN 2789 CAC -5.54 1.85 -3.69 -1.85 -7.39 2788 AUSSTIR NaN 6.11 -12.21 -6.11 NaN Sticking out like a sore thumb is the ASX trade, which merits further investigation. A sample mid price of 5843 was taken at 02:30:30, UK time. 70 seconds later after the signal was processed and when we went to the market to trade the offer was 5823, and the bid was 5822. We joined the bid at 5822, but a few seconds later the price updated to 5856 - 5857. The algo immediately cut its losses and took the offer, with the fill coming in 8 seconds after we submitted the trade at 02:31:48. Looking closely at the bars the price at the time of the order of 5822 - 5823 was incorrect, and at least a couple of hours old. So the fill was a good one, and the slippage much smaller than the figure above suggests. It's worth having a discussion about how one should spot bad prices, and what one should do with them. The basic ways to spot bad prices (apart from obvious SNAFU's like zeros or Nan's) are a large change from a previous price, a large bid-offer suggesting an illiquid market, or delays on timestamps. Its easy to exclude stale prices and to set a maximum bid-offer at which you'd accept a price. Its a bit harder to know where to set the threshold for a large change. After all we don't want to be prompted every time the price moves a 'normal' amount. Even if you set the trigger where it goes off once a month, given I trade 45 markets, that implies I'd get about two prompts every trading day, which is too many. The 30 odd tick movement above wasn't big enough to trigger any concern, and no action was taken. What should we do with a bad price? One option is to live with it, as I did here. If you have enough robustness in your system then that isn't a problem. If you're trading slowly enough then the impact of the last tick on your signal should be minimal. In simple terms if you're using say a 50 day moving average then a bad price will only affect 2%, as long as the price isn't way off, say by a factor of 10. Of course if you're high frequency trading then a 30 tick error over 8 seconds could make a huge difference. We can warn on bad prices. This might make sense if a price moves more than expected, but not enough to justify exclusion. The price can then be checked at the end of the day, and if necessary changed post hoc. Note that this means that simulations will be slightly biased with forward looking information. We can exclude bad prices. If a price exceeds my threshold, I don't add it to the database, but my system emails me. Code: 2015-02-20 20:14:18 CRITICAL: dailyprocess : Sample WHEAT 201612 failed spike check price move from 584.750000 to 565.500000 was -10.71 vols (vol of 1.798195) *RERUN WITH MANUAL ENTRY* (resolveprice) Most of these warnings like this one are on illiquid contracts which I'm not trading yet, that might not see trades for several days. I then have to rerun the price collection process manually, and confirm I'm happy with those prices. To give you some insight into what an automated system trader does with their time (apart from spend more time on holiday) a little project I have been working on over the last few days involves refactoring the storage of prices. The simple and dirty software I use doesn't optimise for things like using a small number of large files, or a large number of small files. The answer depends on how many processes you are running, wether they are in serial, whether they are likely to be locking files to read, and so on. I came to the conclusion that a large number of small files was better in this use case. This is good old fashioned computer engineering. Another thing I've done is to change the way my system diagnostic outputs are produced. This has created a smaller disk footprint allowing the system to be easily run on my less powerful backup machine, and sped up the process. Essentially I'll be using the backup machine to generate additional diagnostic information. This means enforcing backup and replicate, and . All this means it will be easier to switch between the live and backup machines, again something I've neglected. More engineering. It's also an opportunity to think about how diagnosis is done. I created quite a lot of over complicated diagnostic tools that I don't really use, which as a result have become deprecated as I've changed the system. The essential problem is to summarise a relatively complicated picture in such a way that issues can easily be spotted, and then to have drilldown tools allowing you to see where things went wrong. As part of this effort I want to create a suite of functions to analyse various aspects of live performance, with a deadline of April 1st when I'd like to be able to report on my first year of trading. Rather than the big over complicated generic solution, I'm going to try and build a large number of functions to answer some specific questions, but which will sit on some shared classes to get data out of the system. This then leads into simulation. Again my simulation tools are rusty from lack of use. I've now got a better understanding of what I want from them. There are three main areas of interest once I get the tools working again. The first is to be able to use intra-day data better. One is to adapt trading systems to the cost of trading markets (at the moment I tailor my system to the most expensive markets). I didn't want to do this until I had collected enough data on trading costs. The other is to revisit some promising mean reversion systems that I removed from the system as I didn't feel I had enough evidence that they worked. This is quite a modest research program, as I have many other things I am doing with my life, and endlessly polishing my trading system isn't something that interests me, nor do I think it is of much value.