Just saw your point about doing the calculation at the end of the backtest. The problem with this is, again, markets moving into the backtest halfway through., If you're using a fixed IDM then it will be too high in earlier periods with fewer markets. So if you calculate the IDM based on realised vol over the whole period then you'll get an answer that is too low. By using the final correlation matrix you get an IDM that is correct for live trading, albeit too high in the earlier part of the backtest. The most correct solution is to use a varying IDM, which you recalculate periodically using correlation matrices rather than a rolling window of vol estimation, for the reasons below. GAT
Yes, I thought of calculating the IDM on a expanding/rolling window base and have it time-varying. When new instruments are added to the system because data for them becomes available, you would have to wait for a certain time until you have enough data to calculate the IDM. But as you say, this is the same whether you use correlations or realized vol to calculate the IDM. I think one source of noise in the vol of the whole system is the high granularity of positions, i.e. you can only trade full future contracts and they have quite high impact on the system's vol. If you backtest with equity going to infinity, let's say billions of $, the granularity becomes less and less (Ignoring the fact that you could not buy 1000s of contracts without impact on the trade price). So I think, you could use fractional trade positions (this is like using infinite equity) to calculate the IDM. I will try both methods and compare the results, especially the fluctuations of the IDM over time.
I'll do a proper monthly review on 5th January, but in case you're interested here is my 2015: +24% Big winners: Gold 2% AEX 2.1% BTP 2.2% Copper 2.5% MXP 2.5% Crude 4% Platinum 4.3% Palladium 4.5% Gas 5.4% Big Losers: GBP -2.4% Leanhogs -2.5% SMI -3.3% GAT
Monthly update. Last one was December 5th Up 7.2% of capital or 29K. I got a new HWM towards the end of last month. No nice picture this month due to technical issues. Drawdown 1.7% off new HWM Actually quite a 'quiet' month in terms of p&l, with about half the gain of last month, despite all the gyrations in the world at large (for information I was up around 10K on yesterdays move). If you look at my risk report below, this reflects the fact that I'm not really exposed to bonds or equities, and the moves in commodities were less pronounced than last month. The financial futures are really struggling to show a new trend, with sideways movement the rule of the day. I know I keep banging on about it, but this is why diversification is key. Diversification across asset classes has really helped me since the equity / bond rally faded about 9 months ago. Diversification across styles has also helped somewhat, with trend following struggling somewhat carry has come back to the party (most CTA's were flat last year, while I made some money). I hope everyone makes some money in 2016. Reports: P&L Gainers: Corn 5K Crude 6.5K Wheat 4K Losers: JPY -3.5K Livecow -3K Positions Code: code contractid positions Lock WrongContract InFwdNotRoll 14 BTP 201603 2 False False False 16 COPPER 201603 -2 False False False 0 CORN 201612 -8 False False False 9 CRUDE_W 201612 -3 False False False 4 EDOLLAR 201906 2 False False False 10 EDOLLAR 201903 7 False False False 5 EUR 201603 -1 False False False 2 EUROSTX 201603 -13 False False False (Hedge) 8 GAS_US 201603 -2 False False False 13 GBP 201603 -3 False False False 11 KR10 201603 2 False False False 20 KR3 201603 8 False False False 17 LIVECOW 201610 -2 False False False 1 MXP 201603 -4 False False False 12 OAT 201603 1 False False False 3 PALLAD 201603 -1 False False False 6 PLAT 201604 -1 False False False 19 SOYBEAN 201611 -5 False False False 15 US2 201603 3 False False False 18 VIX 201602 -1 False False False 7 WHEAT 201612 -5 False False False Risk Code: code multisignal expected_annual_risk expected_annual_risk_per_contract position expected_annual_risk_rounded_pos Shorts: 28 MXP -13.1 8896 1964 -4 7857 33 PLAT -13.1 8855 7987 -1 7987 17 VIX -10.9 7367 9684 -1 9684 2 LIVECOW -11.4 7721 5584 -2 11169 35 GAS_US -18.4 12496 6504 -2 13008 32 PALLAD -26.8 18185 13786 -1 13786 0 CORN -22.8 15459 1871 -8 14965 26 GBP -23.6 15992 5063 -3 15190 30 COPPER -24.2 16394 8868 -2 17736 4 WHEAT -29.4 19941 3579 -5 17897 3 SOYBEAN -33.5 22719 4049 -5 20247 34 CRUDE_W -43.8 29646 8733 -3 26200 Longs: 36 EDOLLAR 18.1 12249 1394 9 12542 8 BTP 19.7 13315 6594 2 13188 Trades Code: code contractid filled_datetime filledtrade filledprice 7735 AUD 201512 2015-12-08 14:08:25 -1 0.719000 7672 BOBL 201603 2015-12-04 08:39:34 -2 130.670000 7675 BTP 201603 2015-12-04 08:43:09 -1 137.010000 7714 BTP 201603 2015-12-07 08:12:25 -1 136.940000 7855 BTP 201603 2015-12-16 09:21:54 -1 136.690000 7879 BTP 201603 2015-12-21 11:14:25 1 137.650000 8152 BTP 201603 2016-01-04 07:08:48 1 138.290000 7774 CAC 201601 2015-12-11 11:09:49 -1 4563.000000 7861 CAC 201601 2015-12-17 10:20:34 1 4738.500000 7885 COPPER 201603 2015-12-21 16:17:10 1 2.137500 8098 CORN 201612 2015-12-23 15:11:06 -1 390.000000 8122 CORN 201612 2015-12-29 14:30:00 -1 385.750000 8167 CORN 201612 2016-01-04 16:05:07 -1 377.250000 8089 CRUDE_W 201612 2015-12-22 14:25:30 -1 42.220000 8125 CRUDE_W 201612 2015-12-29 14:32:30 1 43.630000 7726 EDOLLAR 201906 2015-12-08 14:45:34 1 97.865000 8173 EUR 201603 2016-01-05 03:18:49 -1 1.083400 7813 EUROSTX 201512 2015-12-14 08:05:09 2 3216.000000 7816 EUROSTX 201603 2015-12-14 08:05:09 -2 3207.000000 7819 EUROSTX 201512 2015-12-14 08:09:55 1 3217.000000 7822 EUROSTX 201603 2015-12-14 08:09:55 -1 3208.000000 7825 EUROSTX 201512 2015-12-14 08:12:26 6 3211.000000 7828 EUROSTX 201603 2015-12-14 08:12:26 -6 3202.000000 7831 EUROSTX 201512 2015-12-14 08:15:08 4 3213.000000 7834 EUROSTX 201603 2015-12-14 08:15:08 -4 3204.000000 7720 GAS_US 201602 2015-12-07 13:34:22 -1 2.192000 8092 GAS_US 201602 2015-12-23 12:02:35 5 1.988000 8095 GAS_US 201603 2015-12-23 12:02:35 -5 2.064000 8113 GAS_US 201603 2015-12-28 14:52:19 1 2.234000 8116 GAS_US 201603 2015-12-29 12:24:32 1 2.340000 8149 GAS_US 201603 2015-12-31 17:24:00 1 2.396000 7723 GBP 201603 2015-12-08 08:53:45 -1 1.502900 7741 GBP 201512 2015-12-09 16:12:13 1 1.517200 7801 GBP 201603 2015-12-14 02:03:04 1 1.520000 7858 GBP 201603 2015-12-16 13:23:53 -1 1.501000 7876 GBP 201603 2015-12-18 13:04:43 -1 1.492700 8137 GBP 201603 2015-12-31 02:06:07 -1 1.482400 7696 GOLD 201602 2015-12-04 16:34:17 1 1086.800000 8164 GOLD 201602 2016-01-04 14:11:22 1 1077.500000 7708 JPY 201512 2015-12-07 03:06:23 -1 0.008110 7738 JPY 201512 2015-12-09 14:48:08 1 0.008190 7744 JPY 201512 2015-12-10 03:28:40 1 0.008224 7747 JPY 201512 2015-12-10 06:24:06 1 0.008254 7750 JPY 201603 2015-12-10 06:24:06 -1 0.008273 7804 JPY 201603 2015-12-14 02:06:37 1 0.008287 7864 JPY 201603 2015-12-17 11:21:38 -1 0.008186 7888 JPY 201603 2015-12-21 17:12:41 1 0.008280 7807 KOSPI 201603 2015-12-14 02:18:21 -1 235.850000 7873 KOSPI 201603 2015-12-18 01:12:07 1 239.950000 8005 KR10 201603 2015-12-22 01:42:35 1 126.040000 8134 KR10 201603 2015-12-30 03:10:36 1 126.280000 7705 KR3 201512 2015-12-07 02:46:39 1 109.170000 7777 KR3 201603 2015-12-14 01:04:52 8 109.390000 7780 KR3 201603 2015-12-14 01:06:57 8 109.380000 7783 KR3 201603 2015-12-14 01:10:51 8 109.390000 7786 KR3 201603 2015-12-14 01:36:56 8 109.370000 7789 KR3 201603 2015-12-14 01:39:25 8 109.380000 7792 KR3 201603 2015-12-14 01:43:28 8 109.370000 7795 KR3 201603 2015-12-14 01:47:49 8 109.370000 7798 KR3 201512 2015-12-14 01:49:05 -8 109.400000 7843 KR3 201603 2015-12-15 01:12:02 -5 109.330000 7846 KR3 201603 2015-12-15 05:56:56 -41 109.340000 8086 KR3 201603 2015-12-22 02:49:15 -1 109.560000 8170 KR3 201603 2016-01-05 03:53:01 -1 109.700000 8146 LEANHOG 201606 2015-12-31 14:32:09 1 77.800000 7711 MXP 201512 2015-12-07 03:12:24 -1 0.059870 7753 MXP 201512 2015-12-10 06:41:09 3 0.058490 7756 MXP 201603 2015-12-10 06:41:09 -3 0.058200 7771 MXP 201603 2015-12-11 03:56:55 -1 0.057800 7678 OAT 201603 2015-12-04 08:43:59 -1 150.000000 8101 OAT 201603 2015-12-28 12:13:31 -1 150.040000 8155 OAT 201603 2016-01-04 09:04:02 1 150.650000 7699 PLAT 201601 2015-12-04 16:40:16 1 883.400000 7837 PLAT 201601 2015-12-14 12:14:13 -1 843.200000 7870 PLAT 201601 2015-12-17 13:47:31 1 857.500000 8119 PLAT 201601 2015-12-29 13:20:32 1 891.800000 8140 PLAT 201601 2015-12-31 13:52:59 1 880.500000 8143 PLAT 201604 2015-12-31 13:52:59 -1 882.500000 7684 SOYBEAN 201611 2015-12-04 12:18:15 1 912.750000 7732 SOYBEAN 201611 2015-12-08 13:42:23 -1 899.000000 7768 SOYBEAN 201611 2015-12-10 12:35:13 -1 893.500000 7840 SOYBEAN 201611 2015-12-14 12:20:57 -1 888.250000 7852 SOYBEAN 201611 2015-12-15 14:42:28 -1 888.000000 7867 SOYBEAN 201611 2015-12-17 12:13:34 -1 873.000000 7882 SOYBEAN 201611 2015-12-21 12:06:03 1 911.500000 8104 SOYBEAN 201611 2015-12-28 12:55:43 -1 885.000000 8131 SOYBEAN 201611 2015-12-29 15:52:07 -1 883.250000 8161 SOYBEAN 201611 2016-01-04 11:30:06 -1 878.000000 7687 SP500 201512 2015-12-04 14:04:18 -1 2051.250000 7690 US5 201603 2015-12-04 14:25:42 -1 118.429688 7717 V2X 201601 2015-12-07 08:31:53 -2 23.050000 7762 VIX 201601 2015-12-10 10:07:09 1 18.750000 7765 VIX 201602 2015-12-10 10:28:08 -1 19.100000 7693 WHEAT 201612 2015-12-04 15:36:35 -1 520.000000 8110 WHEAT 201612 2015-12-28 13:15:41 -1 509.750000 Expected slippage £680, actual £447
~100k GBP in a "bad year" thats not bad at all =) congratulations and keep it coming! btw: have you coded your "anti slippage" algo into something that is meant to profit, aside from diminushing your execution costs ?
I wouldn't say last year was a bad year; in terms of SR at a fraction under 1.0 it was pretty much in line with backtested expectations. Just not as insanely good as 2014. (I also note that there are plenty of people on this site for whom making a 'mere' 24% would be a pathetic year) I've got to make some serious progress on my refactoring project (the public face of which is https://github.com/robcarver17/pysystemtrade); basically to get to the point where I've decoupled my code enough so I can run multiple strategies at once. Since I only expect to spend about 1/5 of my time on this project, that could be a while. GAT
You mention on your blog (http://qoppac.blogspot.co.uk/2015/04/futures-trading-performance-year-one.html) that you are using breakout style rules for your trading. How do you get from breakout signals/events that occur when the instrument makes a new high/low to the continuous forecast needed for your framework? I tried the following: For a given look-back period, calculate the AROON oscillator (http://stockcharts.com/school/doku.php?id=chart_school:technical_indicators:aroon_oscillator) and use it directly as forecast. The AROON basically relates the days since the last high to the days since the last low in the look-back window and scales that to -100 .. 100, so I use a forecast scalar of 0.1. This seems to work and gives comparable sharpe ratios as with the EWMAC rule described in your book. I used lookback lengths of 200, 100 and 50 for a first try. The signal is hard-clipped to -10 .. 10 by design with that rule.
That sounds good. I use a slightly different approach. I take the price range (min/max) from the last N days (depending on the speed of the system) and then normalise the current price within that range. This gives me a number between -20 (current price is at the bottom of the recent historical range) and +20 (at the top). So it's perhaps not really a breakout system, but it will show extreme forecasts when a breakout occurs. Unsurprisingly it's highly correlated (80%) with EWMAC but sufficiently different to make it worth including. GAT