Hi friends, In this thread i would like to discuss a new algorithm variation. This can be considered the natural development of our discussion in <a href="http://www.elitetrader.com/vb/showthread.php?s=&threadid=202078">this thread</a>. There we discussed an algorithm which resulted essentially in a countertrending game. It was actually implemented as an overlay of 2 opposite trending systems, which generated an overall countertrending behaviour. That strategy did not perform badly most of times, and had its worst moment when instruments took long directional moves without sufficient retracements. It has anyway 2 remarkable properties which i think make it particularly suitable for large investors or for traders running separately also a directional strategy: 1. The algorithm "preserves" the original investment. In the sense that if price goes from A to B and then returns to A, then the PNL can only increase. This is a beautiful property, which usually trading strategies do not have (arguably for a good reason!). Assume for instance you BUY at price P0. If prices goes down, at a certain point, possibly after few new buys, you might decide to SELL your position. If, later, the price returns to P0, your PNL is not, clearly, the same as before, but has decreased, due to the substantial loss taken. That money represents a permanent loss for you (just like throwing it in a trashcan). Now, this cannot happen, by design, with the strategy we had been discussing, and this kind of "preservation" also caused nice profits on price reversals. The reason why it could not happen is essentially because the strategy allowed a stop at folio level, but not at trade level. 2. The algorithm has a, say, "deterministic guarantee" of yielding a profit. Let's see what this means. In a directional move, say, for instance, toward the "tip" of a price curve, the algorithm causes a grow of the position (sells for upward moves). [This growth is calibrated as much as possible by the algorithm in order to balance the drawdown with the possibility to return in profit.] As the position increases, the "relative reversal", defined as (say): Reversal / PriceWaveHeight, becomes smaller and smaller, and, through a large position, it can made smaller than any predefined amount. So, there is a sort of "mechanically guaranteed" possibility of returning profitable, in case of large investment. <b> "Weaknesses" of the strategy</b> The above 2 properties are just nice except for a notable detail ;-)) If there is a large directional move, without a sufficient reversal, the investment on a single instrument can get really large: much more than we would ever like. So, in practice, either if there are not enough funds or if the user is not backed up by a "trending" strategy, a serious problem may arise. So, in practice, in case of runaway, only large investors/funds/banks - that can afford waiting for the investment to work, while investing in new folio instances - can really take advantage of those 2 beautiful properties. Or experienced traders, capable to balance with a directional strat. <b> NEW STRATEGY variation: Countertrending + Trending strat </b> In the latest posts, unco smartly suggested that overlaying an independent trending algorithm could be a solution to the above weakness. As that approach is actually meaningful, i have decided, while adding the new strat, to allow, in my application, the selection of the original strategy (in case one wants to play it). The challenge I take here is, instead, slightly different, and is: "to create and overlay a <b>**DEPENDENT**</b> trending game to try to overcome the above "weaknesses". Being dependent, it can become part of the strategy itself, and therefore i have decided to directly overlay it, instead that playing it on a separate "clone" instrument. Unfortunately, doing so, we need to give up immediately property 1. In fact, as we overlay a trending strategy, the latter, by definition, must have the capability to "change" trading direction, and therefore will cause "embedded" losses. So property 1 is gone: the "realized" curve will be no more monotonic increasing, but will have drops where the trender "kicks in", to hedge (direction change). Property 2, *might* remain in place, although probably it can be challenging to prove it theoretically (i mean just by pen and paper). Anyway, intuition would suggest that if we overlay appropriately a trending strategy over the above countertrending strategy, we somehow might often expect a more rapid return to a profitable situation. But that depends actually on the implementation of the trender. So, in brief, i will take, as <b>"core"</b>, the countertrending strategy which we already know well and which has those sound properties, and will try to overlay a "directional game" to try protect it when there are large directional moves. The general expectation should be overall less profits (on the long run), but (hopefully) a smoother equity curve. Let's see what comes out (this is just a first attempt: i may be adjusting the trending game based on your suggestions). For what i can foresee a difficulty here is balancing the 2 games. The general idea is to let the "countertrender" work in a relatively limited range, and as a unidirectional move takes form, let the "trender" take fly. Clearly, intuitively, in the middle of the 2 games there is a "gray zone", a "transition zone" where we can expect to see the lesser performances. Depending on which one of the 2 opposite games we give more emphasis, we can get a whole range of different strategies. Personally, i would like to maintain as much as possible the countertrending game and let the trender kick in for protection purposes. Another weeakness i expect is that, while in this way we "bound" the the PNL when there are deep moves (like those currencies often do), we may still experience (large?) drawdowns when an instrument keep oscillating in a range, causing several direction changes and associated losses. So it seems important for small positions to maintain the countertrending game, and have a "transition zone" as small as possible (and, possibly, "ever changing"). In the following i will denote by <b>CT</b> the original countertrending game and by <b>CT+T</b> the modified algorithm where a trending game is overlaid upon CT. I would like to run a parallel test of CT and CT+T and try to determine the "best" T for hedging purposes. (Note that even when i talk about "trending", by no means, i imply any attempt to "predict" market direction, but simply a "mechanical" device to counterbalance and reduce the exposition of the underlying CT strategy. As probably all of you already know from previous discussion, i never use "prediction" and "indicators", in the sense of "probable indication of a future occurrence"). Tom
For a preliminary test, i have started a folio of 10 instruments: YM FUT 201012 ECBOT 5 Mini Sized Dow Jones Industrial Average $5 AUD FUT 201012 GLOBEX 100000 AUD.USD (6A) GBP FUT 201012 GLOBEX 62500 GBP.USD (6B) CHF FUT 201012 GLOBEX 125000 CHF.USD (6S) EUR FUT 201012 GLOBEX 125000 EUR.USD (6E) CL FUT 201012 NYMEX 1000 Light Sweet Crude Oil GC FUT 201012 NYMEX 100 GOLD SI FUT 201012 NYMEX 5000 NYMEX Silver Index RB FUT 201012 NYMEX 42000 NYMEX RBOB Gas Index HG FUT 201012 NYMEX 25000 NYMEX Copper Index 23 hours trading. Nothing "special" has happened. Just countertrending. Realized about 5K (picture below is CT, clearly). I am waiting for some instrument to begin a move, to start comparing to the version with the trending component and check the behavior. (Most lucky performers were SI, HG and CL fo far.) Tom
Hi No big news so far (unless profit is news). 2.38 days elapsed everything pretty choppy . Practically, only the CT component "worked" so far. Maximum position for the whole folio of 10, was <b>just 11 contracts</b>. Realized and PNL touched <b>14K</b>. Waiting for some move, to take a look at the T (trending) component. [I have the impression that commodities (specially energy) lately yield good profit with the CT game. The eminis don't seem much good for that.] As to the <b>trending</b> game to overlay, i have a few different ideas under test . The common idea is to let the CT works as long as the position is small (say =1,2) then let the T component "kick in". The various questions which arise (and that may lead to different implementations of T) are for instance: what is the optimal dynamic sizing for the T component ? What is the optimal <b>stop</b> rule ? What is the optimal take profit ? Do we actually need a take profit rule at instrument level ? And a tricky one: can we devise a sort of <b>"surpass game"</b> between CT and T so that the "transition zone" will not be, say, static but can move with the price curve ? Tom
Ok. Something happened. SI has made a quick move upward, quickly going from 22.945 to 24.935, pushing the CT position to -6 contracts. In this case, the CT, as expected by design is "investing" in the move, causing a sharp drawdown. <img src="http://www.datatime.eu/public/gbot/sun10_10_CT.png"/> CT + T has activated the Trending component, which has caused a reversal of the PNL curve in profit, and, eventually, a take profit closing all positions: <img src="http://www.datatime.eu/public/gbot/sun10_10_CTAndT.png"/> This gives a preliminary feeling of what we are aiming to, that is to use the T component to hedge the CT. Here, I must say i probably made the T to kick in <b>too late and by far too violently</b> (I exagerated in the reverse position!). I need to smooth the mechanism, but for now i just needed to preliminarily check the strategy <b>overlay infrastructure</b>. So now i (we) can focus on calibrating the Trending component for best results. As to the trending algo, at the moment i can see 2 main options. But please, do suggest if you have more ideas. - OPTION 1. When the T kicks in, the CT get "suspended" and we wait for stop or break out. - OPTION 2. <b>CT continues independently</b> even when T kicks in. The purpose of T is merely to mitigate the drawdown. Hopefully we can profit from bot CT and T. T is stopped at any reversal. <img src="http://www.datatime.eu/public/gbot/TrendingAlgoOptions.png"/> Must say that, personally, option 1 seems to me a little too naive, and my current feeling is that it should probably not be pursued (but i might be wrong). Also because, i can intuitively foresee a weakness, consisting in potential repeated losses if an instrument should keep oscillating in a narrow range. Please feel free to say what you think of these 2 approaches and which one you prefer, or if you see other alternatives and variations. Tom
Hi Tom, I think I'd prefer option 1, as in option 2 you don't really let T take full advantage of the up move. Actually you could test 2 other options: option 3: Similar to option 1, but once T take profit, the system sells all the contracts that it would have accumulated during the upmove for CT. option 4: Similar to option 3, but instead of selling all the contracts at once, you invest gradually during the down move. (so adding to your short). Hope it makes sense.
It surely does! Thank you unco: i will backtest these alternatives and see what comes out (maybe we could try the days around may 6). Let also me explain better my initial intuitive idea. Since CT is a sound strategy which "preserves the investment" (in the sense explained in the first post) and has a "deterministic guarantee" to be profitable, it must remain at the <b>core</b> of the algo. Therefore we should have: T depends on CT, but <b>CT is **independent ** of T</b>. Anything interfering with that, will spoil CT and, its sound properties would be lost. Further, the danger of an instrument keeping oscillating in a range is real, and the consequent <b>accumulation of permanent losses</b> could actually <b>push away the break out point</b> so far that, in practice, one would experience an unrecoverable drawdown. And this possibility can't be allowed. If, instead, we let <b>"T race with CT"</b>, always <b>ensuring that T as a slightly larger position than CT</b>, we will "protect" (hedge) CT while it keeps accumulating the short positions (if price is going up). This means T will have a "dynamic" position, adjusted according to CT's state (dependence from CT). Then, as the price reverses T is closed and the full (possibly additional) profit coming from CT, as originally intended, emerges in all its force. The money is mostly intended from CT. Algo component T, taken by itself, <b>it is NOT even a profitable strategy</b>. I am pretty sure about that. With a T game alone, one, at most, would get even with the market (in the long run). Tom
Ok, we are at the end of this fruitful (thanks to you all) week. CT algo could make work the investment on SI, due to sufficient retracement, ending over 24K. CT+T (prototype) enjoyed a sharp correction and then held on it ending over 35K. (Must say that this start is beyond my expectation.) <b>CT</b> <a href="http://www.datatime.eu/public/gbot/Sun 10 Oct 2010_Port4001_Cli1_CT/GBotReport_2010-10-10_Port4001_Cli1.htm"> detail of Trades </a> <img src="http://www.datatime.eu/public/gbot/sun10_10_CT_2.png"/> [Vertical yellow lines indicate instants where a "global close" takes place; that is folio position is 0.] <b>CT+T</b> (trender "stacked" on countertrender, prototype strat) <a href="http://www.datatime.eu/public/gbot/Sun 10 Oct 2010_Port4001_Cli3_CT_T/GBotReport_2010-10-10_Port4001_Cli3.htm"> detail of Trades </a> <img src="http://www.datatime.eu/public/gbot/sun10_10_CTAndT_2.png"/> On Sunday, i plan to start a parallel test of CT and some variants of CT+T, including the one which blocks CT, waiting for a stop/break out, as suggested by unco. I hope to be able some backtests during the weekend to tune the T game. Must say, I noticed something unexpected: while a was initially thinking that T should scalp a position slightly larger than the current CT position, the live test (where i used a relatively large position for trending scalps) seems to indicate, instead, that a strong reverse is somehow more "enjoyable", as it often contributes to quickly "correct" the situation. This is something still a little "mysterious" and worth to be explored. The T seems to work to accelerate the global take profit. When this happens quickly, due to a large position of T scalps, CT+T is shut down early. So, in practice, blocking or not CT, in several cases might make no difference. What do you think ? Tom
Update. This week <b>CT</b> touched almost 25K, but is momentarily being withdrawn to 13K, mostly by metals (GC, SI). <b>CT+T</b> new variants (with trending component "overlaid") are doing ok (drawdown was small): implementation 1: 22K implementation 2: 34K (CT "blocked" by T) implementation 3: 19K I am still looking into the "best" implementation problem. For now will quit, to free machines for more strategy selection: will resume live tests on Sunday. <b>CT:</b> <img src="http://www.datatime.eu/public/gbot/sun17_10_10_CT.png"> <b>CT+T variants:</b> <img src="http://www.datatime.eu/public/gbot/sun17_10_10_CT_T_1.png"> <img src="http://www.datatime.eu/public/gbot/sun17_10_10_CT_T_2.png"> <img src="http://www.datatime.eu/public/gbot/sun17_10_10_CT_T_3.png"> One nice feature of the "trender overlay" is the frequent folio close (see the vertical yellow lines). During the day a flat situation is occurring quite often. If you are wondering about my strategy selection method, i use mostly 2 things: - intuition (strategy must make sense) - running hundred of backtests on the <b>same</b> dataset (for some given datasets) "shuffling" randomly in all possible ways strategy parameters and entries (i call this process <b>"Robustness Analysis"</b>). Second step is absolutely crucial (in my view), as a bad strategy may appear good just by chance <b>on a "single run"</b>, and vice versa. Running hundred of runs of the same strategy "introducing noise" (in both parameters and entries) allows the "truth" to emerge (this has analogies with "bootstrapping"). [It's a <b>neverending fight against overfitting</b>!] Tom
I am testing (elapsed 4.4 days, from monday) the original strat <b>CT</b> and various variations of <b>CT+T</b> (trending component overlay). It looks like the resulting strategy CT+T has an equity curve particularly sensitive on T implementation. The common performance indexes don't seem much useful for strategy selection as here there is a "qualitative" aspect (say the "shape" of the equity curve), which is not easily expressed numerically. Besides, it seems that chance and price curve have an important role on how the trending component affects the equity curve. It seems that a sharper directional correction ("inversion") is perceived better by a trader because even if the corresponding drawdown can be deeper, it usually has a shorter duration. And a shorter (in time) deeper drawdown is often more bearable than a longer smaller one. Problem is that a sequence of stops can result in large drawdown. <b>CT strategy (original)</b> <img src="http://www.datatime.eu/public/gbot/Mon_25_10_10_CT.png"> In this run CT has performed well (19K pnl and 26 realized in 4 days, max exposition 11 contracts) due to to lack of extreme directional movements. Apart SIlver, which lately showed some activity. This is the "worst" situation for CT+T which may experience several stops, due to the "needless" protection it kicks on at every "false" "trend" start. And this instead of smoothing the equity curve, make it much uglier in the short period. (I might be stopping too soon, anyway). <b>It seems to remain a fact that CT appears "unbeatable" for large investors/funds, due to the absence of stops at instrument level.</b> Especially if they also play separately trending strategies. Our attempt to add a <b>"protection" against long directional moves</b> might perhaps be better pursued with a strangle of <b>options</b>, to avoid hitting stops multiple times. Clearly, time decay will also have a cost, but at least it's a <b>known fixed amount.</b> <b>CT+T variants (tests)</b> <table cellpadding="2"><tr> <td><img src="http://www.datatime.eu/public/gbot/Mon_25_10_10_CT_T1.png"></td><td><img src="http://www.datatime.eu/public/gbot/Mon_25_10_10_CT_T2.png"></td> </tr> <tr><td><img src="http://www.datatime.eu/public/gbot/Mon_25_10_10_CT_T3.png"></td><td><img src="http://www.datatime.eu/public/gbot/Mon_25_10_10_CT_T4.png"></td></tr><tr><td><img src="http://www.datatime.eu/public/gbot/Mon_25_10_10_CT_T5.png"></td><td><img src="http://www.datatime.eu/public/gbot/Mon_25_10_10_CT_T6.png"></td></tr></table> The various tests show a range of outcomes. The difficulty of "tuning" CT+T is mainly in putting in place a mechanism T which, while providing directional protection, anyway does not "hurt" much CT. There are aspects which <b>cannot be controlled</b> though, like for instance how many times stops will be hit. Clearly, CT is still the best strategy one can use when market does not show deep violent movements as it does not "waste" even a penny in stops at instrument level and any investment is turned into plain profit, on retracement (when this occurs before liquidation). There seems to be some factors which are important: 1. When the component T should kick in, that is the "protection level" 2. The intensity of the inversion, that is how strongly T intervenes 3. Stop loss and take profit distance It seems that good calibration of these aspects is very important, as either if T is too late or mild or too quick and strong we can have either insufficient or excessive "correction", and both can hurt performances. I am going to test other mixture recipes, to find the best balance. A conceptual problem remains though, that the losses due to stops can add up. And this, frankly, is something i do NOT like at all so far. Anyway, will continue testing variations. What do you think ? Tom
Been playing with your CT + T approach. T is a dangerous component... can make you either a millionaire or put you in the poor house in a flash. Every time you engage the trending hedge to the CT game you are essentially making big wagers. You look like a hero as long as your luck holds out. I understand why you want to scale large but one unlucky streak of stops may put a nail in your coffin. Much easier to hedge longer time frame vertical range than short time frame position scaling.