Hi AITrader, thanks, that's a very interesting question. Actually, the DD phases are those where most of the technical improvements and and conceptual advances are made. Everyone is obviously able to close profitable trades, but the real question is how you deal with DDs, how do you hedge the unfavorable moves, and how do you recover. In this case the DD was extremely exacerbated (we nearly touched our "limit" of 50%) by some main factors, which ordinarily should not be replicated: 1. Most crucially, the use of RegT margin method with ETFs, which imho should be absolutely avoided because ridiculously strict (we are talking of about 3-4 times less available resources to trade) 2. Insufficient use of protective options. 3. Excessive packet size (which was introduced by error in one of my many game revisions) and it just happened while ERY was experiencing a unusually large move (106%), made even worse by the correlation with TNA and TZA, which with similarly large moves, acted as big amplifiers of the problem. The experience is anyway undoubtedly useful because allowed me to make a number of important technical refinements to make the application totally able to deal with the situation in full automatism, without the instrument now being "blocked" at all by the order rejections due to margin violations. For instance, previously I had an optional security mechanism which blocked the instrument after a number of rejections. Or I had another security measure consisting in blocking the order submission while asynchronously waiting for response messages from the broker. All these measures have undergone important modifications to make them perfectly fit to a situation of extreme drawdown and the necessity to deal with continued order rejections and also continuous broker liquidations. Also, I could perfect the mechanism of automatic order reduction on rejection. These are the kind of things that simply you cannot see or refine until you actually patiently experience them, and can actually see how the flow of events develops within the trading engine. It's a patient process of tuning and refinements that can only be made live, while actually trading and obviously cannot ever be even imagined in a simulation environment. From the conceptual point of view, I think that the reader ready to understand what I am proposing has really seen the importance of the player superposition approach and why storing the past trading information is crucial in order to being able to "unwind" losses. If you look carefully in what we have been doing in the "recovery phase", we have just been "undoing the losses" by using the still open players, or employing the open players to resize the sell players. So, we are not using some concept like predictive "signals" or other similar nonsense, but simply "reflecting on ourselves", on our past actions (which remain "stored"), makings action to build an "internal consistency". So there is no attempt to use "illusory" signals hidden in tickdata (I don't expect personally tickdata to provide any "signal" useful to the trading purposes, but I regard those concepts as "illusions" or self-delusions created by humans through a process of backward adaptation of the strategy to the past data), but we simply look at the order cloud and operate to make it "internally consistent", which means in the long term to realize the buy / sell average "crossover". After all, we have gone down to the "hell" of nearly 50% DD and made most back in a nearly 100% recover in few weeks. How would you trust someone who tells you he steadily makes 50% per year and his max DD is 10% (I receive tons of emails like that) ? And what happens if by chance, when abandoning his dream and simulation world and coming to real-world trading he sees 15-20% DD ? Would he be able to deal with the situation? Does he understand the reasons of the DD ? Does he have the technological and methodological equipment and a real clue about how to to recover, or will he proceed blindly trusting the same illusory "signals" which by the game of mere chance (along with the steady trading expenses) have put him in a losing situation? [There is another conceptually crucial realization which I suddenly made just yesterday morning, just when waking up and moved to immediately to put it into coding while still half immersed in a dreamlike world, about the use of past information, as finally realized how to use some other past trading information I have actually been "throwing away" (this has been for long hunting me, and somehow boiling in my subconscious mind), but this is a rather deeper question and will probably explain it in a next post.]
Not much happening so far in the first 3 days of the week. ERY still around 18. At times I have reactivated TNA to allow some little activity.
For the first time today the two ERY layers returned to a positive PNL. (The instrument however is going down too slowly (still around 17) for our attempted "bet" on puts expiring tomorrow to work.) We have also over 510K "trapped" in "unclosed" buy players on ERY alone, (some of which might be "recovered" in a far future with possible upward moves of the instrument). TNA and TZA are at "sleep" for the moment and we will "recover" them later, when most of the ERY sell players are closed, and we will need to patiently wait for 2 other large price "waves" to complete, because, having accumulated too many losses, it's now wiser to start scalping the instruments when there is a large move. They also "trap" a large amount of players (against the "drift") which could not be closed due to the margin issue (about 600K for TNA, 500K for TZA ). Clearly, even if we have been "blocked" by relative shortage of resources, the overall "logic" of what we do remains the same, the only inconvenient in this case is a large and annoying dilation of time, and patience becomes the key (well, patience is always crucial, in any case).
End of the week. ERY started gapping down to about 16, so bringing more profits for the 2 ERY layers, which currently are positive 26.6K and 8.5K respectively. Actually, the larger layer touched the nice peak of 37.5K today. Unfortunately, the move did not continue and we leave about 22.4K on the table of time decay of the put options (the strike was 15). We missed "by little", anyway the delay caused by the previous move to around 19 has allowed us to close a few buy players which have in part financed the "bet".) TNA and TZA remain still there with their losses and a lot of players "trapped", and we will address them later, when finished with ERY. Today I let scalp just a bit TNA, but then it turned non-shortable, and then I allowed TZA to scalp a bit. Then I "flattened" them again.
End of the week. It started with almost no moves, and then suddenly we had some mild indication of move upward just before the holiday. Today, the day after the holiday (Thanksgiving) ERY opened with a monster gap (probably one of the largest in few years) and shooting up to 22. So we are back to DD, not even having the possibility to make a trade the way up. In hindsight, the monster gap DD could have been avoided by flattening the position before the holiday (we have already seen an example of that). Since we already had "recovered" the instrument probably it would have been a good idea. So, food for thought for next time. These kind of events show the usefulness of using the protective structures made options (which in this case we shamefully did not have) too as hedging instrument. Defending with player superposition can be very effective, but it does not address the entirety of the hedging problem (simply because in some circumstance it is not even possible to place orders, and also because, as noted, it does not cover completely the psychological side). Another teaching we can infer is how easy it is to be thrown out from the mkt when trading with relatively small capital or resources, with not even a chance to place a stop or make a defensive trade. Another reason to carefully plan the option structures for protection and trade conservatively. (Well, at least we can't complain this session is uneventful or too largely benefiting from sheer luck... )
This monster gap represents a major setback. The most annoying aspect is that it happened just after we had recovered ERY (we have gone straight from almost +50K back to -160K on a single layer of ERY, without the possibility to make a single hedging trade the way up). It has been undoubtedly a fatal tactical mistake not to close the large position during the holiday. In fact, having recovered the instrument, it would have been the most "logic" choice not to take chances with the possibility of a gap of such magnitude. (It would have been sufficient to simply "flatten" the position overnight or to place a "protective structure" with options.) Please, do not think I have lost patience with this account, as I have an infinite reserve of that but especially I am realizing that all the mishaps with the RegT margin method are also affecting the discretionary trading decisions, in the sense that in the attempt to recover the losses due to liquidation and "trapped" buy players, I have started introducing some element of pure "betting", like the (lost) bet on the puts 15 (which nevertheless did not prevent us from a full recovery of the instrument), which do not really belong to the systematic approach we have been starting with, and may be actually strongly misleading. It is possible to recover again ERY (although it would take significant time due to the massive liquidation at high price and especially the relative scarcity of resources to trade the way down, along the drift), as we did before. However, I feel that I have been swaying too much from the original methodological intents, and this is not much "instructive", especially on a public thread. At this point, I prefer to quit here this test, accepting the current negative outcome, and move on to a fresh start on a new account with Porfolio margin (remember that you will need to maintain a minimum of 110K on the main account to enable it) where we can trade more meaningfully: https://www.interactivebrokers.com/en/?f=margin&p=pmar (As a side note, I'd say that it would be nice that IB allowed people to try the folio margin method in the paper trading account, independent of the real account size.) So, this run was not economically profitable (at least to this point), however it has been very useful to tune several less explored and extreme mechanisms of the trading engine and to acquire some more experience and stronger awareness on the need to proceed in a systematic and focused way, which we may use in the next run, along with some (new) ideas and mechanisms I wish to put at work. I will post (probably tomorrow or in a couple days) the link to the new test, which we will carry on this beautiful site, and, this time, using the Portfolio margin method. I hope you had some fun so far, and, apart the methodological ideas I introduced, at least appreciated that I have been showing you the truth and the real-world difficulties of trading and massively automating the trade management - even when it meant errors, losses and DD - without "selective reporting" or trying to deceive the reader (I believe integrity to be a necessary prerequisite for experiences to be useful and propel evolution). (Anyway, if possible, I will also let this instance run without too much intervention, and maybe, later on also take a look at what happened to it.)
dear Fullautotrading, I believe that you have taken the right decision. There is no point in continuing an experiment after one realizes that the environment (the lack of the suitable margin regime in the IB testing account) is biased negatively. I hope that you also contact IB to ask that they enable the missing margin regime even in the testing account. What is the point of having a testing account if one cannot experiment with all the live options? I know it is something to ask to IB. Well please let us have the link with the novel thread when you post in it. Thanks for keeping up the good and interesting job.
Hi AItrader >I hope that you also contact IB to ask that they enable the missing margin regime even in the testing account It looks like that this is not possible currently. Imho, it is just wrong not to have the possibility to change the margin method unless you first deposit and maintain > 110K in the real account. I have talked just yesterday to an IB representative. Their "argument" is that the paper trading account behavior is exactly "the same" as the real account (you could display the difference in margin, but that is not the point). Imho, this justification is not reasonable, because it should be normal to give the possibility to a potential investor to test the paper trading account for instance with 800K and the most appropriate margin method for that amount (and not to "trade" as if he had for instance 50K), without asking him to commit real funds on the real account during the test period, which could last months. Another very unreasonable aspect, imho, is not to give the possibility to fix any amount on paper trading reset, but constraining the amount as a multiple of the real deposit (this is a problem similar to above, always inelegantly "pushing" in the direction to deposit real money). Clearly, I have actually represented these issues several times (in their own interest, and in consideration of what potential investors ask me all the time [some are clearly immediately discouraged by that]), but always essentially unheard (actually I even had a rather "animated" conversation with a smart "director" in Chicago some time ago, with no constructive outcome). > Well please let us have the link with the novel thread when you post in it. Sure, I am going to create a new test thread with portfolio margin (on a different account) and post the link below. > Thanks for keeping up the good and interesting job. Thank you!
Here it is: http://www.elitetrader.com/et/index.php?threads/automated-folio-trading-role-of-information-and-hedging.288287/