Today the ERY has surged further (to about 18.46) causing additional drawdown. Same thing for TZA and TNA. The scalping/hedging activity has been ok, apart the problem with the RegT margin, which, taking way too much resources, is interfering a lot with the scalping activity. Anyway, functionally there are no problems, and the app deals with the situation in an adequate manner. The drawdown situations are also good to appreciate the meaning and better calibrate the sizes of the protection and reversing orders. It appears that the reversing player % should be significantly larger than the protecting player % for a twofold reason: first because the current "load" is usually much larger larger than the position of the protecting players, second because the protecting players cause an increase of the distance between the sell and buy avgs, if they remain open (on reversion). Clearly, we are paying dearly the RegT margin method, both in terms of missing scalps and in terms of missing gains on other instrument. Just to make an example, the mere flattening of UNG, to recover some margin, has already costed about 6K (missed profits): And similar considerations apply to the other instruments (GDXJ, ...) I have flattened to recover some resources. Other pretty large losses (or "missed" gains) are due to missed scalps, that is the orders that were rejected or automatically reduced due to margin limitations.
End of the week. We remain in the middle of an "investment" phase. Everything looks fine (apart the RegT margin question, which we need to live with for this run). TNA should have a larger "load" to be able to recover (this can be seen by the fact that the G-L value is currently strongly negative, about -30K). This is due to the fact that, due to RegT margin, we do not have enough resources at the moment to "load" it more (new buy player orders would be rejected), and therefore we need to postpone that to when more resources are available . (This will happen automatically, when the buy orders start being accepted again.) In the meantime, I have though about another feature which consists in increasing the order size on the reversing phase if the G-L is below the max PNL ever seen by the instrument. This should be a useful feature to better distribute the capital on "reversing phases". That is, avoiding to put too much load on instruments that do not need that much, and, instead, put more load on instruments which need it to recover the effect of hedging actions. In this sense, the G-L (gain-loss) curve proves to be very useful.
Today and yesterday the mkt continued to fall, except today for the last few hours where we had a noticeable correction. So the DD stretched even further, with a small recover at the end of the session. Our big problem in this session remains the RegT margin which is causing a lot of missed scalps and trading occasions. This, in practice, means that the hedging orders cannot be closed on reversal and therefore there is a huge amount of "missed profits" (and this is just the trading phase where the scalps would be most profitable). For instance, these are 2 detail pictures of ERY, before and after the correction. Due to the large margin requirements (usually 3-4 times the portolio margin method) most of the "hedging buys" (cyan squares) cannot be closed (message: "YOUR ORDER IS NOT ACCEPTED. IN ORDER TO OBTAIN THE DESIRED POSITION YOUR PREVIOUS DAY EQUITY WITH LOAN VALUE MUST EXCEED THE INITIAL MARGIN") when the correction happens, as they normally would, thus resulting in a huge amount of missed profits (practically, it is almost like having a "stop" in place, except that the scalping engine will try to partially "recover" them later, when the broker allows again the execution). Other "missed" profits are on the instruments which I needed to close to recover some margin, in particular all the instrument UNG, UCO and especially TBT, would have made (if let open) good profits. So there is a huge amount of "missed profits" (probably, well over 120K at this time), but it seems we have no choice than to play with these "rules" (I think one should be allowed to switch to folio margin in a test account, or else it becomes impossible to trade as one would on a real account, with the same resources). Anyway, we will see if I can still pull it out, and when back on a PNL "crest", I will get another account with portfolio margin, for more reasonable requirements. To give some possibility to ERY to close at least some players, I have momentarily flattened TNA and TZA (more missed profits). Note also how violent and sudden is the "correction" (each green line is about 1% move). This is a typical behavior of these instruments, which I have seen innumerable times (...and there are still people who think they can provide "signals" to time those events...) [ In the meantime I had a few more ideas for more precise and meaningful sizing of "reversing" players, which I will explain later. ]
ERY has continued its rally increasing (over 45% move in the last few weeks) our drawdown. Also the position has now inverted to positive (+11K shares). (Everything else is closed, because margin.) Today there were several issues with execution with most orders not receiving any "response" whatsoever from IB after cancel (this is probably a paper trading thing, as, for what I know, it is rarely seen in real money accounts) which has made the DD situation noticeably worse (as scalping was blocked). Apart the big RegT issue, I have done various (management) errors in this run, which I will summarize later, so we can, hopefully, avoid them in the future. It should be interesting to follow now how the scalping engine deals with this extreme situation and how it copes with a possible future reversion (where we will need a negative position). This would also be useful to tune the order sizing in reversing mode.
Today not much changed from yesterday, ERY still fluctuating high and we are mostly stack with hedging players which cannot be closed (apart small fraction of them). We have definitely taken too many losses (too many hedging buys which cannot be closed due to margin requirements, and therefore working in practice as ordinary "stops", and too many missed profits on other instruments, which we had to flatten) due to the RegT strict regime (and, of course, due to some sizing errors I will discuss later). This shows as the RegT margin, is not actually viable at all in practice in this type of folio application with ETFs (portfolio margin reduces drastically the requirements when dealing with ETFs and their options). Anyway, it is still interesting to follow the evolution, at least to continue improving the game rules. I have also found a technical improvement for the scalping engine under these conditions. There was in fact a problem with the rejected orders (due to margin) causing a "block" of the instrument when a response to a cancel command received no feedback from IB. This was intended to be a safety precaution to avoid that IB could later execute orders which received the status of "inactive" or "rejected", however in this case it appears evident that the block of the instrument is unpractical. As scarcely intuitive and unexpected it may be, note that even if you receive a "rejection" or order "inactive" status, the order can still be executed later by the broker. This was actually observed several times in real trading, and for this reason, I enforced the criterion that only if we receive a confirmation response to a cancel, we can consider the order "dead" and continue with further submissions. However, I noticed that this was still not good enough, because in fact sometimes orders go into "inactive" status and if a cancel is issued, IB may not respond at all. In that case, when a cancel confirmation was missing, the instrument was "paused" for safety reason (to avoid accumulation of orders which could be executed later all together by the broker, thus causing huge problems.) Now, instead, I automatically remove the instrument suspension if a a margin rejection is received but the orders in inactive status and a "cant' find" order message (error code 135) from IB was received (apparently the only reliable way to make sure that the broker will not execute the order at some later time). This is a good functional improvement, as it avoids to block the instrument and the intervention of the manager to unlock it. I have also reduced the execution max timeout to 10 seconds, and the action seems more fluid. This is particularly evident under this condition where there are a huge number of rejections and the scalping engine automatically reduces the order size, to scalp at least the tiny fraction allowed by the broker (even if a very small portion of the original order).
Hi fullautotrading, as it seems that the RegT margin regime is not allowing Gbot hedging strategy to work. Have you considered to switch to a different margin regime more suitable to Gbot? Regards, AItrader
Hi AItrader, yes, in fact I normally use "portfolio margin" on real accounts (https://www.interactivebrokers.com/en/?f=margin&p=pmar) and I am used to have that kind of resources and dynamic (much larger, for the same capital). In this case, the relatively premature impossibility to close the hedging players creates disruption in the general logic, as it blocks the scalping activity and it is nearly equivalent to placing stops and locking in the losses: just what we want to avoid altogether with the various hedging techniques. In this case, being this a demo account, it seems (unless I am missing something important) that IB does not allow to switch to portfolio margin. Sure it seems meaningless, as if one is testing with an amount like this, he should also be allowed to test with the proper margin method, before committing funds. It's true, like you say, that in practice, the RegT margin is not viable, imho. For these reasons: 1. If you have a good capital there is no reason at all not to use the portfolio margin (when trading stk and etfs) 2. If you have tiny a risk capital (say less than the 110K+ limit) you are forced in the RegT and in that case, if you have to trade by any means (which I do not suggest), you need to trade very small and carefully, possibly just 1 or 2 instruments maximum, waiting to enter after large moves against the possible instrument drift (like 45%-50%), to hope save some DD and cover with options. To escape the limitation you need to either resort to forex ("cash instrument") or to use futures. Using futures with small capital is readily suicidal with this approach (and probably with any approach, imho), because while margin requirements are "relatively" small, the single "packet" (even just 1 contract) is generally "enormous" relatively to the account size, and, in practice, will not be practically possible to sustain the drawdowns for long time (and tight stops will be even worse). Finally, for the poor trader, it remains the forex, but here again the few pairs tradable just do not seem ideal (to me) for scalping activity. That said, I will continue anyway this test for a while because still technically very useful to tune the game rules (especially the sizing rules) and refine the various trading mechanisms under "extreme" conditions (automatic size reduction, to cope with order rejections, etc.). (Later, I will switch to a new account with folio margin and restart our normal testing activity in a more meaningful way.)
End of the week. Here is a situation update. Net liquidation dropped to 514K from the original 739K (a 43%), which would not be too worrying if we had the resources to trade as intended: we would have simply "invested" in a good move with very strong potential for reversal. But, as already noted, we are instead "stuck" with buy players which could not (and cannot) be closed on reversal due to margin restrictions, and therefore they are, in practice, acting like "stop loss" (= take loss), locking in huge losses (just as stops do). Further, the icing on the cake is we have missed pretty large profits on TBT and GDXJ which were happily reversing with a pretty good short position, which we had to close. The margin method issue, does not mean that the "management" made no errors, and they have been made more evident by it. Here is a list of some errors which we need to avoid next time: - Do not use RegT with etfs [ If RegT is used for any reason, trade much more conservatively (very small sizes, few instruments, setup options, wait large moves) ] - Sizing of reversing orders was excessive (% value was too big) - Size of hedging orders was excessive (% value was too big) - Entries too tight - Max order size needs to be limited (avoid circular order size increase) - Maximum number of packets on the same side needs to be limited to a max value - Options need to be placed earlier and in much larger measure, and, above all, according to planned/expected maximum position of each instrument - Options expiration needs to be longer - Also, avoid excessive correlation (remember the saying "During market corrections all correlations tend to one."). In this case TNA, TZA and ERY are causing problems all at the same time (strong correlation), which of course worsens the margin problem (3X). [A conservative approach could be to use portfolio margin, start with protective option structures, sized according to our desired/expected maximum exposure, and activate the scalping after a 45%+ move against the drift. In that case, the risk would be much smaller, but we would also be trading much less frequently.] If you think that portfolio margin gives you some 3X-4X more leverage (and even more with options). And on top of this you add the 3X of the TNA, TZA, ERY "duplication", you see we are way off with the figures, and this makes the simple difference from a move which can be successfully exploited (as this should have been), and a move which locks us into unneeded losses, which will be difficult to recover in the future (due to the "drifting" nature of the instruments). (Another consideration is how real word scenarios can be different from sim environments, where many big factors (for instance the effect of margin method, order rejections, etc.) are often neglected, and in practice the real situations just require to look better at the bigger picture and main strategic factors, instead of wasting time in minute details which in practice may be totally secondary.) In a next post, I will discuss a bit how the application deals with this situation, and what are in detail the mechanisms (past "trading information") in place which can help to recover some of the losses we have accumulated.
PNL has been crawling back a little in the last couple days. The margin method mishap is clearly preventing us to get the "deserved" profits, due to the buy hedging players which could (and cannot) not be closed because of margin restrictions. The app provides the exact figure about the value of the "loss" created by the buy players remained open and these are respectively: ERY: about 360K, TZA about 170K, TNA about 58K. In addition, we have the "missed profits" of TBT and GDXJ whose short positions had be closed. So, a pretty devastating effect, which nevertheless is always less than what we would get with plain stops. How the app deals now with the situation is to automatically reduce the order sizes when it receives an order rejection message, so that even if it cannot close the buy players, it continuously attempts to reduce those which are still positive while the price reverses. This can be seen in the picture below where you can see some buy players which are currently lower then the original open position (bottom right). This is due to the fact that if a buy player is only partially sold, it experiences a reduction of the average, according to the formula which I have illustrated in a previous post (I think): New Avg = Avg - (FillPrice - Avg) * Fill / NewPosition where FillPrice and Fill are the price and size of the portion sold. Since (FillPrice - Avg) is always positive by design, the player avg will be reduced if the sell order is executed. Another point that differentiates this approach from "ordinary stops", is that the memory of the ("protective") buy orders is now used to size the following sell players (this is a rule that the manager can specify). This is clearly an example of use of past trading information, which is clearly not possible to do with ordinary stop loss (="take loss and forget"), as the relevant information is not stored into "players". This should make intuitively more clear why, independently of the trading approach we like to use, the "loss of past information" is conceptually inadmissible, and why it immediately resolves into "dominated" strategies.
Some more DD recover which is slowly putting at least ERY to gradually start closing some new scalps (even though the total "buy-player loss" accumulated by ERY alone is now about 398K). TZA e TNA (now flat) have missed a nice opportunity to make easy money by remaining "locked" with the buy players, as while the strategic intent was correct and also the occasion a good one, the margin restrictions have not made possible to play the game as it should have (close the buys (for TZA) or the sells (for TNA) on reversion). Now we need to rely on the application and "game rules" recovery capacity to see the development of the situation, and how much of the losses can be recovered. Clearly, the problem is that now since a significant amount is gone into "losses" (hedging orders which could not be closed) we have much less funds available to "ride" the possible favorable move, and of course the margin restrictions do not help. (In fact, once you make a 50% loss, then you need to make a 100% gain just to return where you started.) Anyway these tests are still very useful from the developer's point of view because they allow to refine the app behavior and a lot of tiny details which show only under "pressure", where actually the fund manager needs, at least, total support and reliability from his trading tool.