New thread: applications and methodology discussion I am going to start a new thread with emphasis on hedge fund algorithmic (automated) management. In this thread, we explore and refine in detail hedging techniques by making a complete exploration, with particular emphasis on: - "Player superposition" - Protective "option units" We will also review and discuss systematically all the details of the scalping/hedging rules ("game"), including entries, exit and order sizing. We have initiated this journey in previous threads, and here we will be continuing this development. Constructive criticisms, conceptual contribution and new ideas are always welcome. As to destructive comments, trolling, or gratuitous discredit attempts, please keep them for yourself: we don't need them. If you feel compelled to promote your own projects within this thread, please do it properly, through a meaningful contrast of methodological approaches at a conceptual or implementation level (and avoiding insulting our work). "Real world" expectations for results To avoid the usual comments from some well known trolls (fortunately very few on this great site), coming out and interfering in the middle of an "investment phase", let me clearly say the following. We are not expecting to get rich "overnight": this is systematic process of investment requiring patience and we may well expect to stay several weeks "underwater" before our "investment" starts showing results. The equity curve will (typically) have a "cyclic" behavior, alternating load phases ("troughs") to return phases ("crests"). As the trading proceeds the load phases will be deeper and deeper, and the returns will periodically hit new higher watermarks. Each "cycle" may last several months (3 to 6 or more) and the cycle length may tend to grow over time. This will, in practice, mean that we may very well go 2-3 times "underwater" again, even if we previously reached pretty good profits, before (hopefully) remaining permanently "above water", in the next cycles. The folio composition and the hedging techniques may "smooth" a bit the cyclic behavior, depending on how much investment is absorbed by single instruments and on instruments' leverage. Clearly, an ideal world situation would be to keep the allocation rather "uniform" across instruments, but that is not always achievable in practice, as the instruments themselves undergo periods of variable volatility with occasional "blasts" of very high volatility. Account initial status To avoid psychological "interferences" I will be using an Interactive Brokers' simulated account with the following initial situation (values received via API): while the situation shown via API is the following (retrieved by the app, via API) <code>ACCOUNT INFO from IB (Accounts in login: DU163281 Selected: DU163281) ============================================================ DU163281 ============================================================= Acct update event last received from IB: Mon 21 Jul 2014 14:52:29:995 [ Mon 21 Jul 2014 08:52:29:995 edt ] - Currency: EUR Exchange rate: EUR/EUR 1.00 - Current values (received on: Mon 21 Jul 2014 14:52:29:995 [ Mon 21 Jul 2014 08:52:29:995 edt ]) - AccruedCash 0.00 EUR [Min: 0.00, Max: 0.00] AccruedDividend 0.00 EUR [Min: 0.00, Max: 0.00] BuyingPower 2,957,606.40 EUR [Min: 2,957,606.40, Max: 2,957,606.40] (4.00 x 739,675.04) FullAvailableFunds 739,675.04 EUR [Min: 739,584.78, Max: 739,962.39] FullExcessLiquidity 739,675.04 EUR [Min: 739,584.78, Max: 739,962.39] FullInitMarginReq 0.00 EUR [Min: 0.00, Max: 0.00] FullMaintMarginReq 0.00 EUR [Min: 0.00, Max: 0.00] NetLiquidation 739,675.04 EUR [Min: 739,584.78, Max: 739,962.39] - Currency: USD Exchange rate: USD/EUR 0.739675 - Current values (received on: Mon 21 Jul 2014 14:52:29:995 [ Mon 21 Jul 2014 08:52:29:995 edt ]) - AccruedCash 0.00 USD [Min: 0.00, Max: 0.00] - Initial values (received on: Mon 21 Jul 2014 13:32:12:690 [ Mon 21 Jul 2014 07:32:12:690 edt ]) - AccruedCash 0.00 EUR AccruedDividend 0.00 EUR BuyingPower 2,957,606.40 EUR FullAvailableFunds 739,584.78 EUR FullExcessLiquidity 739,584.78 EUR FullInitMarginReq 0.00 EUR FullMaintMarginReq 0.00 EUR NetLiquidation 739,584.78 EUR Current - Initial (elapsed: 1.34 hours) Î AccruedCash 0.00 EUR Î AccruedDividend 0.00 EUR Î BuyingPower 0.00 EUR Î FullAvailableFunds 90.26 EUR Î FullExcessLiquidity 90.26 EUR Î FullInitMarginReq 0.00 EUR Î FullMaintMarginReq 0.00 EUR Î NetLiquidation 90.26 EUR</code> (not sure why Î NetLiquidation keeps fluctuating in time, while no orders are executed yet, but, apparently, so it is.) and mixed with the methodological discussion, I will show the folio progress, the various fees and also show the details of all trades (which I can also provide at any time in form of an excel sheet with all details, to anyone interested). "Rules" and assumptions All orders are limit orders. Orders are made based on a set of automatic trading rules specified by the funds manager (called "scalping/hedging games"), which we will examine in detail during this thread. Fund manager intervention is possible at any degree to possibly supervise or adjust the programmed scalping/hedging game. To "simulate" some of the possible "psychological interference" that might happen in the real world, I will assume an "emotional pain threshold" of 50%, which means that if the PNL reaches -50% of the initial "risk capital", our session is over. As M. Patel says in its book "Trading with Ichimoku...": "Emotions and trading don't go together. The emotional trader will ultimately become a losing trader - even with the supposed proper techniques and knowledge" Therefore we will supervise our hedging action to stay within this constraints during the "load phases", but at the same time, and at the right time, "push" the investment to maximize the use of new scalped profits. Technological setup We will be using IB trading facility, which is a pretty good one. I will simultaneously use a real account for data comparison and my own tests and the sim account for this test. We use the Gateway set up on post 7496 4001 being used by a real acct). Note that I can receive the same data both on the real account and the sim account because both instances are running on the same machine. The trading application is my ongoing work (and obviously acts "on top" on their API layer, for executions aspects) and I will continue adding possible conceptual improvements during this thread (even if, at this point, it is quite "stable" from a technological point of view). Initial folio We will be dealing in this session with a folio of leveraged ETFs. But similar concepts will apply to futures. The initial folio is the following: <code> DGAZ STK SMART VELOCITYSHARES 3X INVERSE NA ERY STK SMART_USD DIREXION DLY ENERGY BEAR 3X FAS STK SMART_USD DIREXION DAILY FIN BULL 3X FAZ STK SMART DIREXION DAILY FINL BEAR 3X GDXJ STK SMART MARKET VECTORS JR GOLD MINER NUGT STK SMART DIREXION GOLD MINERS BULL 3X QID STK SMART PROSHARES ULTRASHORT QQQ TBT STK SMART PROSHARES ULTRASHORT 20+Y TR TNA STK SMART DIREXION DLY SM CAP BULL 3X TZA STK SMART DIREXION DLY SM CAP BEAR 3X UCO STK SMART PROSHARES ULTRA BLOOMBERG CR UNG STK SMART US NATURAL GAS FUND LP VXX STK SMART_USD IPATH S&P 500 VIX S/T FU ETN </code> NUGT, FAS are kept for the moment in "manual mode" (not traded), because they are blatant "repetitions" of other instruments. UNG is also kept manual, to take a look at it for a while before starting it. So we have initially 10 (out of 13) instruments in "auto" mode. We start relatively small, using a fraction of the capital, with packets of 0.2% of 700K, scaled according to volatility. Later (in a few months) we will gradually ramp up to full usage of capital. I will be discussing various hedging techniques, along the way, and will also show how to possibly mix the automated hedging action with various option configurations.
Most of the inquiries I receive focus on the "player" concept and the "scalping / hedging rules". Let's start from there and from practical examples, by gradually explaining the methodological reasons behind this algorithmic architecture. I will use a method which goes from examples to "theory" rather than vice versa, which is probably more natural to someone seeing this for the first time. Let's start from this picture (realtime). The cyan/red circles represent 3 entries made on the instrument called FAZ just yesterday (FAZ is a well known ETF: https://www.tradingview.com/e/?symbol=FAZ). You see now 3 circles with some numbers nearby: 2 are red and 1 is cyan. Red 71 shares @ 17.43 (sell) Cyan 61 shares @ 17.55 (buy) Red 80 shares @ 17.33 (sell) (complete info can be fetched by clicking on the players while in "player view mode": see yellow circle at the bottom of the picture.) Those are obviously 3 orders which have been made. "Red" means sell, and "cyan" means buy. The 2 numbers nearby are respectively the avg fill price and the the quantity (number of shares or of contracts). Now, in our architecture those are not "simply" (open) orders, but each of them represent the creation ("birth") of a distinct "player". You can start picturing this in your mind, like if you had an account where now 3 actually different persons or traders (the 3 "players") are trading simultaneously on the same account. And more players will be added as we proceed. So a player, intuitively speaking, is "like an individual trader". He, just like human, has some degree of independence, but it is also bound to follow some "rules" imposed by the environment and the community where he lives. What is the "purpose of life" of a single player ? Mostly he lives to fulfill these purposes: - 1) Maintain useful trading information that the application will also use to govern the life of all players (and for it's "higher purposes") - 2) Realize an "individual" profit sometimes in the future (through a unique "trade") - 3) Possibly exercise a hedging ("protecting") action towards the rest of the players, while pursuing his objectives This can also be seen pretty much a metaphor of human life. And in fact, making an analogy with Conway's "Game of Life", these players will allow, from relatively simple concepts and rules, to build and evolve unimaginably complex strategies and differentiated trading scenarios. (One difference with Game of Life, is that it has mainly a purely theoretical interest. Here we add to the "academic" aspect, also the more practical aspect of transforming the trading information into an actual edge.) So, intuitively, a player can actually be imagined as an individual trader playing on the given account and with a unique "trade" at his disposal (apart partial fills), after which the player "dies" (or it's "closed"). The player can ordinarily only be closed in profit. (In more abstract terms, it can also be imagined as a single "component" of the average cost of the current position. But we will see that later.) A player is open and closed according to "rules", which are specified by the fund manager (representing a "governing entity"). These rules are called a "scalping/hedging game". A s/h game represents the "use" we make of the "past trading information" (accumulated by the traders). I sometimes use the term "player cloud" to refer to the set of currently open players. There are several reasons for (the necessity of) this architecture. Which I will explain soon. Let's also anticipate that, in addition to the players, we also have another "type of orders" which will be called "manual orders", and their main reason for existence is the "temporary suspension" of the automatic game logic and evolution, while retaining completely all the trading information. We will see that in the next posts.
"Manual" orders We have introduced the concept of "player". We have seen that, leaving aside the philosophical and conceptual aspects, in very practical terms it works like an order which opens a position according to certain rules, and it also has the "built in" capability to preserve some information and automatically close in profit, according to some rules. Now, we need to focus on "manual" orders. What are them and why they are important ? As the name suggests, a manual order is not a player. It's simply an order which is placed by the manager (properly "assisted" by the application) and there it remains, ignored by the algorithmic engine, except, obviously, for the purposes of computing PNL, unrealized, realized, etc. Within the application, manual orders are identified by different colors. While the players take the colors cyan (buy) and red (sell), the manual orders will appear in green (buy) and pink (sell). What is the use of these "manual" orders ? Let's say, first of all, that "manual" orders are not intended as a mean for the fund manager to interfere with the algorithmic game. (For that, there is another specific tool, called "player injector", which I will explain later on). They have instead very precise and important uses which I will now explain. "Memory-full stop" concept One of the main uses of a manual order is the crucial function to "suspend" the algorithmic game by flattening the actual position, while the players remain all "active" (open). Since the "manual" orders are ignored by the algorithmic engine, flattening the position and putting the instrument in manual mode will cause the instrument to retain all the trading information. When it is necessary to resume the automated game, it is sufficient to place an equal and opposite manual order, and everything will proceed "as if" nothing happened. Clearly, there will be an extra "trade" caused by the manual buy/sell, which will show in the PNL. When is this useful ? Here is a quick list of most obvious applications - Overnight gaps Sometimes one does not want to take the risk of an overnight gap (maybe because the position is large and the mkt is highly volatile). At the bottom of this post I will show an actual example of that, applied to our current folio. - Reallocation of investment We see that some instruments are definitely "going nowhere" and we would like to move the corresponding investment on more "active" instruments. Or we are running out of funds and we want to just focus on some specific instruments. Same as before, we may place some instrument in "stand by" and use the money for other instruments. If we want, sometime in the future we can resume those stranded instruments. - Futures Rollover Here I provide a very simple assisted feature to do this in a clean manner without losing any trading information. The concept is anyway similar to the above cases. We flatten the old instrument, we move the trading information to the new instrument (this is done automatically), then we resume the position on the new instrument (in this case, obviously, we will take the possible "gap" due to contango/backwardation). - Protective structures with options and (extra positions) We will see this in next posts about hedging techniques. As special "protective structures", we may create new layers containing combinations of options and some shares (or contracts) which will remain "outside" of the game automation. In our approach, it does not exist the (destructive) concept of "stop loss" (= "stop and forget", or "take loss", or "stack up losses") which is the current "common" approach, because our treatment obviously revolves all around the preservation and use of past trading information. Let's now see a real world case of use of manual order to avoid a possible "overnight gap". I do this just as an example, and not, obviously, because it is actually needed in this case. For this purpose, I have added a specific feature which, in addition to placing the manual order, will also "remember" it, so when one needs to resume the instrument, it automatically suggests the order needed to restore the position. Assume I had this huge short position of 10 shares on DGAZ and I am terrified by the idea that some huge overnight gap might occur, to the point I cannot sleep. The following 2 pictures show this simple process. We simply click on the "moon" button and then place the "flattening" order (the instrument will go in "manual mode" with a 0 position). Tomorrow, using the same, we will resume the algorithmic game. This way, whatever happens during the night our PNL will remain unchanged, but tomorrow we can just restore the algorithmic games just as nothing happened. Clearly, this can prevent either a large loss or a large profit (with respect to remaining with the open position) should the instrument open with a huge gap. Finally, I am not suggesting that one should do that routinely. I am just indicating a possibility. A sort of "alternative" to this, which might avoid altogether any gap concern, is the use of "protective option configurations", which we will see in the next posts.
Now that the market is open we can complete our "example" by restoring the position on DGAZ. We had closed (buy) -10 shares at 4.85 and now we are selling them back at the new current bid: sell 10 @ 4.81 (then we can resume the "auto" mode.) In this case we obtain a small "loss" (clearly with respect to not flattening) consisting of 10 * 4.81 - 10 * 4.85 - comms, because the price has gone down a little bit. (Those 2 "manual" orders, in green and pink, will be ignored by the automatic s/h game, and only be considered to the purposes of all computations related to the PNL.) [ So in this case, the good sleep of our fund manager has probably costed more than a sleep pill ] In the next post, I will show some examples of option/shares "configurations" to create protective "corridors" for the scalping/hedging engine. This would provide a more general solution to the problem (of hedging and) of psychological comfort of the fund manager.
I am going to show soon how to place a couple of "protective option/shares configurations", just as examples. Before doing that let's take a look at the current folio situation. (When we place the options, clearly we will have a sharp "drop" of the PNL curve, because we are paying in advance for the "protection", but that should be slowly recovered in the next month or so.) We are currently autotrading 9 instruments, in the middle of the 3-rd day and made about 80 fills so far. (Commissions look relatively expensive, which is also a sign our trading "packets" currently are relatively too "small". But we know that.)
Let's now see one example of the "protective option configuration". I will show this process step by step in the next post, but let's first examine here, first of all, the final result of those steps, that is the whole complete "configuration", so when looking at the steps will be clear where we are going. (When one is familiar with the feature, it takes just a few seconds to create this kind of configurations.) For this first example, let's select for this example TNA (DIREXION DLY SM CAP BULL 3X) The pictures below show the situation before and after placing the "protective" structure. You see we end up with the following scheme: ------------- -1 CALL 75 (to finance the puts' decay) +100 shares on a new manual layer (to cover the call) free scalping with long only position "constraint" (which is a "default" for TNA) +4 PUTs 58 (to "protect" the s/h game) -------------- The minimum size of the structure is such that it clearly "screams" for an increase of the scalping "packet" of TNA (18 shares is way too small). So, for the moment, I am raising all packets to 0.5% of 700K (rescaled by volatility) and, in particular, setting TNA packet to 50 shares. If we sense during the next weeks that it is still too small we can rise it later (possibly closer to 100 shares). In the next post, I will show how the application automatically selects the option strikes of this kind of configuration (of course it can also be done completely "manually" if one wants to). In the meantime, you can think about the general logic, and maybe suggest more configurations. The ratio 4/1 (for puts/calls) is just one "parameter" of the option unit "auto search", which will will see soon, the manager can select the desired ratio at will (a smaller ratio would clearly reduce the "corridor" size). Later, I will also show an analogous configuration, for instruments like NUGT, TZA, which trade with opposite long term "bias" (due to ETF decay).
And, as promised, here is a quick summary on how to add the "protective" options and shares, with the help of the application. What the application is essentially doing, based on the protection ratio that you select (4:1 in our example), is to scan all available option chains and find for you the strikes such that the decay of the "defensive" (or "protective") option (the PUTs in our example) is repaid by the premium of the sold option(s) (the CALL in our example). Of course, that could be done also "manually", but the feature allows you to do that quickly and without errors (which in the case of the options can be quite costly, due to the large spread. In practice, you just indicate the "template" of the "defensive" (or "protective") options, which you can retrieve by using the option scanner, which will retrieve for you all available chains (for all classes and expirations) and strikes. In this case it found 819 different options (call or put). For instance, we take the template: TNA OPT 20140829 85 P SMART 100 (the template is just necessary to indicate the expiration and the "side" we want to protect, while the strike is immaterial.) Then, you paste that template in the template window of the option unit auto search dialog, select the "type" of protective structure you like (PUT/CALL in this case) and click on "Retrieve price of underlying". Then, click on "Search Options" and this will automatically scan and search all the strikes and corresponding bid/ask prices for the nearest "repay option" (in our example the CALL), which, being ATM, will be full of value, and, at the same time, will look for the nearest "defensive" option strike (PUT in tour example) which fulfills the condition such that: value of 4 PUTs = value of 1 CALL (the 4:1 ratio we specified). Once found, you click on "Add Options", and, finally, on the execution button to buy/sell the options. For the "cover" shares, you simply add a new manual layer, by using the context menu, make the manual order, and "lock" the layer in "forced manual" mode. This is a very simple few-step procedure, which can be done in one minute or so, but that, in the long term, can save a lot of serious worries, and provides a tremendous psychological help to the fund manager (and the investors as well). Once done, this will last until the selected expiration (end of August in this case) and, if the prices moves a lot, or on expiration, we can add more "protective" structures like this (or whatever others we can imagine). In a next example, I will show the of construction of an "inverse" structure, to protect a short biased scalping game. In case of "symmetric" game (no "bias"), 2 of those structures (or similar ideas) can, of course, be "superposed". We might actually prudentially add this kind of "protective structure" to all the instruments. Or we can wait for some hint of "troubles" (an instrument starting a "runaway"), which sometimes is already too "late". That, of course, depends also on the personality of the fund manager and, most importantly, the kind of risk that the investors are willing to take in their venture.