Investigated one of the big positive execution profits: so the soybean order was placed at 11:75 EST when Bid=1134 Ask=1135, I was selling 1 contract, but the fill price was 1139, so I sold at 5 points better than the original bid (made 250$). The next price I have is from 15 seconds later, my order was filled in just 5 seconds (I delete a lot of historical RT prices every day..) and it was bid=1151\ask=1154. So the price moved in my favor very quickly and I got a better fill than even the best side of the spread. Investigating every such case is hard, but if I blanketly remove all positive execution profits > 100$ then overall the execution algo will be loosing money, but I wander that maybe in that case I should also remove some negative points, because the spread could've ran away from me in the other direction and I would've lost more than the spread even with a market order.. So I don't know, will check that statistic again in a year
I come from a HFT background, and 15 seconds is like a century to me Interested to what you guys think about the following: 1) How do you define 'slippage'? Sorry this has probably already been mentioned before. It may be 'unfair' to use midpoint, especially for fat tick sized futures e.g. if bid-ask is 100(20 lots)/101(1 lot), the 'fair' value is much closer to 101, clearly much easier to sell at 101 than buying at 100. Some people then decide to use a volume-weighted average on BBO as 'fair px'. IMO this is more accurate, but HFTs are also known to mix up their sizings to confuse people, so not perfect either. 2) How often do you think you traded against a market maker? IMO almost always. The way I see MMs they have an 'execution algo' switched on both buy and sell, and actually achieve -ve slippage overall on both sides, and by the zero-sum nature of futures, non-MMs without a short-term 'signal' will have +ve slippage. The goal of an execution algo as a non-MM is like trying to 'bargain' with the MMs for the cheapest price possible they are willing to give, but you know they are making $ out of you. 3) Do your algos have settings/behave differently for specific futures, based on ticksize/competitiveness etc? For competitive futures with very tight bid-ask spreads e.g. MES, MNQ, N225M etc, probably can't do much better than crossing the spreads imo. For less competitive stuff it's more interesting. Spreads are wide due to not enough competition to push the MM to quote tighter. Say BBO is 95/105, if you put a buy limit order at 102, the MM probably snipes you out with an IOC. However, the MM may also 'frontrun' you and changes his quote to 105/115, what do you do then? Been a long-time lurker of this thread!! Thanks GAT for all the useful information!
Hi Japan_trader, good to see someone with deeper understanding! So in general, I'm way over my head here, my execution algo is also a rip-off of the Rob's algo described here: https://qoppac.blogspot.com/2014/10/the-worlds-simplest-execution-algorithim.html . Basically I issue only limit orders, I start on the best side and will cross the spread if either: - too much time passed (we're talking tens of minutes) - the price(the side of the spread I'm currently at) moved against me - there's an unfavorable bid\ask volume imbalance (i.e. if I'm buying and bidSize > 5*AskSize or I'm selling and askSize > 5*BidSize) 15 seconds I mentioned earlier isn't the actual speed of my algo, when the system is running I'm processing every IB tick (I think for futures they aggregate them on their end and send price updates 4 times a second), so my reaction time is 250 milliseconds + 2x my internet ping. Which is still I guess snail-pace compared to HFT . The fact that I can't troubleshoot my execution post-factum more precisely is because every day I delete most of the historical RT-price records(trying to keep approximately 1 price per minute), to keep the database smaller.. So of course I'm not trying to compete with HFT, the only valid question for me is whether doing this simple algo is better than just sending market orders all the time. It seems to be better, because as I understand, with a market order I'll pay the full spread every time for sure, and with the algo I seem to do a little better on average, and that's really all I'm after here.. The logic in #1 seems reasonable, but in terms "what can I do about it" it seems that maybe when the bid-ask volume is imbalanced against me from the beginning I should just submit market order right away because trying to capture spread in that situation is unrealistic. The #3 implies that perhaps I could add more granularity to my algo for futures with wider spreads (and just use market orders for others). I.e. instead of having just 2 modes of operation (trying to capture the full spread or crossing the spread), I could adjust it more granularly., i.e. start on the best side, if in 10 minutes nothing changed and no fill - move by 1/3 of the spread and wait some more, if nothing happened again, move another 1/3., at the same time watching bid and ask volumes.. What's +ve and -ve exactly ? I tried googling it, nothing came up.. (sorry for a dumb question, but my options are really limited here, I can either continue the conversation pretending I understand something I actually don't or ask the dumb question ) Apart from that, #2 implies that because I'm trading against MM, capturing the full spread is just not going to happen, and instead I should give them a deal by submitting my orders at a slightly worse prices than bid or ask.. Makes sense I guess.. If they choose to 'frontrun' me and widen the spread, well, I'll just cross it and pay more, but how can a single MM just widen a spread? There are other MMs there which all compete with each other, so if this guy doesn't want to sell me that one lousy contract at that price, then maybe someone else will
These abbreviations are regularly used to indicate positive and negative in a mathematical or finance context. So if it relates to numbers and values and not about opinions or impressions. At least, that is my understanding of how these abbreviations are used.
Why not just store all trades into something as simple as json? That's what I do but it's not a standardized solution. I just write the positions (and some other information) that I need to keep track of into a byte encrypted json file.
Hi Kernfusion! Thanks for your detailed explanation of your execution algo. Yes +ve = positive, and 'positive slippage' means losing $ on execution. The 'frontrun' case I mentioned only applies to 'non-competitive' products with only 1-2MM, exactly for the reason you mentioned. I see this behaviour a lot in mid-cap stocks. All the futures I actively trade are very 'competitive', so I have no idea whether this behaviour will start appearing in less popular/competitive futures. A couple follow-up questions/discussion points regarding your algo: 1) how often is the condition 'too much time passed' triggered? Given it's 'tens of minutes' my guess would be pretty much never? 2) Apart from looking at the bid/ask quantity ratio, maybe it's also worth looking at the absolute quantity. 1000 vs 100 may not be the same as 10 vs 1. Generally, if a level has very small size, the 'fair px' is very close to it, hence MMs are scared to put big orders in there even when they can be 'front of the queue'. A small move will flip the level and they can easily be 'caught'. MMs fight to death for time priority, so this is quite reliable. On the other hand, seeing a big size on the bid means someone is 'comfortable' with it. I believe HFTs may do this to 'trick' people into lifting the ask, (and I think your algo will do so) even though they think the fair px is mid-point. 3) 'Crossing the spread when it moves against me' Most of the 'execution algos' I have seen have this functionality. From a hft perspective, this is more for risk management, because these sort of algos are often used to close positions/hedge out exposure (your risk manager will be calling very soon if you don't do so). I'm not sure if it makes sense for your 'slower' type of trading. I feel you should 're-evaluate' your 'signal' (I'll be honest I don't understand exactly how you guys do this), when the price move against you, whether it's still worth doing the trade after the move(applies to both entering and closing i believe). If it's still worth doing, why not apply the same bid/ask quantity ratio logic again on the new BBO? If there is no 'rush', using the same logic again should reduce slippage further. 4) Why use market orders instead of a limit order x% out? This is a very minor point, and I understand in practice it makes no difference. I'm coming more from a risk perspective, if somehow your algo malfunctions and start ordering 100x the intended size, market orders may blow through thin order books. This may sound pedantic, but I've actually seen this happen a couple times in my career. MMs do put in orders 1% wide just to catch fat fingers. 5) I like your idea for 'wide spread' futures. I think it should work quite well as long as the MM isn't trying to game your order, and even if so, given the initial spread is so wide, you may still do better than just crossing the spread straight away.
Fascinating to see someone from the 'dark side' in this discussion! 1) The markets where you could be waiting tens of minutes... or longer... for fills tend to be those with large size (much of which isn't intended for execution) and little volatility. I'm thinking of STIR markets, especially near the front of the curve (where I tend to avoid trading anyway). I could tell you about the time I was helping out on our execution desk and I whiled away 6 hours waiting for a 2 lot Euroswissy order to be passively filled before finally bailing and crossing the spread... but it was boring then, and it would be boring now 2) Is an interesting idea. It's fair to say I've never really done much with my execution algo; it works well enough, and I don't really trade often enough to conduct experiments in changing it, or seeing if some markets do better or worse than others with a given set of parameters. (One cool idea would be to set up a system whereby retail traders can pool execution data in some common format, producing a much larger dataset) 3) "I feel you should 're-evaluate' your 'signal' (I'll be honest I don't understand exactly how you guys do this), when the price move against you" This is the critical point of difference. I generate signals daily overnight, and once a required trade is generated it is not changed by any further price movement that may occur after the generation has occured. The behaviour is that of a slow hedge fund, with a portfolio management team that cogitate and decide what rebalancing is required, then throw the trades over a one way wall to the execution team which has one simple mandate: execute this order at the best price you can, irrespective of what happens next, within this time frame (by the close of the market today). Over the other side of this wall could be a sell side algo, a bank of human traders, or as we have here an in house algo (at AHL we used all three routes to market, and ran them in competition against each other). It doesn't matter. So in practice, we could be trading the next day at a price far away from the price at which the trade was generated. The backtest deliberately simulates this behaviour, assuming the price is executed at the following days close. In practice we'll be filled somewhere between the open and the close. A good thing to check is that the difference between 'signal price' and 'mid price when execution begins' is a source only of small random noise, and not of bias. If you consistently lose money through this delay then your underyling system is capturing a source of alpha that is too fast for this lazy execution strategy. With such a system it is true that the price moving away from you isn't neccessarily a source of panic (unless the market closes in 5 minutes); the logic behind this design decision is purely simplicity (my intention was literally to build the dumbest execution algo I could that would still add some value). Without it you'd need to implement some kind of utility function that balanced risk and value, and /or had some capacity to predict short term movements in price. You'd basically be grafting a short term strategy on to a longer term strategy, and breaking down the wall. Of course you could just rerun the system every few seconds (which I guess is what you are alluding to), and with enough computational power and some smart coding this would be possible. It does make things complicated in the execution stack since you need to have the ability to cancel existing unfilled orders when they are no longer required to hit the optimal position. I toyed with including this logic in my own code, and it got very complicated very quickly. Yet another way of doing this is to keep the seperation, but include more complex conditional orders. So you could have orders that said things like 'buy 50 but only if the price is 30 or better'. Or 'buy 20 NOW' for a stop loss. Or 'buy 20 but only if the price has gone above 10 but is still below 20'. You get the idea. Then your overnight run generates these more complex orders, and throws them over the wall. This is hard for a system set up like mine to achieve, as it uses contionous signals you'd need to effectively monte carlo the system to see what position it would have on at different price levels. In fact there used to be a very large CTA that used exactly this system. The problem with all of these solutions is that 99.9% of the time it doesn't affect what the daily system was going to do anyway: even though the price may have changed a fair bit since the original signal was generated we'd still want to do the same trade. And it's even less likely that a small change in price during the few seconds or minutes we've been executing would affect the trading system. However there are some systems for which the conditional order solutions would make sense. One (which I'm thinking of testing) is a relatively fast (holding period a couple of days) mean reversion system; for this 'buy if the price is 30 or better' makes perfect sense. Another (which I'm not implementing) would be an option delta hedger. 4) Another reason to use limit orders rather than market is that with IB you can't change an order type, only the price. So if you start trying to executive passively with a limit order, and you want to change it to a market order to be aggresive, you can't: instead you have to change the limit price so it's crossing the spread. GAT
This is indeed what I do with my trading software: if the spread is more than one tick I place a limit order at midpoint. If this order isn't filled after one minute waiting I modify the price to cross the spread. The posts by @Japan_trader are interesting to read. However, in my trading it is usually about adding or subtracting one contract at a time from a position. It rarely happens that a trade occurs with a quantity of more than one contract. Which makes that things such as liquidity and depth of order book much less relevant.