Ok, so I basically made a Sharpe 8 algo (trading IWM/SPY/QQQ), with the chief unrealistic assumption being that fills are executed at the last trade price observed. If I delay fills by 1 s after order, almost all of my alpha goes way. I can to some extent get better results by predicting what will happen until the fill to avoid adverse selection, but the algo is still pretty disappointing then. So the question is very simple: Is there a point in pursuing lower latency fills (by switching from IB to lower latency brokerage) or is filling at last trade price such a huge data snooping bias that the algo is going to be worthless either way?
Of course it is a completely false assumption. You actually fell prey to look ahead bias, you assume you know the future fill price which you set as the current, observable, fill price. Did you test how performance changes when you assume a fill at the next traded price?
A big problem is I'm also working with second stamped tick trade data without bid/ask, so there's a limit to how far I can take my simulation. Yeah, maybe it's time to buy the data I do need rather than using free tick data... True, it's definitely an unrealistic assumption, the question is how large the effect is. My hope has been that with low enough latency, there will be a closer relation between last observed trade price and the bid/ask my market order gets filled against. Using the next available tick price (with up to 1 + 1 second of precision error plus accuracy mismatch with bid/ask) for simulated fill only marginally improves the simulation results over a strict 1 s minimum delay (in reality still up to 2 s of error due to data). As far as slippage goes, I've been using average slippage for these ETFs when using the immediate fill model. When using the (at least for high latency IB fills) more realistic delayed fill model with 1 s or more order latency, the actual slippage will be much higher. Most likely this is because while I'm somewhat accurately predicting market direction, other market participants with lower latency make the same conclusions and move the price adversely before I can fill. So the crux is, while I can actually predict the direction (compared to last trade's price) of changes to the price 1-2 s in the future (and actually longer), it might not be usable information in practice due to latency. Why am I even asking when I have inferior data? I'm hoping someone else has run into a similar situation and can tell me just how far into fantasy land I am with immediate fills w/ manually added slippage. Very thankful for your comment by the way, it helps discussing these things.
Minute stamped data are not tick data and are rather worthless for the type of strategy and frequency you intend to target. Or am I understanding you incorrectly in that those are tick data (meaning, each and every single trade produces a separate print)? It would be odd to then drop millisecond precision but one never knows...
It is tick data that has second precision rather than millisecond precision. Yeah, it's pretty silly but that's the terms of the data provider to the algo development platform I'm currently using apparently. Anyway, it seems getting millisecond precision data should be step 1 before I make any other plans.
So your current data provider is not IB? You mentioned that you simulated the impact of being 1 second late in filling your order line. Have you also simulated the impact if there is zero delay but your fill price is at the ask instead of the latest trade price (if buying ETFs)? With the ETFs you mention I guess that is only a 0.01 or 0.02 USD difference as the spread on these is rather small.
Yeah, that's what I mean by "using average slippage". I deliberately targeted these ETFs due to the low spread the algo by design might seek to switch direction often. Anyway, the cost of the spreads seems small compared to the wider slippage due to adverse selection.
These two comments seem a bit in contradiction to me. At first you say that getting actual tick-data is important as time delay negatively affects your strategy. Then you say that it is not delay/spread, but your selection process which causes a reduction in performance.
W/ adverse selection I mean adversaries selecting the same trades as I do, but faster, thus leading to a worse price for me (manifesting as higher slippage). Of course, I don't actually 100% know that's the true cause of the slippage, but it's a reasonable hypothesis. If someone has a better idea why latency is killing me I'm listening.
When you get down to "seconds" granularity, everything becomes more execution dependent. I don't think the issue here is the latency though. Rather, it costs to cross the bid and ask spread and that wasn't accounted for in the backtest. Let's assume I have a trading system that buys everytime price goes down one tick and sells everytime price goes up one tick. For the last hour, price has went up and down this 2 tick range 40 times. Does that mean I could have made 20 ticks in profit? NO. Price really did go up and down 40 times. But the only way to get those backtested price is through limit order, which requires waiting of unknown time. For contracts like es, my order could be like number 400 on the queue. So the fact that price go up and down 40 times only means I am now like 20 contracts closer to execution, which is still 380 contracts away. If you don't want to wait, then issue market order but it will cost you 1 tick per order or 2 tick per round trip trade to cross the spread. I think there are studies that show that prices can be predicted for a brief few seconds. To profit from that, I think you need infrastructure similar to HFT. Think about the above example, if I can just capture that one tick gain 20 times in an hour, then I should be able to make 100% of my capital by the 17th day, thus your high Sharpe. Changing the broker by itself probably isn't going to cut it. You can, however, put it in sim just for fun.