I'm just starting to get into intraday stuff so possible newb question.. US equities, using realtime feed from polygon.io Is it normal that trades can be out of order by 10 or even 30 seconds, even if I filter the trades to a single stock and single exchange? For example, today these two trades comes through the feed, in this order: AAPL,1528401553906,2018-06-07,15:59:13:906,193.5000,100,81,0,0,0,0 AAPL,1528401546835,2018-06-07,15:59:06:835,193.5200,100,81,0,0,0,0 The second trade is timestamped more than 7 seconds BEFORE the first one. Both trades are from exchange 81 = NASDAQ. Even though it's way out of order, the second trade is not flagged with any of the special condition codes relating to being "out of sequence", or anything that could plausibly explain it. It's not a one off either.. 7 seconds is quite a large one but there are tons of much smaller lags in the data. about 20% of the data has a nonzero lag w.r.t the previous max timestamp from the same exchange. Can someone enlighten me as to what's going on here?
umm no... it's a generic question about realtime tick feeds. Is what I'm seeing here expected or isn't it.
Thanks for the reply rb7, but can you elaborate on what is normal? What kind of ordering guarantees would be expected? I don't want this to be about polygon I just mentioned it for completeness. I'd also like to know what I'm talking about if this is a bug that needs to be reported to them.
'Normal' is what you would expect, i.e., all trades in sequence of time. Trades are generally broadcast by exchanges when they are executed. The timestamps that you are showing in your AAPL examples, are they the official exchange timestamp?
None. It is not unsual to get trades out of sequence, especially near market open, market close, or after hours. However, a 7 seconds difference while the market is still open is a bit much. Not all trades reported by an exchange actually take place at the exchange. For example, if a broker connected to the exchange internally performs a transaction, it will report that transaction through the exchange. The broker may report that transaction late, or the broker/exchange clocks may be different. Ideally the Trade Condition flag should reflect the out of sequence condition, but it is not always reliable. If doubt that polygon is at fault. They probably just pass on what the exchanges feed to them.
Thanks for that DaveV. So out of sequence trades which are not flagged as such are expected then. It seems to contradict what rb7 said, so who is correct? DaveV sounds like he knows what he's talking about, but rb7 feel free to prove otherwise. DaveV you said that 7 seconds difference is "a bit much" but what does this mean really? Differences of this magnitude are quite common in the data, happening in hundreds of symbols per day. Is this plausible based on what you know, or does it suggest that the feed is buggy somehow? If this really is normal I'm not sure how to handle it. Without flags that allow filtering the out of sequence trades, historical data can't be played back in the same way as it arrives in real time, so backtesting becomes super inaccurate and possibly meaningless.. some more stats: for AAPL on that day, NASDAQ exchange only, market hours only, "normal" condition flags only: 20% of all the trades show a difference of at least 1ms (between the latest seen timestamp including the trade's own, and that trade) 10% >20ms 5% > 180ms 2.5% > 900ms 1% > 2.6 sec 106 trades > 5 sec maximum 7 sec looking at all symbols with the same filtering, 1593 symbols have a maximum > 1 sec 701 > 5 sec 58 > 10 sec 22 > 30 sec
I can't speak for rb7's credentials, but as for mine, I coded part of the Nasdaq trading platform. However, my internal knowledge of how the exchanges work is 15 years old. Things may have changed since then. It is plausible. I subscribe to NxCore for streaming data, and I remember once reading a post from the guy who coded NxCore stating that one of his most difficult tasks was trying to figure out if a trade was out of sequence.
All Trades that go on 'the tape' are coming from the exchanges, not from brokers. Brokers need to report all trades that they do internally and the exchanges then broadcast them, by using their timestamps, not the timestamp of the brokers. Seeing that many 'out-of-sequence' trade is abnormal. The 'Late-Trade' trades can explain the out-of-sequence, but they are normally flagged as such, and there aren't that many per day.