I have compared the intraday tick volume data between two vendors. There seems to be a ~16% difference in every bar. Does anyone know why? I think it may be one vendor including something while the other vendor is not. Contract: ES Date: 6/29/06 Code: Time Vendor1 Vendor2 1000 6469 5439 1030 4570 3883 1100 3779 3150 1130 4674 3911 1200 3023 2586 1230 2370 2037 1300 2540 2074 1330 1910 1565 1400 1059 877 1430 16321 13576 1500 13220 10853 1530 7537 6266 1600 6870 5654 1615 2760 2297 Total 77102 64168
Welcome to the wonderful world of tick aggregation. You'll find this is the norm in the industry. Let us begin with the CME aggregating ticks. Then each vendor (eSig, TS, IQ, etc) does something on top of this. IB is excused as they provide snapshots. Care to identify who the vendors are in your test?
can you explain cme tick aggregation and what vendors do in more detail? there is a consistent ~16% difference for each intraday bar. The percent difference is around this number for other days as well. One of these vendors is eSignal.
As to what the CME has done, read this: http://www.cme.com/files/M6Aggregation.pdf Also do a search for "tick aggregation" on these forums, as these issues have been discussed before. Basically they clump a bunch of trades (ticks) together and send them to you as one unit to make life easier for themselves. The CME already does this at their end. Some of the data vendors add their flavour of aggregation on top of this as well. Nice. IB is a broker, not a data vendor, so they send snapshots - but I gather everyone knows this already right? In their case the data is highly aggregated. So was eSig Vendor1 or 2 in your test? Just curious...
okay, i just read about cme's new tick aggregation. the number of ticks are reduced. wouldn't this affect all vendors? why would one vendor have different data than the other? what is the vendor doing on top of the cme aggregation? edit: lol we posted a few seconds apart. if the cme is already sending aggregated data, why would a vendor mess with that? which of the two i posted is closer to the cme aggregation? does anyone know?
Yep, but this is where some vendors add their own flavour to the aggregation soup. Basically to make life easier for themselves, fewer messages to be sent, reduced storage and bandwidth requirements. Of course, this reduces the accuracy and increases the latency. Not good if you need accurate tick data (I don't trade the YM these days, but I do not believe that eCBOT peforms any aggregation - I could be wrong - in their case you should be getting accurate tick data from the exchange - unlike the CME) So, some vendors are better than others thats all. Some don't touch the feed from the exchange, others only remove outliers (filtering), others aggregate ticks, others combine a few of these techniques. It varies. If you need accurate data with a low latency feed, then it gets a little expensive.
Sure, it is explained rather well in this thread: http://www.elitetrader.com/vb/showt...age=6&highlight=ib and snapshots&pagenumber=1 the salient phrase being: "Every 300ms or so TWS gets the aggregate changes for the items you monitor. Then it passes the data on to your app. This means that you sometimes lose a high/low price since if the price jumps up and then back down you only get the final value."
if i understand you, a vendor will only further aggregate the tick volume, so the tick volume will only further decreas, not increase. This would suggest that the vendor showing the higher volume is closer to the CME tick volume?