i have been looking for info on exactly how IB filters the ticks sent to client's TWS - i have read here that only 25% of all ticks are transmitted - i am specifically interested in ES and DAX - i can't find any further details either on IB's site or here - is there any logic to how the ticks are filtered out, i.e. volume filter etc. - would appreciate any info from IB or anyone else who may know, thx.
I think I read somewhere that what you see on TWS is in effect a snapshot of the market taken every 0.6 seconds. Sorry I can't remember where I read this so I have NO idea if it's actually true.
... taken every 0 - 0.25 seconds depending on the number of symbols you are watching and the speed of the market.
fader, Below is a post on this subject from the IB api forum: <b> Generally speaking, IB only sends a message when something changes. So if a trade occurs at the same price and size as the previous trade, IB sends neither price nor size messages (but it may send a volume message). When the size has changed, but not the price, it sends only the size message. When the price has changed, but not the size, it sends both price and size messages. When both the price and the size have changed, it sends both the price and the size messages, and then sends the same size again. I've no idea why, but that seems to be the way it is. This description applies to index futures. For stocks, it seems to be altogether more bizarre, and I haven't been able to make any sense of what it sends. I should also point out that IB sends price data at regular intervals. For index futures, the interval seems to be 300 milliseconds, and for stocks it seems to be 200 milliseconds, though this may have changed since I investigated it in early 2004. At the end of each interval, it sends data for the last trade that occurred during that interval, if any. It compares the prices and sizes against the last ones sent. So suppose the last price and size notified were 1187 and 5, and volume was 22841. Then during the next interval there are two trades, one at 1187.25 for 10, and another at 1187 for 10. At the end of that interval, IB would notify only a size of 10 and a volume of 22862, because the price is still the same as the previous price notified. The fact that a trade occurred at 1187.25 doesn't get reported. Note however that the volume has increased by the amount of all the trades during the period. The great benefit of this scheme is that the total load on IB's market data servers is well-defined, no matter how busy the markets become. As a result, IB's data feed keeps pretty much exactly in step with the market at all times. Compare this with other feeds that send every price, like eSignal. At busy times, like when there's a sudden drop in the DAX or the S&P and all the other index futures follow, there's a huge surge of data and quite a noticeable lag happens as their servers struggle to pump it all out. I've noticed two occasions this week where my eSignal feed has been a good 10 seconds behind the IB feed. The disadvantage is that because the IB feed misses out some prices, your 1-minute or 5-minute bars (or whatever timeframe you use) might occasionally be slightly inaccurate in their highs and lows. Whether this is a big issue depends on your trading strategy. It hasn't been a significant problem for me. </b> What it basically all boils down to is that IB wrote their data api to send requests to their TraderWorkstation. This application only displays quotes in a grid type fashion, hence, no need to really send every tick to the application. If something changes, you just update the cell that changed. Because a lot of people were asking for a true time and sales flow, IB introduced the TotalVolume message in their api. This shows the volume of all trades that occurred since the last update, regardless of if any of the price data changed. So, technically, you don't miss any data, but you do miss the tick updates. This is what you are seeing get filtered out. Individual ticks that occur at the same price for the same volume will not get sent to you, because neither a price nor trade size update event occurred to trigger sending the tick data to you. I have run tests against IQFeed and IB's feed and for a typical day on the ES you don't miss much data(assuming you don't mind that some trades are added together). You do, however, really miss the feel of the data on a time and sales window...... - jeff
Thanks for clarifying that, like I said I can't remember where I read that and apparently it wasn't right anyway.
Further, not all ticks are sent to the API. On rare occasions, I won't get a new LOD or HOD on ES if only a few contracts trade very briefly in heavy action.
So Esignal lags on volume spikes as well? dang and here I thought it was only a QChart problem. Question: Are there any tick by tick direct feed services that DON'T lag on volume spikes?? CQG? BLOOMBERG? REUTERS?
Cdntrader, The quote from IB's forum was not my own. I found it when I was researching this exact problem a few months back. So, I cant comment directly on eSignal's performance vs. others. I have heard others say there is some lag during really heavy times, but I have never personally dealt with eSignal. As for recommendations, I really prefer DTN's IQFeed(www.iqfeed.net). I have seen no lag during heavy markets on the es, nq, or ym, or stocks. I have not watched the bonds. IB's feed is really good as well during heavy markets, but obviously it's not tick-by-tick. - jeff
IBSoft. Thanks for the action on the memory use issue .. have sent my logs. On this ticks issue: what is the maximum number of symbols that will still result in the maximum number of symbols? I am on the HK server and currently collect 1 or 2 contracts each of: HSI, SGXNK, SGXTW So thats 3-6. Would I get max ticks with 6?
folks thanks a lot for all the replies - jeff's message seems to be a more precise explanation of what IB means by a "snapshot" - could IBSoft confirm that this is accurate? - also am interested to hear an answer to kiwi_trader's question - thx.