Be aware that this problem is not in YM alone. Itâs in all MD-tickers when market gets volatile (see my previous post about DAX, ES, NQ). Further the MD-protocol itself is vulnerable for transport-glitches (Internet disturbance). Actually there are two issues here: a) the construction of the MarketDepth-window on the back-end server in the Account-memory-block b) The as fast as possible, and as less overhead as possible, transport to the MD-window in the TWS-client. Problems like (a) can be solved quite quickly, to which âdstâ pointed in his posts. Problems like (b) are design decisions and when needed to re-design can be costly and take time to implement. As I wrote before: the MD-transport protocol is designed to be fast without overhead (it doesnât have sequence-numbering nor checksums, etc). It uses the same transport-mechanism as for the Quote-feeds, but there lies its flaw: for the Quote-feed it doesnât matter if one price-tick-update isnât received correctly (the next update will correct it), but it is devastating for the MD-window to miss one row-update (the whole MD-window will become corrupt and will stay corrupt until it is re-initialized completely). In other words: if (b) isnât taken care of, the MD will keep giving problems to clients having a bad internet-connection from time to time. Hope it helps.
i am glad people are here 'whining' - if it weren't for this how would we all establish the good from bad brokerages - i thought that was the whole point. it may have been mentioned time and time again - but there will always be people who are new to the industry/this website and who will find this type of feedback very useful indeed without having to trawl through historical pages to find what is useful and what is not - so please 'whine' away if i get bored of reading it i will skip that post and move on.
Hoi, I may be misinterpreting what you are describing in your point (B) but I just want to clarify. As far as the transport from our backend server to the TWS client, we use the TCP protocol. TCP uses checksums, and guarantees reliable sequenced packet delivery. The TWS cannot miss an update or get a corrupt price packet. So it is not at all vulnerable at to internet glitches.
Samaritan, You are correct about TCP in as far as you go. But if you have repeated losses because of corrupted packets two things will happen: - you will definitely get delays in delivery - you may time out because at a certain point most apps give up So if Hoi comes back in the affirmative you need to look at what you do when the lower layers of your Comms stack are not responding in a timely manner or if their counters run over. You might check it anyway.
My only point was to clarify that the the market depth window will not go out of sync because of missed or out of sequence packets. Yes, if your connection is bad you will have delayed market data. If the delays are bad enough or the packets are being delivered corrupt for long periods then TCP itself will disconnect. But TCP itself will not deliver a corrupt or out of sequence packet to the application.
Maybe Hoi is talking about UDP protocal ??? It doesnt have the overhead compared to TCP - faster with less than half the overhd in bytes of TCP. Just server to client(TWS) - nothing else.
Well, donât bet your life on TCP/IPâs reliability. Sure itâs much better than UDP (which was used by TWS before 2002, if I remember well), but this TCP communication-layer doesnât ensure by itself that corrupted data is not received by the application-layer. This because it uses only a 16 bit CRC checksum, which is quite unreliable as it cannot detect all possible errors. A simple swap of two 16 bit values gives the very same checksum, on corrupted data. See for instance these two sentences, both give the same checksum (43087): âhoi is my nameâ âhoisi my nameâ Further I have read an empirical study about the question on how many times such an 16 bit error can occur on the Internet, without detection by the checksum. They found 350 cases in 30 million messages, which sounds a small chance. But this is actually one per 85,000. Still small? Not really, as with 3 Market-depth windows with 3 updates per second, this will give you 10,800 messages per hour. So hypothetically once every 8 hours a TWS-client can encounter a Market-depth corruption, not detected by the TCP-layer. Luckily the Trading connection (for Orders, and Executions), uses the FIX-protocol-layer on top of the TCP-layer. The FIX protocol has its own sequence-checks and uses a much stronger 32-bit checksum. Actually TWS uses a ZIP-compression for the FIX-messages as well (this gives 3 checksums in total: 16 bit TCP, 32-bit ZIP and 32-bit FIX). I donât ask IB to implement the same reliability (and overhead) for MarketDepth as for the Trading-information. But something should be done in the application-layer to ensure reliability and recovery of corrupted MarketDepth windows. I suggested in an earlier post to implement a MD-monitor in TWS and on the Server, to check if the same results occur as empirical study above. And if so it would deserve a redesign. Hope it helps Hoi
Kiwi's point, is that in the event of a TCP connection reset, is the mechanism for resynchronizing the TWS notion of the state of the order book with the server's notion of the state of the order book reliable ?
In the event of a disconnect, the market data is requested again and the backend server sends the full book again. So there is no problem with synchronization.
The problem is showing itself again .... <img src=http://www.elitetrader.com/vb/attachment.php?s=&postid=2577663> The ES is populated 10 levels deep but not showing on ES BookTrader.