What technical and/or physical limitations do we have if we want to go really high frequency, i.e. (1) download real-time tick data at 1 second frequency, (2) monitor 500+ stocks simultaneously, (3) execute trades at real time (though not that high frequency) Does FIX/CTCI still have the infamous data throttling problem? Thanks!
I too would appreciate if anyone has any information on this... Tick data at one second frequency is way too slow. Can anyone confirm if this is true? Basically I.B's feed as far as I know updates once every 100 or 200 ms so a good ATS would need to be able to match this in order to perform effectively. I have experienced the pains of 1 second updates firsthand and would not pay 500 dollars for a connection which is bottlenecked this way. Prices can change 3 times or more in one second meaning that your bars end up incomplete.
I don't know if CTCI/FIX of IB can do 1sec, or not. However, if you are accessing data over the Internet, bear in mind that unless you have a dedicated line(as those funds), your Internet connection have 300-500 ms or even higher delay/jittering anyway.
I use IB FIX, but for only for order execution. Market data over FIX (not FastFIX) is an overkill, unless you trade illiquid issues. BTW, where did you get 300-500ms delay? From a modem connection?
I would of thought that any global standard for information exchange such as FIX should basically do its best to offer true real time access to quote feeds. Because this is not possible without crashing the system the feed is bottlenecked so that it only changes once every 100 ms or so at ib. This is 10 times per second. Fix as a global standard should be able to process every change meaning it should be capable of matching price update for price update. This means that it should be capable of handling price updates every 100ms rather than once per second. Otherwise you cant draw bars without missing prices. I would really like to know from someone like Def at I.B how often the fix can accept price changes. Unfortunately if you ask this question in chat they direct you to their Api chatroom which is impossible to get an answer from. I swear Ive asked questions with say Angel from Ib logged in at API chat and theyll answer a simple question and then they will simply ignore any more detailed questions point blank which is so $%%$$*& rude I cant believe it.
Any TCP/IP Internet connection is best-effort. Of course you will probably make it UDP packages which is like the quality of streaming video, but you see even Internet Video has delay and jittering, right...?
What do you mean by FIX is overkill? My design needs to monitor 500+ stock data simultaneously... FIX is an overkill?
As I previously stated, it really depends on the nature of your system and the issues you trade. Some systems a fine having snapshot based data, some systems require co-location and sub-ms latency for proper market-making & arbitrage. Have you ever seen tick-by-tick data for a stock doing 5-8M vol daily? Quotes would peak 20-30 updates/second when a few algos are in play. For very liquid stocks the update freq is insane. FIX is text-based, so both peers have to code/encode text which is very inefficient for RT data processing.
Text decod/encode is not inefficient at all. Look, when you do Internet videoing, you have gone through already many layers of coding/decoding. For text, it's actually the simplest...