good points in general. my whole thing on this thread is that threading should never really be needed with IB... it seems lolatency is also agreeing that IB does not care about going after high frequency, low latency customers... which is probably one reason why they rate limit their feed so it serves the lowest common denominator. although this thread also proves it's not a bad decision for the mass market, because even most well-intentioned people don't understand threads and probably shouldn't need to.
They rate limit their feed because many of their customers can't cope with the full speed of the market. Some of their customers are sitting there with 200ms+ latencies and can get upto 1-3s latencies because their connections are so bad. If they (the customers) go into a state where they have to do TCP retransmits, by the time they finish sending ticks to a slow customer, he will be hopelessly behind the market. Thus, IB samples the market and sends their sampled information down to everyone at the same rate. This way, the slow guys get the most up to date stuff at the cost of losing information between each time interval. Brokers like genesis actually will just start dropping information and overwrite their queues and then transmit what hasn't been sent. They are not reliable for feed data either, as you won't get a tick-for-tick representation of the market. So it defaults to the lowest common denominator -- some dude on dial-up sitting on a mountain top in Canada who wants to see 100+ symbols and put on trades for 10-20min+. This is why it's not practical to implement HF strategies using their data feed. It's better to subscribe to a real feed, and then use the IB ticks as a low-speed confirmation of the market picture. No one doing anything serious is getting market data over TCP. Multicast UDP feeds are the only way to trade the market quickly without penalty.
LOL. You're making yourself sound like an arrogant sod, tradelink. "lowest common denominator" What they've chosen is to offer a no lag feed for those people who don't think they need either: - every tick - every tick now. My guess is that most who fall into those categories aren't the highest uncommon anything. They just believe they can make money with a different trading strategy - and lets be honest, if you're going down that fast path you want the decision making computer as close to the exchange computer as possible. A good business choice on IB's part.
kiwi, my statement about lowest common denominator meant firms/people who don't have a need for high frequency low latency systems. It was not a judgement, I like IB. You can trade a billion dollars and have very simple technical requirements and thus be well-served by LCD. Google uses google apps internally, and google apps is very LCD as an example. I just thought it was interesting that lolatency worked at IB and they made design choices along this path. Didn't intend to upset anyone. apologize if I didn't acheive this goal for you.
I didn't work at IB, I meant I worked at an investment bank (IB) working on issues that were similar to those in this thread. lol. I know that they were hiring when I was looking a few years ago, and that they wanted people to take math tests at the time.
lol I guess these are the sorts of misunderstands that happen when a thread goes from a 'java interactive brokers api' to philisophical musings on industry at large... I am going to bow out of this thread so I don't contribute to more confusions. remember... don't use threads unless you know you need em! -josh
Tradelink. Yes - there is a new thread created every tick - but the first thing it does is hitting a synchronized method where it will block if there is too much work to process. The code handles the situation where you have tickPrice occuring at a regular interval - say 5 seconds - and you have your process method some times taking more than 5 seconds. If your process method on average takes more time than 5 seconds then the code won't work but then you got another problem than the stated one and you will have to not process some of your ticks. There is some overhead in creating threads but it is very small in Java and surely it is not a problem when we are talking 5 seconds intervals.
Final comment on Java and threads. If you have any ambition to write Java code, you should get a handle on threads. Otherwise you will inevitably end up with a mess. Java is inherently and unavoidably multi-threaded.
No doubt there is a lot of truth in what you are saying if you are dealing with very high frequency event streams, but it's orders of magnitude more demanding than what most here are doing (or aim to do). Problem is nobody ever defines what they mean by HF trading, quantifies the frequency of incoming events or specifies what they consider an acceptable system response time is. There is also the general and mistaken impression that "real time" means fast. But it is much more accurate and useful to define "real time" as a system with defined worst case response time. As for allocating new objects on the heap, I read some Sun stuff somewhere that claimed Java new outperformed C++ new by quite a bit. Can't remember the details. Certainly Java JVM does not call brk () for every new object created.