Tradelink, Still the code suggests that processing of each tick is fast enough not to create the performance bottleneck.
Because the processing of each tick is much slower than the speed at what the ticks are sent. The system simply is not capable to chase after each tick it receives.
Well in that case you obviously cannot process all your ticks in realtime. (But as I read newguy05's post this is not the problem.) If you are on a multicore machine then spawning more threads may allow you to better utilise your multiple processors.
chvid, starting a new thread is not a problem per say... but the code posted by the guy above (can't remember his handle)... starts a new thread everytime a tick comes in. this will result in thousands of threads being started, the overhead of managing them (since his code did not use thread pools) will likely dramatically be slower than not using them at all. threading is very application specific, that is why I requested he post more details on his setup if he thinks he needs threads. I get the feeling he is sort of trying to plan for the future more than solving a present day problem. gvi, your point is only true on a one core machine... on a one core machine, adding extra threads would relatively slow down the processing because of the extra overhead of starting/stopping and managing them.... thus the solution would be as you suggest to ignore or aggregate some of the ticks coming in, and do it quickly enough such that you avoid the bottleneck. there are many ways to do this. on a multi core machien however, which probably most of us have in the past few years... if processing a single thread tick stream is too slow, offloading the high latency processing off to seperate threads does increase the tick throughput of the entire system. I think your example is very general and you're trying to say that if the data feed is too fast foro the machine, you can't really do anything. .. I am not going to touch that because it's not a very practical example. Generally speaking just handing off the tick from ib/broker/feed is very fast... <1ms... so it's really the processing you're doing where you can get into trouble. And when I say processing, I am talking about anything... building bars, ignoring ticks... all of these things require finite cycles. Some more than others. That doesn't seem to be the case here. anyways this has turned into a sort of hypothetical thread, so unless the original poster really needs threads and can provide reasons why he thinks so I am going to bow out.
tradelink - no - the snippet I posted most likely will solve his problem if it is as described. There is no need for thread pool. And there is no need to make this harder than it is.
chvid, you are creating and starting a new thread each time tickprice is called. this is going to result in hundreds of thousands of threads being started. this is not right. I am not recommending a thread pool, I am not recommending he use threads at all. but creating a new thread for each tick is wrong.
Indeed, creating a thread for each tick is VERY wrong. One decent way of doing it is to add the incoming ticks to a queue. Lookup the Javadoc for LinkedBlockingQueue This class is thread safe (in package java.util.concurrent) The code would look something like Code: public void tickPrice(final int tickerId, final int field, final double price, final int canAutoExecute ) { tickQueue.offer (new MyTickObject (......)); } The "black box" should run in it's own thread and poll the "tick queue" The main line of the black box will then look something like Code: for (;;) { MyTickObject tick = tickQueue.poll (timeout, timeUnit) . . }
david, yes. an even better method is not to poll at all, but to send a message from first thread (populating queue) to the second thread, that one or more ticks have arrived. this way you do not waste polling cycles. that is method we used here: http://code.google.com/p/tradelink/source/browse/trunk/TradeLinkCommon/AsyncResponse.cs this is a slightly simplistic example still, because it only uses one thread for tick processing... if you had many cores and were doing some very itnensive calculations you might want to use multiple threads... but this is VERY rare and most people don't need threads, especially IB traders because of IB's tick throttling. otherwise above method works for 80-90% of the situations even requiring high latency computations.