This isn't bar data. The packets are numbered sequentially so we are fully aware when something is missed. I don't want take this thread off on a tangent. This is raw data, nothing that feeds into charts, bars, or an execution platform.
In no uncertain terms: NO! Either in-house software or 3rd party applications that have nothing to do with excel.
what? http://www.softwaretoolbox.com/Tech...ObjectSoftware/DDEVsActiveX/ddevsactivex.html your DDE suck big time.. all my apps are using IB's activex control. without any data calculations receiving streaming data for 100 tickers from TWS consume <5% of CPU on ancient single core PC. it all depends of how you receive the data and how you display it. and excel sucks at it. even using various Microsoft controls will cause tremendous difference in performance. MSFlexgrid for example consume more CPU,displaying same data as a listvew control. excel cells is probably least effective way to display the quotes. specially streaming quotes. however-using VBA you can do pretty much whatever. including using activex to trade\receive quotes etc. buying 8 core CPU for displaying 50 tickers in excel is hilarious..you shouldn't have problem displaying 500 or even 5000 tickets on 8 core,if programming done right.
You can use Excel to back test and build these types of strategies... However, You really want to write an application that executes directly with the exchange bypassing all API's. This game is played in microseconds and your achilles heel will be your brokers back office for orders, confirmations and clearing. Advantage Futures may be worth a look... they have solid infrastructure and facilities for colocation and can assist with getting your application certified to trade directly with CME Globex. We were testing cross exchange arbitrage algo's ie. SPY calls covered by ES puts and vice versa... pulling the trigger when the instant covered trade was $ positive, We were able to identify significant trade opportunities in Excel processing the data feeds but were unable to reliably coordinate and execute orders cross exchange. This is a really fast game where your processing ticks and making trade decisions inside of 1ms but your orders and clearing run 20ms - 1000ms. Definite advantage to the member organizations that clear their trades internally.
Now there's some sound thinking. However, almost all signal-based systems can be considered to trigger "immediate" orders. Only gray box systems just alert the trader who then pulls the trigger....a very slow process relatively speaking. That being said, the salient point is regarding average-time-in-trade (ATIT). Unless you are co-located, I'd say you've got to have an ATIT of at least 10 times your order processing and clearing average time interval. Anything less than that and you are just rolling the dice. So, if it takes 1 second to process, the trade should have lasted at least 10 seconds....again, on average.
So let's say you trade a 3 min time frame. You have a trigger, and it took me, with Excel, 1000 ms or 1 second to send an order in via IB TWS automatically. So, it took about 100 ms in Excel to grab the data and calculate the signal, 300 ms for me to send an order from Singapore to IB's HQ in US, and another 200 ms from their systems to NYSE. That's a total of 500ms Now you said changing it to Java/C++ means it will definitely be faster, better, more profitable. Whatever. Then think again. You are only solving the 100ms problem. And maybe I colo to near New York. Save another 200 ms. Total round time is 500ms, instead of the 1 second. But it may still fail. Why ? 1) As a trading system yes I hv saved 500ms but so what ? From the time it triggered to the point I actually send in the order, the strategy doesn't really mater b/c its a 3 min time frame. I can execute anytime between 5s-10s slower. its fine. Maybe I lose a penny or 2 X 200-500 shares. 2) So after you send in the order. What ? Bound to make money ? No. It still depends on the market, something I cannot control. If the order gets filed, fine. What if it doesn't. I can only act on it. No matter how godly or fast my algorithm, I cannot control the market. So it doesn't matter if its 500ms or 1000ms. 3) And you trade long enough, you notice for 500ms, last done prices and data ticks don't jump 50,000 times. It probably stays there as it is, unless at volatile times. Except for quotes that are staffed in out 50,000. I don't care about that. So in a way we can argue until we invent the space/time machine. Challenge each other's limitations. Pros and cons. Good luck.
Savage - great post. Speed requirements in the millisecond range IMHO only make sense for algo's that work in the 1-10 second time-frame. They're most likely taking advantage of the level 2 distribution of bids and asks using a statistical model of probabilities of movements based on certain bid/ask metrics.
Buy off the shelf HFT solution from http://www.milleniumit.com/media_room/gui-press_releases.php The new release of Millennium Exchange can handle sustained volumes of over 100,000 orders per second, with core latency remaining below 1 millisecond. Moreover, the platform scales evenly, with each increase of 1,000 orders per second typically requiring only $50,000 of hardware