All true .... to a point. It's what you do upstream with the data that may or may not stress your processing and/or storage facilities. I have no trouble whatsoever in handling 500 high volume stocks Level I off IQFeed which claims to supply every tick. Can't even keep one core of a 9550 quad busy. Java application. I have never even profiled the code for performance as it is not an issue. In fact most of the CPU utilization is for "christmas tree" - quote sheet - tables. Having said that 1000 Level 2 instruments could be another matter altogether, requiring some serious design for performance. Depends on what you want to do with the data.
Nice post. "DTN NxCore is extremely efficient in bandwidth utilization and will save you thousands of dollars in monthly communications costs. In a single T-1 (1.5mbps), it will fit 40,000 quote and trade updates per second; In 6.0 mbps, it will fit 160,000 updates/sec." "DTN NxCore databases everything: all quotes, trades and other real-time updates are stored on your computer." This is so NOT necessary... Few will use > 5% or even > 1% of the ticks... This is a potential bottleneck... And the core problem with streaming the entire US Tick Space... As the only available configuration. http://www.dtnmarketaccess.com/
it's only not necessary for someone who doesn't need nxcore. which in that case, they should be using iqfeed. i can assure you, people who are willing to pay 1k+ a month and pull every quote are going to be interested in more than 5% of ticks.
As is obviouly your brain, your money, your girlfriend. Potential means nothing if it has a 0 chance to ever be real. They do that for 2 great reasons: * One, it is easier on THEIR side. They stream only a number of different exchange sets. Filtering thousands of symbols per second for thousands of different users is a LOT of resource waste. * They target people whorealize they may want that at one point in the future. If you switch symbols tomorrow... you have all back data since you started from the exchange. I bt most people take the stream and load it into their data handling infrastructure to start with. Where the heck is the bottleneck when people can get 20mbit connections those days?
$1-2K/month is chickenfeed... I manage mid-7 figures and do 500 trades/day... And the 200-300 stocks I track account for < 1% of all ticks. Probably 80-90% of ticks... Are generated by crippled, trash stocks like C and CIT... And 100 others like it... The sort of game where you will be raped by Goldman in all 3 orfices... Certainly no place for a serious quant.
Geniuses simply and simply... to an elegant end point... In contrast, smart people complicate the hell out of everything... They cross the street via the North Pole... And think that makes them a "genius". As a software engineer... I have spent 15 years optimizing and ** simplifying ** my systems... Where the focus is all on TRADING and EXECUTION... Not data warehousing countless T-bytes of useless data... From dodgy vendors like NxCore... Or pattern matching random tick streams... The ultimate Losers Game. We have very different philosophies.
No, we just have one person (you) not interested in any more modern work and thinking he may develop forther - end of evolution. We have another person here (me, most) that say "ok, i may have some thing that works, but maybe I have an idea tomorrow for which I may actually want to analyze order books, but damn, stupid me does not have a history of them now because I was thinking like that other evolutionary dead end". GB of data is damn cheap to store these days. Getting it is hard in the first place.
my point was 1k/mo is rgt around the mark where you'll start paying for a 'real' api feed. and if you can afford to pay for a real feed the odds are exceedingly high that you are an institution, hedge fund, or an advanced indi trader. of that group, the odds are exceedingly high that you're going to be watching > 2-300 symbols and working with > 5AUM. otherwise, you just pull quotes from a good broker api and call it a day. i'm not calling you a piker, but these types of products aren't built for guys like you. they're built to provide a cost effective way to have access to exchange like feeds without having to write to and support exchange like api's and fix protocols, which do not support 2-300 symbol requests... they send you the whole bag. so, while a high percentage of ticks may be in 'trash' stocks, those ticks are hardly trash. you obviously haven't found a use for them yet, but like i said, these types of products aren't strucutred for trader's like you. they're strucutured for organizations that can make use of the data. calling a product protocol 'unnecessary' just because it doesn't work for you is silly. because for the money... pulling anything more than 1k quotes, nxcore is a steal. i haven't hear you proivde another solution though, so if you can, be glad to hear it. ps i don't use them, have a couple of trader friends that do, but have been considering them primarily for the data warehousing. would take me tons of work and mucho $ to pull and db that much data any other way.
Several K/month on a feed... Is dwarfed by the $100K/year that a competent C++ engineer commands. Even a shoestring Algo operation... Will have 2 traders and 2 engineers plus some admin staff... So you are looking at about $500K annual fixed office costs... If you are dying to self-destruct competing head-to-head with Goldman. Google all you want... But you will not find a single intelligent review or discussion of NxCore... (Maybe in their Forum, but it's private)... Which is an extremely bad sign... If you wanna build your business around such a tiny vendor. These guys are too lazy to even write a simple Quote Server... That runs on the Client PC and strips out unwanted data. My 3 basic points are: (1) Enterprise data feeds are a dime a dozen... http://www.bobsguide.com/ And even a tiny firm with a $500K tech budget... Is unlikely to chose NxCore or anything from DTN. (2) If you decide to specialize in Algo trading... You are, by definition, chosing to compete with the strongest players in the Game... Kinda like going to Vegas and playing high stakes poker at the Bellagio. In sharp contrast, I exploit the weakest links in the markets. (3) Hobbyists with 15K (not you)... Should STFU because 99% of them are delusional... Most will be busto soon... Or become "subsistence level gamblers" grinding $1000/month. NxCore is for the high end of the hobbyist niche.... For people that enjoy coding more than trading.