Here is the story of my quest for the retail data feed provider. I used to be with an Opentick for 2 years. I was complaining about them - you know, almost no support, some bugs (that I was able to fix myself - it was open source), some (although small) inconvinuences. As many of you know, company closed the on Mar 16 - just 4 days before I finished my program that I was working on for almost 2 years and was ready to go to production. Bammer! Now, I needed someone else as a replacement for the data feed. Here what I needed: - Data feed provider should have an API. Yes, I have my own blackbox, and I don't need any of those lousy 'trade stations' - I want the program to work for me, not me working for the program. - I should be able to use that API from my program. And, surprise, it's C++ compiled to 64 bit - yes, I use hell of a lot of memory for my logic, and my workstation is quite a beast (32GB RAM, and I would be happy to get more). Well, to clarify, after being unable to find any other provider that could give me that (Opentick worked with no issues - I recomplied their source myself to 64 bit), I found a way to overcome this problem. Solution is not ideal, and I don't like it (I have 2 programs now - main for calcs, which is 64 bit, and one for data supply, which is 32 bit, and they talk over shared memory), but there is no other way, as NO OTHER data feed provider gives me 64-bit DLL. - The API should be a) efficient and b) provide all the data I need. Let me elaborate here. Here what I need from the feed: 1. Give me list of ALL equities available for trading right now, along with the yesterday's volume. I then sort them, get 500-1000 most active ones, and then apply my trading logic to them. I'm not going to manually enter all these equities - after all, companies open/close every day, and I dont' want to spend my time tracking them. I made a program to be ABSOLUTELY automatic. 2. Give me level 1 and level 2 tick-by-tick real time data feed WITH TIME INFO. You would be surprised how many data provides skip time information in their bid/ask data. As for the efficiency, I was tracking 1000 equities in realtime with Opentick, and feed alone was taking less than 1% (this is not a typo - yes, less than ONE percent) of the CPU power. 3.It should be reliable. I know, there are many grades, but make it at least this way - if you can't provide good data (for exampe, during market surge), just stop the feed - don't provide late/false data. In my view, I'm not asking too much, am I? I think, these are the basics of what any decent data feed provider should provide. And finally, I'm not a millionaire to pay skyrocket prices for the feed - otherwise, why I would need that program in the first place? I would be happy to pay $100 per month, can (not happily) pay $200/month, and would be almost dying, but still possible to pay $300/month. Over that - sorry, I can't. So I started my quest. And very soon realized, that Opentick, that I was complaining about, in reality was an ideal, that no company can even touch. Darn! First of all, about 90% don't have an API. You can access the feed with their program only - thank you very much. Half of the companies I called don't even work with individuals. The others charge $750 and more per month. Two companies hung their phones as soon as they heard word 'API'. Three companies didn't answer an email (they didn't have phone# listed). Two companies never returned voice mail. Many companies keep the DOCUMENTATION for their API under the hood (like IQ Feed and Marketfeed/Datainc), so you don't even know if they have what you need until you pay them for the API (!). Let me rant here a little bit more. I understand payment for the service (the feed). I have no idea why to pay for the DOCUMENTATION for your API! It's like saying 'I might have what you need, I might not, but you will not now until you pay me'. Some brokers provide you with the good feed (>500 symbols realtime) if you have an account with them, and actively use it. I'm not against it, but either their comissions are too high, or they don't have trade API, or they can't provide what I need. Ameritrade is the bright exception here. Over half of the providers want to have an additional piece of their junk software to be installed on your machine in order to provide you with the feed - IQ Feed, eSignal, IB etc. Man, they should fire their programmers immediately for this idea. Who the heck needs another link in a chain to break?! Why Opentick, Ameritrade, Genesis etc can provide you with the API that is standalone, and others can't? I spent a week online researching companies, and called over 30 of them. All-in-all, here are my findings: 1. First place (with exception) goes to Ameritrade. They have a standalone API (accessible as DLL), which is very efficient for level 2 quotes, and very bad for level 1 quotes (they pass too much non-needed data with it). Their feed is free if you have an account with them, and they satisfy all my needs as a broker - decent pricing, good (although unnecessary complex) API, and ability to place complex orders (one triggers another etc). The exception is - their API does not supply me with the list of equities. 2. Second place goes to IQ Feed. I really don't want to award them with it, but there is just no one else out there. Pluses: they have an API, and they have list of equities. Minuses - they place an additional piece of software of your machine (sort of mini-server), that you need to connect to. It is reported to be buggy (crashes) and boggy (takes up to 80% of CPU time). And, oh yes, they want money for docs - you would not see them until you pay $300. Nice, right? The only feed that comes close to what I need, is "big brother" of IQ Feed - NXCore. But there is small "but" - they start from $500/month. Actually, I can't believe this is 21st century, and there is still NO decent real time market data feed provider. Where the heck is the so-called 'competition' of the companies for the customers?! OK, I vented. It looks like I will be going with IQ Feed (unfortunately) for the list of equites + Ameritrade for the rest of the feed, but... Opentick, where are you?!
It's always like that if you go where few have gone. I saw that complex problem looming years ago and simplified my setup by deciding to trade just one future. Whether manual or automated, the list of stocks thingy always was a royal frigging pain.... Currently I can do with just Openquant and IB. A bonus is that I can run the whole trading setup through my whitelisting firewall with just two numerical url's in the list. Even the guys at the DNS plants can't hack me! Einstein said to simplify as much as possible but no further, I'm in that sweet spot currently... when the world of autotrading actually makes it doable to trade a zillion stocks and still meet the Einstein criterion, I'll be all over it like white on rice....
Did you check out DTN Satellite? They have external hardware, not sure how it interfaces to the computer, maybe you could hack that somehow... the service does not limit the number of instruments, that is the main thing, but they dropped something from the mix in recent years, I think it was options... you wouldn't think it would be the stocks... another advantage of that service is in the security realm, it's a one way transmission, if you are whitelisted they can't be a venue for somebody trying to steal your intellectual property... What are you doing for security there btw? I found that I had to become my own security expert. I spent more time working that out last year than I did in strategy development. Currently I'm pretty tight, one computer dedicated to the business, hardware SPI firewall that whitelists and only allows two urls in or out... about all I have to do here now is to maybe figure out which processes the os doesn't need at startup and prevent them...
You may want to check out these guys... They cover ALL US stocks (no symbol limit bs). Rock solid and very fast. $350/mo. & up depending on your data needs. (if you need bid/ask data then figure $500/mo.) Here's API page... http://nxcoreapi.com/doc/concept_Introduction.html
Well, I solved all my security issues years ago - I'm in the 'stealth mode' in the Internet now. And yes, I keep only needed services running on my XP - I need every bit of CPU power, and want my machine to be stable. In fact, even though it's Windows, I forgot when it was last time I reboot And no, I'm not THAT paranoid about the security, although my day job is for the security department )).
I mentioned them in my post. And they still have mini-server program runs as a service (well, in this particular case I must admit this is almost reasonable), plus still no fund data (well, this could be solved by requesting yesterday's data, and sorting by number of trades, so this is just a minor inconvinuence). But again, $500/m is way over what i can pay right now. BTW, I forgot one glaring minus for the Ameritrade API: they have ... NO TEST server for it! You are supposted to test your ordering software on live account (!). I guess there are very few brave souls for that - including me
I hadn't read your post completely, but with regards to NxCore taking up a lot of cpu space, that's not true at all. In fact, I have it running on a machine here and it's using about 1% of cpu and never crashes. This thing is rock-solid.
I'm not saying that it's bad. I'm saying there are still some (though small) inconvinuences, but the main problem is the price
I have been with OT also and think I know how you feel. Yes, they had a pretty good design for their API, not such a mess as e.g. IB. If you look what positive could come out of this Fractals may be right: This is the way I went too (well not exactly limited to one future). For the effort there is much more reward in futures. The good message is you have the worst already behind you: Learning how to build a program that deals with a quote feed API. Not so many have this capability. Changing the quote feed provider, API interface and even the overall strategy is much less work. Also the changes will make your whole program more robust.
This is not a problem. Problem is to find "jack of all trades", i.e. provider that meets minimum requirements. I can't believe that they still don't exist. I've already told my boss that there is perfect idea for the start up. Heck, if I would not be doing my auto trader program, I would venture into that startup myself!