ActiveTick vs. IQFeed vs. IB (TOFTT)

Discussion in 'Data Sets and Feeds' started by SeattleAlgo, Sep 6, 2012.

  1. Take One For The Team

    I was frustrated not been able to find any reasonable comparison between AT and DTN feeds short of "DTN is the best out there" (without much clarification of what the "best" actually means). So I had to spend about $500 and a good part of the Labor Day weekend to do my own assessment. As these threads are searchable in Google, I am hoping that somebody will someday find my notes useful.

    To better understand my remarks below it helps to know what I was looking for in the first place. All my trading is 100% back-tested and 100% automated so I needed a reasonably accurate and reasonably priced source of historical tick data for US equities and a reasonably fast and reasonably accurate source of real-time data for US equities. And I wanted both to hopefully be better than my "free" IB data. So I looked at:

    1. NASDAQ-On-Demand. This is historical-only ticks and quotes. Rather expensive (~30 cents per symbol per day of trade data, 2x-3x that if also want quotes and not just trades). Minimum is $100 per month. I downloaded trade data for a number of stocks and days I was interested in. The data is very clean, all trades come with the corresponding attributes (open, close, cross, intermarket sweep order, etc.). Nice clean web-service interface - very easy to start. I used the downloaded trade data as a reference for those other options below.

    2. Active Tick. They have both historical and real-time data. The historical data is somewhat less perfect compared to NASDAQ (example: sometimes the NASDAQ open trade is only reported once as the market center open and not twice as it should be (the second one as a regular cross trade with an open attribute). NYSE open is reported correctly. Likely because of this bug their bar data has the NYSE open as the open for the day, even for NASDAQ stocks. But individual trades are very consistent with NASDAQ history and also properly marked with the correct attributes. Their real-time data seems OK though unfortunately they have another bug there - the intermarket sweep order attribute does not get reported for the real-time data, while it does get attached to the corresponding trades when they are later queried as history. I reported this as a bug to them. Then see below w.r.t. feed latency. The quality of their samples and docs is the worst (their COM API is half broken, their C++ API is OK), but having 15 years experience in software development I don't care much about this.

    3. IQFeed. I am disappointed. Seriously. IMHO, DTN/IQFeed is not in the business of providing tick data for serious traders. They are in the business of extorting money from the wanna-be traders, and I admit this business is going great for them. Their positive is that they do provide both historical and real-time ticks and quotes. They even have Level II if this is of interest to anybody. Also their sample code and their documentation are most likely the best in class. Their negative is that the timestamp resolution on all their tick data is 1 second. That is milliseconds are not even reported. And the trade attributes are not reported either - neither for the real-time stream nor for the history.. And the way their pricing works is like "this will cost you $xx for the feed, $yy for the setup fee and $n in exchange fees - now, please, enter your cc number. Then on the next page one learns that on top of the $n exchange fee there is also a $4 per exchange "maintenance fee". Really? And then it turns out that access to IQFeed does not come with access to the API for it. The latter is $300 more, non-refundable, per year. Enter your cc number. And then on the next page one learns that on top of those $300 there is $25 for something else. And after finally being approved for dev access, you realize that what you have got is a stream to feed maybe a charting application. Sure, there is a big market for these kinds of things, and they are milking it. But for my needs they are basically a toy. I cancelled my trial, but still paid $325 for access to the API.

    4. IB. As most folks are well aware, they provide 1 year of 1 second and up history data (no ticks though) and real-time quotes for a nominal price of $10 / month. The real-time trade ticks now come with a timestamp, the quotes do not. No trade attributes are reported at all. Comparing a recorded sequence of trades against the NASDAQ-on-demand history for the same symbol and day reveals some interesting discrepancies - the total volume for the day matches within like 1%, while individual trades are often missing, while others are reported with a larger trade size and yet some other cannot even be found in the NASDAQ master. While obviously not suitable for any serious trading, this feed cannot be beat for its value for money :)

    Latency: all the threads here with folks complaining about their feed latency - you have checked the stability of your real-time clock, haven't you? I was surprised to find out mine drifts by as much as 300 ms in 15 minutes! I had to configure NTP to sync every 15 minutes - my client did not allow for more frequent syncs, and having a 300 ms drift in between was really bad. My next investment is likely going to be a dedicated time-keeping card. Still, the numbers I have got for the real-time trades (just trades, not quotes) feed:

    IB: Min: -220 ms, Max: 2800 ms, Average: 570 ms, Std. Deviation: 308 ms.

    AT: Min: -300, Max: 12100 ms (12s), Avg: 268 ms, Std. deviation: 276 ms. (These numbers for AT held until the last 30 seconds of the trading day when a bunch of late reported trades came up and moved the stats to Average of 294 ms and Std deviation of 509 ms.).

    IQFeed: Did not try due to lack of millisecond timestamps.

    (I know that having the median instead of an average would be much more useful, but average is just so much easier to calculate.)

    Being on the West coast (in Seattle) and having a 300 ms clock drift, I think these numbers are not bad at all. AT appears to be better than IB by about300 ms. I am having a hard time explaining an occasional negative number though. Getting quotes from the future would be a real easy way to make money :) Then as far as I understand it, the difference between 100 ms to 500 ms does not matter much as by the ~100 ms mark all the HFT has already been done and all the vacant positions on the bid and ask have been taken. One needs to trade with this in mind.

    I am cancelling Nasdaq-On-Demand and IQFeed. I am keeping Active Tick. We'll see how it goes.

    Hopefully this helps.
    AKTrader786 and TooOldForThis like this.
  2. Eight


    I'm using chart geometries [price extremes are what's important] for indications and trading at a far slower pace than any HFT. Relative volume bar/bar is important but only has to be accurate enough to know if it's >/< the previous bar. I'm thinking that IB data, especially if collected in realtime, might be good enough.
  3. Bob111


    good info! Q about IQ and ActiveTick-in their streaming quotes-do they included data from ALL market centers and ECN's?
    cause as far as i know-on most brokerage firms ARCA for example required separate subscription per user,per account. same in IB. if you not subscribe to arca-you not going to receive any data from it.
    same question for their L2-all data included? even from ARCA?. and last one-if you accessing their L2 data(both IQ and Active)-do they provide DATE,TIME of the order,displayed in L2? i remember back in a day MB trading was perfect,cause they provided all info.-price,size,exchange and time,when order was entered. they also displayed ALL orders(odd included,even if it was 1 shares). it does helps sometimes to know,which one is passive order,that was sitting in query for a while.
    what about field ,showing the name of exchange,where trade was executed? can we get that via API or whatever?

    my experience with any 'retail ' level providers is that ALL of them are unable to provide complete data(see Q's above). and they are basically a joke. every single one of them is missing something or sometimes half of the basic fields that should be included in every decent data package..
  4. vicirek


    Thank you for sharing. There is not much quality reviews on AT and I am considering using it.

    Active tick goes back 3 years for historical data, IB 6 months to 1 year depending on bar period. What IQ offers in terms of historical data?

    By the way there is no feed that would be 100% perfect for many reasons including data coming from many centers. Handling bad feed is part of good software and strategy design (obvious but many times lost in quest for perfect data).
  5. Yes. I am not sure about ALL, but a quick look at AT shows trades and quotes from NasdaqOmx, NasdaqOmxBx, Arca, Bats and BatsY. IQ will likely limit you to the list of exchanges you agreed to pay the fees for.
    AT does not offer L2, only IQ does. I didn't subscribe to L2 as I dont have a use for it, but looking at the fields for the L2 update, it does include things like timestamps for both bid and ask, their sizes and the exchange code.

    For L1 trade ticks, yes you can get the exchange code in both AT and IQ. Not with IB though.

    > What IQ offers in terms of historical data?

    120 days of ticks. Longer for bars. It is somewhere on their website, I don't remember exactly.

    One last remark about AT - they even provide you an API to get a number of pending updates, which have already been delivered to your PC, but have not yet been processed. Very thoughtful of them. To achieve the same with IB I remember I had to do something like "process the tick - wait a few ms - is there another tick? No means this was the last one in the batch delivered. Otherwise keep processing the next tick".
  6. selcon


    Great review SeattleAlgo. I'm looking to sign up for the AT feed API, working on the windows platform.

    You mentioned that the COM API is half buggy, can you elaborate more on that? Was thinking of implementing something in Visual C#.

    Which do you think is the best language to use on a windows box?

    Thanks in advance.
  7. macdice


    About IQFeed truncating all times at one second resolution, I asked about this and I'm sure they won't mind if I quote part of the reply I got here:

    "Better timestamp resolution is high on our priority list right now. We are evaluating what resolution is on the various exchanges, and taking a look at the overall impact on bandwidth before we commit to a solution. However, I can tell you we likely will have an option in 2013."

    I have to admit the $300 thing irked me too, as it doesn't seem to appear on the 'Basic Fees' page. It means that the basic cost before exchange fees is $65/month for people subscribing via certain 3rd party tools, but effectively $90/month for programmers writing their own software to talk to iqconnect.exe.

    That said, I assume the vast majority of users are using 3rd party charting tools etc etc, and we programmer types are probably just annoying. Running their software on Linux, expecting microsecond resolution times, hammering their servers with requests etc etc...
  8. The bug in the COM version of the API is that it never returns quotes in the tick history data, only trades. The flag indicating that one wants quotes is for some reason not propagated down to the native API, where it is always false. Triggerring a break-point right before the native API is called and patching the flag on- the-fly does result in getting the quote ticks returned in the query response. I reported this to AT, but it is not clear they have anybody even looking at these reports at all :(

    Anyway. I am sticking with their native API and implementing my own C# wrapper using PInvoke. About half-way done at this point.

    Also I am still waiting for my MasterClock timing board to arrive. Once I receive it, I will report on the "true" latency value for AT and IB.
  9. vicirek


    IQ Feed developer access: what does it let you do with the data? Any restrictions?

    SeattleAlgo who was so kind to post his review mentions in regards to data through developer access:

    "stream to feed maybe a charting application"

    Does it mean that there is no way to store, transfer or manipulate received ticks or historical data or it is not allowed?
  10. The "charting app" remark was meant to highlight the lack of milliseconds in the timestamp as well as not having any trade attributes. Otherwise there are no particular restrictions short of the obvious ones like not to distribute the data to others.
    #10     Sep 24, 2012