intraday snapshot data feed needed

Discussion in 'Data Sets and Feeds' started by lugnutz, Nov 30, 2008.

  1. lugnutz


    Hello ETers,

    After searching and lurking for a while, I haven't come up with a good answer... hopefully someone here can share some wisdom.

    I'm a longtime software "Elitist D*ck" (tm) :D but only a few years into the world of trading. I've developed my own daily swing trade identification software that mines commonly available free daily equity snapshot data and backtests very successfully (also my own app), but I am now looking for an intraday data provider to better pinpoint my entries. This is not a full ATS, but more of an advanced trigger that notifies me when the planets have aligned for a good trade.

    -real time trade snapshot OHLCV (open/high/low/close/volume) data
    -ideally 1-minute intervals, though 5-minute would be acceptable
    -domestic equities only: NYSE, AMEX and NASDAQ
    -around 250 simultaneous symbols, but can work with as few as 100 if that's the limit for a given plan.
    -ideally 5 years of daily historical OHLCV data for backfill (more is better, but I have a decent database already in case daily backfill isn't provided.)
    -ideally 4+ months of intraday 1 or 5-minute historical OHLCV data for backfill (obviously more is better)

    LIKE TO HAVE, but not absolutely necessary:
    -common indexes for DOW, NASDAQ Composite, NASDAQ 100, S&P 500, S&P 100, and VIX

    -foreign equities
    -options or option chains
    -real time tick
    -fundamental data
    -book/depth/level II data
    -for level 1, do not even need bid/ask/size. :)
    -automated order entry/exit

    As someone who has done extensive protocol development in the past, I prefer a well-defined TCP protocol interface so I can read/write my own packets... rather than run some hacked up/slow/bloated/buggy API code written by the data provider, but will use what I must to work with the provider.

    Here's what I've come up with so far:

    OpenTick - cheap, packet interface OK and fairly well defined, but questionable availability. Been on waiting list forever trying to get an account.... :(

    Interactive Brokers - well-defined API but must run alongside their client. (i.e. probably slow and/or inefficient) Cost structure not really immediately clear. Negative threads here mildly spooky.

    IQFeed - reasonable cost, API not very well documented from my quick overview. Packet interface may be OK, but would have to be reverse-engineered from packet-sniffing their client. Not sure they have snapshot data... may have to build it from tick data, which isn't ideal.

    CQG - mentioned in another thread here... looks very good, but unknown $$$. (i.e. likely expensive.)

    DDFPlus - looks great, but very $$$.

    Quanthouse - seems highly regarded here, but again, unknown $$$.

    eSignal - I have a question in with an eSignal rep to see if there's anything I can do with their feed (I'm a QCharts customer.) Their QLink product doesn't look like an exact fit.

    Many thanks in advance for any input you may have on these providers or others.

    (In case anyone is curious, my current system averages about 50 trades a month, 86% average p/l ratio and net return of about 8% a trade including losses over the last year. I currently have around 15 algorithms that identify trades... varying in quality and # of trades identified... it's a work in progress but a good start.)

  2. "Interactive Brokers - well-defined API but must run alongside their client. (i.e. probably slow and/or inefficient) Cost structure not really immediately clear. Negative threads here mildly spooky."

    IB API is not slow. No problem at all handling futures data and futures book data that updates 10 times a second.

    IB costs are specified in great detail on their website. Best value for around for stock commissions and data feeds. Can easily be beaten on futures commissions.

    One problem is limit of 100 simultaneous symbols (unless lots of commissions paid to get additional symbols).

    Nothing spooky about negative IB threads - some traders just don't like IB and even traders that like IB would like to see some improvements made in different areas.
  3. lugnutz


    Thanks for the feedback on IB. Do you know if their app is multithreaded and takes advantage of multi-core systems? Seems like a few modern Java apps are almost acceptable performance-wise, but the legacy of inefficiency is still there, so I default to skeptical.

    I went back and found their equity pricing details on a second look. I missed the tab-subtab the first time around. :) I agree about their fees - very reasonable.

  4. Euler


    I think it depends on the compiler/vm you use, etc., doesn't it? You can use gcj to compile Java into a (likely far faster) binary; no idea whether it works with IB, but I'm sure someone's tried.

    Another option might be TAL:

    I think this uses C#, which may not necessarily be faster than Java. But it does seem to have a pretty nice interface. I think they used to offer win32 C++-type API's; not sure if this is still true(???).

    Nanex NxCore, ActivFeed, OpenQuant, and Interactive Data PlusFeed all have C++ interfaces (amongst others), but they are significantly more expensive -- see other threads for discussion.
  5. lugnutz


    Thanks for the heads up on TAL ... quick glance is pretty promising for $300/mo, which is more than I wanted to spend but a lot less than many of the other feeds.

    Java is just a symptom of the underlying disease. :cool: The sad state of software engineering in 2008 is another thread altogether.

  6. Does anyone know what Realtick's symbol limits (for API usage) are now?

    I used them in the past but that was 5 years ago.
  7. Euler


    Apparently, it's 1500 now.

    Beyond that, my guess is that you may be better off (both API-wise and $ wise) with one of the other vendors I mentioned above.
  8. Tenfore QuoteSpeed - 250 eur/month (no API, DDE only, you can try reverse-engineer their client)

    CQG - 500 eur/month (client+API), +300/month for API trading

    DTN NxCore $500?
  9. lugnutz


    I wanted to post an update in case anyone was following this thread. Sorry, but some of the costs include exchange fees for my needs and some don't... and other costs will vary, so take the numbers as a rough approximation.

    Opentick - would love for these guys to be real, but can't wait for an account. Someday maybe????

    OpenQuant - someone recommended this, but from what I can tell, they are not a data provider.

    Taltrade - looks very nice. 1500 symbols for $300/mo.

    Stockwatch - not sure of api, but very reasonable cost - 100 syms for $24/mo and 300 syms for $54/mo. I think based out of Canada.

    DDFPlus - barchart's pro level service... $1500+/mo. Looks good though.

    Neotick - can't learn anything from their site for data and fees. Didn't bother contacting them.

    Rightedge - from what I can tell, they are not a data provider. - $45/mo, but not sure if have API. Didn't bother contacting.

    CQG - has API product, but costs $$$ (see post above.) Very professional site and highly recommended. Didn't bother contacting.

    eSignal - has a desktop API avail, looks mildly clunky on first glance. Also requires eSignal app running to connect.

    Genesis - 250 syms (I think) for $125/mo. Web site isn't clear if API is available.

    Barchart - 200 syms for $65/mo and 500 syms for $80/mo if sign up through NinjaTrader. Tried contacting to see if have API or allow use other than through Ninja and NO RESPONSE from sales dept. Guess that means no.

    MyTrackPro - 100 syms for $50/mo. Cheap looking web site and stock is in the crapper...

    IB - great fees, but limited to 100 syms unless whale commisions. :) Must run their Java-based app to connect spooked me, but others say performance OK. This was #3 choice.

    IQFeed - 500 syms for $76/mo. They also mention a $300 annual developer/api fee... but I never pinged them to get a quote. Their $50 setup fee is supposed to be waived if you go through NinjaTrader. This was my #2 choice.

    .... and the winner is....

    TD Ameritrade

    Although their web site doesn't mention it, they have a very nice API available to get 2 years of hist 1-minute snapshot bars, real time tick and/or snapshot bars, backfill, hundreds of simultaneous symbols, automated order placement, etc. All at no extra cost or monthly fees (for non-pros like me). No historical option data, etc, but fits my needs perfectly. They have a well-documented DLL as well as documented protocol - so you can code it all yourself and connect directly from linux, mac or whatever platform floats your boat. Hopefully the performance and reliability lives up to the expectations. :)

    (sorry if that came across like a sales pitch; I have no affiliation with TDA but will be a customer soon... will follow up after I get my app running with their data.)

  10. Just a comment. OpenQuant is not a feed, it's a strategy development platform. QuantHouse is the provider of a low latency data feed ,

    #10     Dec 10, 2008