Data feed for intraday market scanner every 60 seconds

Discussion in 'Data Sets and Feeds' started by thecoder, Feb 5, 2019.

  1. thecoder


    for the US market I need a data feed for an intraday market scanner that scans every 60 seconds all the 8000+ tickers.
    The data list should have NBBO quotes (National Best Bid and Offer), Last trade price, cumulated intraday volume, and date+time, and perhaps also the intraday High and Low price.

    If we assume 6.5 hours per trading day then there are 6.5 * 60 = 390 minute bars per day, and
    assuming 21 trading days per month makes 390 * 21 = 8190 minute bars per month, so for 8000 tickers this makes in total about 65 million data records per month. If we assume about 100 bytes per record then this makes about 6 GB/month (about 300 MB per trading day).

    Which data vendors can deliver such data, and how much would it cost per month?

    Last edited: Feb 5, 2019
  2. ZBZB


  3. thecoder


    I'm writing a scanner for a client (in C++ language) and need a data source.
    Thx, will check esignal, but the others I assume mean scanners of themselves, not data source for such a scanner of my own.
  4. How about web-scraping the data from sources like yahoo, google, etc.
  5. thecoder


    I think that will take much time for 8000 tickers. I need 1-minute bar data for all the 8000 tickers.
    Or, do you know of such a single snapshot list for all tickers?
  6. Not all at once but if you use threads you can download say 10 pages at a time.
    BTW web scraping is way easier in Python.
  7. ZBZB


  8. Metamega


    Could try shooting them an e-mail. Not sure of cost but wouldn’t be cheap. Did see some posts here more recently about having some issues when they stopped using DTN(IQFeed) as a partner. Believe they were handling the billing and data requests but they’ve since gone to their own setup.

    E-signal for the price is going to have some lower symbol limits. Think from seeing on Amibroker your looking at 200 simultaneous real-time streaming data.

    IQFeed has a 500, 1000, 1500 symbol limit for simultaneous streaming. Think base fee is like 90$ now plus 50$ for extra 500 symbols plus exchange fees.

    Could e-mail them and ask, have seen some post from them where they mention their more lenient with requesting new symbols and their limits are more for simultaneous.

    8000+ tickers is a lot. To be honest you could cut that down to 2000 or less if your looking for decent liquid tickers.
  9. thecoder


    I briefly looked at all the mentioned vendors here, but haven't found the right one.

    Maybe it indeed makes more sense to do it like in your suggestion; besides multiple threads I also could use multiple different IPs to overcome possible overuse-limits etc.
  10. I agree, do some filtering through IB market scanner or something first.
    #10     Feb 5, 2019