How does IB's API work with historical data?

Discussion in 'App Development' started by MJM, Mar 28, 2015.

  1. Butterfly

    Butterfly

    that code is a bit better but as useless, you are going through loops and layers to build some very simple trading tasks. Seems you just learn how to program, playing with patterns rather than focusing on functions and utility of your code. This makes your whole system very clumsy.
     
    #21     Apr 5, 2015
  2. Sometimes there might be meaningless post.

    Let me know if anyone was successful, hopefully using R/Python. Will be paid!!!
     
    #22     Apr 5, 2015
  3. vicirek

    vicirek

    This statement is incorrect for technical and practical reasons. Just for the record so any unsuspecting reader would not take it as real information. Please do your research on the topic.
     
    #23     Apr 5, 2015
  4. Butterfly

    Butterfly

    what do you mean incorrect ? MoM for feeding quotes into your script is probably the worst way, even though a lot of people do it.

    You are welcome to elaborate if you feel you have a better technical reason for wanting a MoM for quote feed.
     
    #24     Apr 5, 2015
  5. Instead of those two, welcome to provide work to the public, probably at the web.

    If need, please estimate the labour since there i sno free lunch.

    But it should be a few hour of work in the weekend, not the hard and time-consuming work for one entire week.
     
    #25     Apr 5, 2015
  6. vicirek

    vicirek

    You talking about price feed which is continuous stream of data with variable number of messages per slice of time. For all practical purposes all communications occur using sockets - both Zeromq and Rest in this case. Zeromq and Rest are running on top of sockets adding extra layer of programming overhead and serving different purposes. Zeromq is a message broker which can buffer messages and is fast but still has some overhead (desirable or not - one can implement it themselves) over raw sockets. Rest has the overhead of the Web protocol (HTTP) which is much more overhead than needed for servicing data feed and has the disadvantage of two way interaction (request - response) that is not really needed for fast streaming data. I do not know if Zeromq is best or worst for data feed but certainly Rest which uses HTTP protocol GET and POST is not better suited for handling streaming data with variable and unpredictable number of messages per second. Do your research.
     
    #26     Apr 6, 2015
    eusdaiki likes this.

  7. Thanks but, due to my short knowledge on technical terminology, it will Not help meto understand.

    After 5 or ten years later I believe everyone will be familiar with those, similar to that TradingWorkStation like IB is well spread in the last 10 years.

    Although my area is close to computer, my time also valuable too, for learning the terminology. So if someone was successful to feed data, let borrow his time after pay with amount > my hourly wage.

    Furthermore my personal logic does NOT require real-time feed, but every one-minutes data for roughly 400 equities.

    Thanks again for Xerom and REST.
     
    #27     Apr 6, 2015
  8. Butterfly

    Butterfly

    nothing wrong with my research, maybe you should investigate more on your side.

    do you know the purpose of a MoM ? quote feeds are probably the worst application of such a technology. Query based tech are better fitted than MoM tech for such an application. As for overhead, I do not disagree, both are costly, but for most pre-trade applications that can kind of "high" cost is not that meaningful, so a REST query is perfectly acceptable and recommended to get quotes.

    If you want speed and no delay, then it's not MoM, but direct streaming of data through the network pipe, a listener on the client side, and a pipe stream of quotes being emitted from the server side. After all, quote feeds are nothing more than a continuous ticker that is "printed" as trades happens.

    Even a fast flat file ETL can do a better job than MoM. Actually, I think one infamous company is doing exactly that for the major Exchanges.
     
    #28     Apr 6, 2015
    eusdaiki likes this.
  9. vicirek

    vicirek

    zeromq was originally designed to handle financial market data and then the project was extended for other uses. It was highly optimized for finance in the first place. This is not true for Rest.
     
    #29     Apr 6, 2015
  10. Butterfly

    Butterfly

    highly optimized for finance ? what the heck does that mean ? do you mean market data ? if it was optimized for market data, it's still irrelevant. It's like saying a screw was optimized for a hammer. MQ is not the best tool for quote feeds.

    again, using a stack to store information sequentially is not the best way to access that information effectively. And since it has an high overhead cost, it's hilarious to see some are still not able to figure it out that MoM is not fitted for that job. Seems to be a lazy solution.

    REST is the future.
     
    #30     Apr 6, 2015