Unreliable historical intraday data

Discussion in 'Data Sets and Feeds' started by I$land, Feb 24, 2006.

  1. It depends on what you want to do with the data. The API just gives you every quote/trade at every level of the book for the sources you subscribe to. It's completely up to you what you want to do from there.

    For me, I've done quite a bit of development but if you just want the latest best price then it would be trivial to code in about 3 or 4 lines..

    The API doesn't offer any historical info, but I don't believe that is something an API should offer anyway.. that is best handled by some more batch oriented process.

    Also, you have to construct the bars, etc yourself if thats what you want to do.

    The api is fairly elegant in its simplicity.. saves you the headache and overhead of developing to complicated APIs.
    #11     Feb 24, 2006
  2. ============

    I$ land;

    By the way I determined this also with ;
    ''getting my hands dirty with data'' also.

    There is another public company [besides exchanges ] that sells ''cleansed'' data.

    But not only is it unnecessarily expensive for individuals like me, would rather ,as my retired banker dad used to say,
    '' don't be afraid to get your hands dirty'' with the data.

    Can & have printed like 5 days intraday data/ save plenty of those weeks for years;
    HPQ printer helps, but after you do plenty of that ,
    cheaper & more helpful to ''get hands dirty with data''.

    So prefer to simply record candle chart [or bars] info with blue/black/red ink on white notebook paper.:cool:[Do it before end of day] A human brain with discretion can ''cleanse '' focused prices also:cool:
    #12     Feb 25, 2006
  3. Try these guys....

    Separately, they have uptick/downtick data as well.
    They'll give you a free sample also.
    Unfortunately, I've not had the time to test the integrity of this data source.
    #13     Feb 25, 2006
  4. tickdata.com is the best source for clean data that I have ever come across - the qcollector software wroks great - but the data it collects from (esignal or quote.com) leave alot to be desired - all the quotes might be there but the time stamps are off
    #14     Feb 26, 2006
  5. tomhaden


    #15     Feb 26, 2006
  6. The problem with spending all this money to buy all this scrubbed data for testing is that real world trading results are unlikely to resemble results obtained during backtesting. Market data is dirty. If the filtering is not fully disclosed and/or suitable to be applied to real time data then you may be setting yourself up for a lot of duplicated effort.
    #16     Feb 26, 2006
  7. I$land


    syswizard :

    Thank you for the link but I only trade stocks.

    trader-rachel :

    QCollector + ESignal : just made some tests and I must say the results are not good. There are a lot of missing bars. I made a test program and here is the output for RIMM (May 2005 to Feb. 2006, 5 min bars, reg. trading hours) :

    20051007 : BarCount = 46 Some bars are missing !
    20051021 : BarCount = 53 Some bars are missing !
    20051026 : BarCount = 65 Some bars are missing !
    20051125 : BarCount = 42 Some bars are missing !
    20051130 : BarCount = 48 Some bars are missing !

    In a day, there are 78 5-min bars during the regular trading hours. For RIMM, on the worse day (2005/11/25), there were only 42 bars !

    tomhaden :

    Data CD : great link, thanks ! I downloaded a sample from their site and I will test it soon to see if their data is reliable.

    originalskunk :

    Agreed, but what do you suggest we do ?


    Is anyone with TradeStation ? I would love to test their data.
    If someone sends me data (RIMM, PLAY, 5 min bars May 2005-Feb. 2006), I will post the results here.

    Why pay for historical data if it is not reliable ?
    Anyone working for ESignal or Prophet.net can explain this ?

    Thanks !

    #17     Feb 26, 2006
  8. Paccc


    Has anyone tested the historical data available from IB? I know they offer up to 6 months of 1sec data, but I have not checked to see how accurate or reliable it is.

    -- Paccc
    #18     Feb 27, 2006
  9. I$land


    Paccc :

    I didn't know they offered that. Is it an add-on service ?


    #19     Feb 27, 2006
  10. papace


    #20     Feb 27, 2006