Quality of IB backfill/HistoricalData?

Discussion in 'Data Sets and Feeds' started by twinx, Jun 26, 2006.

  1. twinx

    twinx

    There are bad votes about the quality of IB Backfill data.

    1.) that there should a lot absence-ticks...
    2.) or generally no Backfill

    What are your experiences,...?

    re 2.) From a thread of IB-API forum:
    IB responded offline that the requestHistoryData request will not return data from expired future contracts. IB provides 6 months of future data but only from the current contract.
    Is this still the fact, no data from expired future contracts?

    What are your experiences, there are a lot of third-party software which use the IB backfill/HistoricalData from IB, so the data should be not so bad? What about there should a lot absence-ticks...?

    Thank you very much for unswers!
     
  2. It is not tick data - it is OHLCV time bars. In my experience it is accurate.

    I've found the backfill is be fairly reliable. The API is a little clumsy, but it works reliably.

    To my knowledge there is as yet no historical data for expired contracts. A couple of posts on another thread have requested this.
     
  3. If you need data for futures only, it's accurate. I only d/l minute data, so I have no comment on tick.

    As far as stock, it sux big time. There are just too many spikes here and there make data unusable for anything.
     
  4. i have been looking at the SPY backfill data (1-minute) and agree that the quality is really poor...lots of out-of-market spikes renders the data almost useless..i dont know why its like that
     
  5. I think it is crap. I was using it with Sierra and while gone on vacation for 2 weeks I missed all of that data for the Euro, but had it for the Canadian? Go figure, the settings were exactly the same. Since it was gone of their server I could not get it back. I will close my account with IB. Cheap reliable data was one of the reasons I went there, not very reliable IMO.

    Dan
     
  6. Calip, how have you verified that the IB 1-min backfill data is accurate for futures? Did you do it just by visual inspection or did you compare it to another backfill source? Please elaborate..it would be great to know for sure that futures at least is accurate
     
  7. twinx

    twinx

    Hello,
    can we try to compare some data?
    For example an:
    1.) IB-tick realtime recorded Times&Sales" to a
    2.) backfill IB-tick "Times&Sales"?

    So, than we could look a little bit the differences between backfill and realtime data.

    I have not the instruments for, but did an "example" with IBCharts "Times&Sales" and QuoteTracker "Times&Sales"....
    Code:
    DAX IBCharts				DAX QuoteTracker					
    Price	Size	Type	Time		Time		Price	BID	ASK	Size	Date
    
    5546	1	Ask	13:30:44	13:30:34	5545.5	5545.5	5546	1	06/27/06
    5545.5	1	Bid	13:30:29	13:30:33	5546	5545.5	5546	0	06/27/06
    5546	1	Ask	13:30:22	13:30:30	5546	5545.5	5546	1	06/27/06
    5546	3	Inside	13:30:06	13:30:29	5546	5545.5	5546	1	06/27/06
    5546	1	Ask	13:30:06	13:30:26	5546	5545.5	5546	1	06/27/06
    5546	5	Ask	13:29:58	13:30:18	5546	5545.5	5546	0	06/27/06
    5546	2	Bid	13:29:57	13:30:15	5546	5545.5	5546	0	06/27/06
    5546.5	2	Ask	13:29:56	13:30:12	5546	5545.5	5546.5	0	06/27/06
    5546.5	2	Ask	13:29:56	13:30:12	5546	5545.5	5546.5	3	06/27/06
    5546	1	Ask	13:29:52	13:30:11	5546	5546	5546.5	1	06/27/06
    5545.5	1	Bid	13:29:50	13:30:06	5546	5545.5	5546	0	06/27/06
    5545.5	1	Bid	13:29:50	13:30:03	5546	5545.5	5546	5	06/27/06
    5546	5	Ask	13:29:49	13:30:01	5546	5546	5546.5	0	06/27/06
    5546	5	Ask	13:29:49	13:30:01	5546.5	5546	5546.5	2	06/27/06
    5545.5	1	Bid	13:29:35	13:30:01	5546	5545.5	5546.5	2	06/27/06
    5546	1	Ask	13:29:34	13:29:57	5546	5545.5	5546.5	1	06/27/06
    5545.5	1	Inside	13:29:31	13:29:55	5545.5	5545.5	5546	0	06/27/06
    5545.5	2	Ask	13:29:30	13:29:54	5546	5545.5	5546	5	06/27/06
    5545.5	1	Ask	13:29:29	13:29:54	5545.5	5545	5546	1	06/27/06
    5545	1	Ask	13:29:15	13:29:50	5545.5	5545.5	5546	0	06/27/06
    5545	8	Bid	13:29:10	13:29:48	5545.5	5545	5546	0	06/27/06
    5545	2	Ask	13:28:54	13:29:41	5545.5	5545	5546	1	06/27/06
    5545	4	Ask	13:28:52	13:29:40	5545.5	5545	5546	1	06/27/06
    5545	46	Bid	13:28:51	13:29:39	5546	5545.5	5546	1	06/27/06
    5545	5	Bid	13:28:39	13:29:37	5545.5	5545.5	5546	0	06/27/06
    5545	5	Bid	13:28:39	13:29:36	5545.5	5545	5546	1	06/27/06
    5545.5	1	Ask	13:28:37	13:29:35	5545.5	5545	5546	0	06/27/06
    5545	1	Bid	13:28:34	13:29:34	5545.5	5545	5545.5	1	06/27/06
    5545	1	Bid	13:28:34	13:29:34	5545.5	5545	5545.5	2	06/27/06
    5545.5	7	Bid	13:28:23	13:29:33	5545	5545	5545.5	0	06/27/06
    5545.5	7	Bid	13:28:23	13:29:19	5545	5544.5	5545.5	1	06/27/06
    5546	1	Bid	13:28:16	13:29:15	5545	5544.5	5545	0	06/27/06
    5546	2	Ask	13:27:59	13:29:14	5545	5545	5545.5	8	06/27/06
    5546	4	Ask	13:27:57	13:29:05	5545	5545	5545.5	0	06/27/06
    5546	7	Bid	13:27:56	13:29:00	5545	5544.5	5545	2	06/27/06
    5546	2	Bid	13:27:56	13:29:00	5545	5544.5	5545.5	0	06/27/06
    5546	1	Bid	13:27:53	13:29:00	5545	5545	5545.5	0	06/27/06
    5546	2	Bid	13:27:47	13:28:59	5545	5544.5	5545	2	06/27/06
    5546	1	Bid	13:27:44	13:28:57	5545	5544.5	5545	4	06/27/06