Well I have constructed 5min and 1min bars this weekend for 600+ stocks and found that I have a 97% match between IB TWS bars and those provided by my charting platform eSignal.
So for stocks NYSE/NASDAQ the data is good enough.
I wish people on this forum would give actual real statical data rather than just rant this is bad this is good e.t.c.
Give us some facts so that we know you have actually done back testing or do actually use these platforms or do have a programming background.
The HFT guys obviously cant use IB TWS as you need a platform that actually resides close to the mkt because of data latency.
And anyone who says back testing is a waste of time, how the hell do u test your code with real $$$ if so I know your talking BS.
* I have coded against IB, as well as directly to an exchange or two.
Currently, HFT is not so much about data latency as network latency. Location is about physical network length & # devices. That's how perverse it has become.. Any retail trader who thinks they can compete by running similar strategies from home is .. wrong.
As for IB, I see about 500ms between sending an order, and receiving an ack. That's a lot. And, considering that delay, and adding whatever their data latency delay is, I think we're talking up to 1 second in latency.
So .. that moves IB based strategies away from HF, and toward something else.
Given the history goes back to 2000 for many instruments, this is more history than most popular data providers offer. Do you know if there is an easy way to extract this data from files provided for use outside Trading Simulator platform?
IB provides individual back contract futures data for a year, more or less. To see what's available without having to remember the symbol exactly, put a current futures contract in the TWS main window, then select "Previous Expiry" and repeat a few times. Then select Chart and if the data is available it will plot.
Most decent commercial charting packages can retrieve the data. It is not continuous contract data. You have to link it.
Thanks for the pointer! I will give it a try. Maybe there is a mess-up in my charting software with contract codes, such as using 1-digit year instead of 2 digits... still works fine for the current month.
Edit: I just checked it and my favourite charting package is indeed able to extract data for expired futures contracts from IB when contracts are specified individually. Perhaps, there is a problem with requesting data for expired constracts in the "continuous contract" algorithm.
If you are relying on anybody's tick data to reconstruct bars then you are fooling yourself. I have used Bloomberg, IB, and several other data providers. They all have issues.
IB takes the best approach in that they provider 300 ms bars as that tends to filter the outlyer data. And if you can deal with 5 sec intervals you trade their 5 second interval bars, which reconstruct perfectly to the historical.
I agree there is no "ideal world". In particular, in an attempt to privide higher-quality historical data many providers "scrub" both historical "tick" and "bar" data in addition to filtering they do in real time.
The difference is just in how often there is a discrepancy... and with IB the problem is more severe than witha few other data providers:
1) IB real-time data is "sliced". The last trade price is sent 3 times a second or so as opposed to every tick. So, bars constructed from real time data have incorrect high and lows very often.
2) IB real-time data lack time stamp. As result, beginning/end of bar depends on how long it takes the data make way from IB to the trader's computer and on the trader's computer clock. So, open and close of a bar is off on most bars during periods of higher trading activity.
IB made a reasonable trade-off to make sure they can deliver data that doesn't lag during spikes of market activity without investing in infrastructure required to deliver reliably every tick. Given the price you can't really complain.