i am currently coding my strategy using the interactive brokers API. All of my backtesting has been done using historical data. as a result my system does not perform nearly as well with live "request market data" data (the two sets of data, though similar, are NOT the same). problem is that IB apparently restricts access to historical data through the API to 60 requests in any 10 minute period. this restriction is playing havoc with my automation efforts. is there any way around this? can I pay for the privilege of being able to make unlimited historical data requests?
As someone NOT using IB but doing extensive backtesting with his own platform. What the f*** for do you need more than 60 requests per 10 minutes? Are you not smart enough to store the data? I have a whole cluster in the basement doing strategy validations based on some terabyte of historical tick data that we have, including full order book, and I do exactly ZERO historical requests during the day.
IB feed is good for order management but simply improper as a datafeed. Get IQfeed for a very good retail-level (reasonably priced) datafeed (and I would suggest a retail platform, NinjaTrader or MultiChart .NET so that your CME fees are waved). If you go with NinjaTrader, then Kinetick is a rebrand of IQfeed, just cheaper (I would advice to use the IQfeed data-connector with Kinetick, as the imbedded data-connector for Kinetick has issues under Internet disconnect reconnect scenarios).
thanks for the replies "nettecture", I have plenty of stored data but only up to the close of the previous day. I was talking about "today"'s data. obviously it can only be stored if requested in the first place. ideally id love to be able to make one historical data request every single second, just for the last second, for every stock I am monitoring, all day long. once again, the reason why I would need to do this is because the historical data I am getting is (slightly) different from the live data, having been averaged out in some way - and I have no idea how they arrive at this average, and neither does any member of staff so far...! you might think that the odd tick here and there shouldn't really make much difference in any case, but unfortunately it does and in all cases where there's a discrepancy, I prefer to use the "historical" data as this is what I have always used when I trade manually (constantly refreshing the chart as the day unfolds). dom993, thanks but again, what I have devised tends to work better using interactive brokers historical data then any other data, because this is precisely the data ive always used to develop my rules in the first place. so using a different data feed entirely would defeat the object, even if it is more "accurate"
Man, you have problems. Like failing to understand tha you never get tick data from IB to start with - only snapshots. And that you dont get all order book updates. SERIOUSLY: DUMP their data feed, get a proper one.
i do understand I am getting snapshots, of course. as far as i can remember they come in at a rate of one every 300 milliseconds or something like that. I specifically do NOT want tick by tick data. the snapshots tend to have the effect of ironing out "crazy" spikes, which is exactly what I need! this was never the issue. the issue is that these snapshots are themselves different depending on whether i am looking at a live chart updating on its own, or a historical chart afterwards, looking back at the same time period. both are "snapshots", but there are (apparently) 2 sets of snapshots, and it is the latter (historical, ie "refreshed" version), that i am after. sorry, im not trying to be awkward, forgive me if i have indeed misunderstood something or overlooked something obvious - which is certainly possible!
You need to start from a "pure" datafeed & historical database (all the ticks, always), then find out what kind of filtering you need to apply to it - for both backtesting & realtime.