A question for those people who have written their own back testing frameworks and automate though IB, do you use the IB historical data to backtest or do you source tick data from somewhere else? If you source other data do you not run the risk of optimizing systems which may exploit data quirks which may not exist in the live IB data?
Hi there, I just use IB as an execution route for my systems - all my backtest/live data comes from DTN. About the 'data quirks' - unless your system is tick based, I dont think you need to worry about that. I hope that helps -MrTwo
For me it comes to around USD100 per month. I decided to use DTN instead of IB because their data is tick-by-tick while IB's data isnt. Also, they have more historic data available than IB. Good luck MrTwo
I think you're on the right track. My backtesting framework is written using IB's api and my approach is to save IB's market data exactly as it came in during live trading. I do nothing to "sanitize" the data and even insert tokens in the data stream as I save it to indicate when the server goes down and back up again. Recording all the error conditions that can occur along with the data helps minimize surprises when going live because you can "replay" a session where something unexpected happened over and over and fix the system to handle it. Sanitized historical data is fine for proving you're a math genius to your college prof but running a strategy based on that stuff is risky business. The real world is not "sanitized".
I have a number of systems that are automated through IB. I am currently using their intraday data and have not really had problems with it. Most of my backtesting has been done with daily data from TC2000. The rules for my systems are very simple (so far) so I don't really need tick data or to be able to produce intraday charts for analysis. I plan on backtesting on intraday data soon but IB just does not have enough history for me to even bother with. I have looked at OpenTick but have had a few problems with it so I am not sure it is going to work either. mrtwo - how much intraday data is available from DTN and what is there interface like?
I have had several connection issues. Servers unavailable, no response, empty response, etc. This is when requesting historical data during non market hours (and some times very late at night). Also, data does not match with other providers, including web quotes directly from the exchanges. I posted about this on the forums but never got a response. The forums do not see much activity at all so I don't think there is any good way to get additional information/help/support. I have never tried real time market data so I can't speak to that. On the positive side I did find the Java implementation easy to work with. I was able to register, download the api and get a working test running relatively quickly. The data is free so it is hard to complain, even with issues. I have not completely given up and may try to work with it some more if I can find the time.