Problems with Interactive Brokers Forex data If anyone has any comments, idea or suggestions on this problem your help would be appreciated. Goal: Get Forex price data from Interactive Brokers TWS using jTWSdump for download into an ATS for trading 15 to 60 minute bars. Results: jTWSdump works fine. I get my files downloaded in about 5 seconds. However there is never any data for the time period 16:00 to 16:14 as shown below. This is consistent day after day. 20081030 15:57:00 127.44 127.44 127.41 127.41 20081030 15:58:00 127.41 127.41 127.265 127.285 20081030 15:59:00 127.285 127.345 127.22 127.345 20081030 16:15:00 127.245 127.295 127.2 127.29 20081030 16:16:00 127.29 127.29 127.25 127.27 20081030 16:17:00 127.27 127.375 127.27 127.345 20081030 16:18:00 127.345 127.365 127.34 127.34 Are other people using different software for automated data acquisition having the same problem? If so then the problem is with IB. If not then it would be a JTWSdump problem. Can anyone suggest other data acquisition packages that work with Interactive Brokers TWS? Thanks, Jerry
IB data has too many gaps throughout the day for an effective source for your ATS. Look at ESignal. Integrates directly into the TWS - IB has an option built right in.
Sstfx, Thank you very much for the insight. How in the hell can people use if for auto trading? This is mind blowing! My system uses a Neural Network trained on historical prices to make all trading decisions. So missing data on a regular basis is a fatal problem in that the NN would assume a price pattern created by the missing data that does not exist in the real data. I appreciate your response! Jerry
From the timestamps, I deduce that you are in Central time zone. IB forex shuts down every day at 17:00 Eastern time, and doesn't start up until 15 minutes later. That, of course, is the reason you are missing data. The missing data shouldn't be a problem, because you can't trade during those times anyway. You will find that nearly all forex brokers are shut down during approximately the same time period, so it doesn't help to try other venues. Even those who have quotes typically have very little liquidity. IB's data is really pretty good, for a retail broker. ESignal forex data has too many other problems to recommend it.
That would explain it nicely since I'm Central and the gap at 16:00 is 5 PM Eastern every day. This is fine as I don't case about the market not trading just missing trades as it kills the pattern recognition in the models. I wonder why they don't shut down at midnight or some time before Asia opens when few would be interested in trading. Thanks for the insight. Jerry
Jerry030, if you are using sliding window averages in your model, then why not use a larger time frame, or maybe even different models for different time periods - looking at "mood" and volatility during the whole session and last hour, closing minutes etc. There are so many different possibilities, depending on what kind of model and trading target you are going for ... As you know, the market before, during and after US RTH is somewhat different - as is European RTH and Asian RTH.
In Cash FX 5pm NY time has traditionally been the time when the FX "Day" comes to a close and all open positions are rolled. Typically if a position is open after 2pm it is considered open and good to roll through till the next day. I used IB as well and found it kindof funny that the system is shut down for 15 minutes but thats just their way of doing things.... for what its worth your not missing anything during that time.
Thanks...this is starting to make sense. I have historical data from several sources where there are trades at this time, but if everybody sets thier own policy on this then it makes sense that some sources have a gap and some don't.
I use a "Russian Doll" model structure. The same model has data from multiple time frames from 1 min up to the bar length used to generate the tradiing signal (30 to 60 minutes). If any are missing then the whole thing dies. Trade duration is a max. of 2 hours.
I customized my charts to eliminate the 15 minutes of down time. This gave me a clean chart with IB and if I went anywhere else I wouldn't miss much of anything. If that works for you that's my suggestion.