Interactive Brokers - IB _Gateway_ memory leak when subscribing to options

Hi folks,

I reported the following memory leak to IB API team sometime back, and their developers acknowledged it. But after following up with them a few times since early January, I was disappointed that they are still "investigating".

I am posting it here, for one because I have seen the other issues that are reported here actually got addressed, so I am hoping to somehow get lucky with this issue as well. Also there must be other options traders who is in the same boat and found a workaround?

It is naturally frustrating for me since I do need to rely on the IB platform at this point. IB is far from being rock solid itself but I have been able to build a robust system that has been running around the clock for months without a single issue, while fully subscribed to 200 most liquid futures and stocks symbols. So long as I don't touch options!

/end rant

So here goes:


=== My initial bug report in January ===========================
Hi,

I have a API client that subscribes to 90+ option contracts at the beginning of each trading day. (by calling reqMktData) The API client connects to an IB Gateway instance.

I have been testing this client for a few weeks, and the IB Gateway process is very prone to out of memory exceptions, even after I assigned 1GB of memory to the gateway JVM I could not prevent such OOM exceptions. I think this has to do with the option models running inside the IBGateway.

I will attach the IBGateway JVM stacktrace for your investigation, but I would like to turn off the option model computation via code or configuration. Please let me know how I can achieve this.

regards,
XXXXX

java.lang.OutOfMemoryError: Java heap space
at javax.swing.text.GapContent.allocateArray(GapContent.java:77)
at javax.swing.text.GapVector.resize(GapVector.java:197)
at javax.swing.text.GapVector.shiftEnd(GapVector.java:212)
at javax.swing.text.GapContent.shiftEnd(GapContent.java:328)
at javax.swing.text.GapVector.open(GapVector.java:184)
at javax.swing.text.GapVector.replace(GapVector.java:125)
at javax.swing.text.GapContent.insertString(GapContent.java:115)
at javax.swing.text.AbstractDocument.handleInsertString(AbstractDocument.java:709)
at javax.swing.text.AbstractDocument.insertString(AbstractDocument.java:693)
at javax.swing.text.PlainDocument.insertString(PlainDocument.java:114)
at javax.swing.JTextArea.append(JTextArea.java:470)
at ibgateway.zb.a(Unknown Source)
at jutils.dd.a(dd.java:111)
at jutils.dd.a(dd.java:101)
at jextend.ge.a(ge.java:1369)
at jextend.ge.W(ge.java:1355)
at jextend.ge.a(ge.java:2368)
at jextend.p.stateChanged(p.java:500)
at jutils.yf.a(yf.java:58)
at jutils.yf.a(yf.java:52)
at optionmodel.cb.run(cb.java:718)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:619)


====== reply from API support =====

On Jan 13, 2011 10:59 AM, <api@interactivebrokers.com> wrote:


Hello XXX,

We are currently investigating this type of issue. I've believed that the Ib Gateway is getting some memory leak.

Regards,
XXXX
IB API Support


====== sometime later ==============
Date: Fri, 25 Feb 2011 11:55:21 -0500
From: api@interactivebrokers.com

Hello XXXX,

The development team is still investigating this issue.

Regards,
XXXXX
IB API Support

 

After a couple more months they will tell you to exit TWS Gateway and restart it every day like they do their servers. IB has been told about this memory leak with TWS Gateway before (here - http://www.elitetrader.com/vb/showt...9705&highlight=tws+gateway+memory#post3039705) and do not seem to care. They seem to think that only being able to keep an app or service running for only a day or so uninterrupted is normal behavior unfortunately. // end rant 2

 

I am not really asking a lot. Either

1) Fix the leak - not really hard since I have already pin-pointed the cause and showed them how to reproduce. A few heap dumps and a few hours testing should suffice. (assuming any competent developer at the task)


2) OR just allow the users to turn off the option pricing models by configuration. I have no faith in IB to get the pricings right anyway, and their methodologies are not documented.

I have tried to maximize the JVM heap settings and still was not able to make it through one day without bailing out. I don't really mind restarting my gateway instance after close.


ps what you posted seem to be a different issue. A memory _leak_ as I have reported cannot be cured by configuration, but delayed garbage collection in JVM (link above) can be cured by fine-tuning. but Anyways. I still dream of being able to trade options with my ATS but that hope has faded now.

 

IB is getting on my nerves too.

What would be the top alternatives to IB at the moment? API support, comparable commissions, better trading platform for automated trading.

 

It seems there aren't any.

Just restart the program once a day.

 

The new beta version of IB Gateway includes memory optimization enhancements. Please PM me if you still experience a memory leak problem.

The beta Gateway can be accessed here:

http://www.interactivebrokers.com/java/classes/ibg.beta.jnlp

 

Thanks. I will try this out over the weekend and will update.

 

This does not fix anything. I know the first thing to try is always to try to point users to a latest and greatest version, but this has to be taken up with the developers. Thanks for your reply though, I really appreciate it. Please take the attached log files and let the developers reproduce and fix the leak.

Specifically, do the following:

1. Have a IB gateway client connect to the gateway.

2. Have the client subscribe for (reqMktData) for the most ATM options (say the 8 strikes on either side) for the first a few options expiry, on SPY.

3. Let the client run, it will receive not only bid/ask/last, but also the greeks.

4. Eventually (less than a trading day) the Gateway itself goes out of memory, or some other sort of error. (see log file)

I have other use cases that proves to me that only with the options I have such problems. How do we turn off the leaking options models? This is a case where I don't even need you to fix it immediately, but provide a work around by letting the using turn OFF the options models. (since the IB gateway/TWS can't do it without a leak)


Please see the log file attached. (of the gateway process)

 

Oops, log file was too big, i had to truncate it. Attached below:

 

The new beta version of IB Gateway does includes memory optimization enhancements, but we have not been able to reproduce your issue BulletTooth. Please check your PMs.