IB TWS taking a lot of memory

pfranz,
maybe the issue occurs only for the specific combination of Multicharts and TWS/Gateway 933+?
But that's pure speculation, I didn't try any other API client than Multicharts yet.

 

Just to be precise, here are the switches I use:
-Dsun.java2d.noddraw=true
-Dsun.java2d.opengl=false
-Dsun.java2d.pmoffscreen=false
-Xmx96M

 

The verbose:gc switch is redundant so leave that out if it helps it fit. You can probably omit -XX:+PrintGCTimeStamps too, as I think it's turned on by default with Xloggc, but not 100%.

For the classpath on windows, just use -cp followed by a semicolon separated list of jars

 

Below is the heap usage log of TWS 938.1f on Win 7 64 bit with Java 1.7.0_21, API client is Multicharts 64. TWS froze after 3 hrs.

The log shows that heap usage is constantly growing when TWS runs.

I used the following parameters for TWS:

C:\Windows\System32\javaw.exe -cp jts.jar;total.2012.jar -Dsun.java2d.noddraw=true -Dswing.boldMetal=false -Dsun.locale.formatasdefault=true -Xloggc:mygc.log -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintHeapAtGC -Xmx1G -XX:MaxPermSize=128M jclient/LoginFrame C:\Jts

Any suggestions what I should try next to identify the issue are welcome.

 

This is indeed troubling. It is freezing because the JVM is eventually spending all of it's time doing GC to try to stay below the -Xmx1G limit. You can see this in the purple "Other full GC" entries. Presumably a larger maximum heap size just delays the time taken to reach this point.

I know you prefer TWS, but for completeness, you should run the same test with the Gateway.

I don't know anything about Multicharts, but I presume one of the main differences between our setups (assuming this happens on the Gateway as well) is that yours uses the historical data API, while I only stream data/submit orders. One might therefore hypothesize that a memory leak has been introduced somewhere in this area.

There are further things that can be done to narrow down where this is occurring, but it's not really your job to be doing IB's testing for them. I'd repeat the test with the Gateway, submit all of the results to IB and demand they investigate.

 

Below is the heap usage for IB Gateway 938.1f (Java version is 1.7.0_21 64 bit and OS is Windows 7 64 bit).

The log shows a very similar behavior compared to TWS. After around 8.000 seconds of operation, when the -Xmx1G limit is reached, the Gateway freezes.

Start parameters are:
C:\Windows\System32\javaw.exe -cp jts.jar;hsqldb.jar;jcommon-1.0.12.jar;jhall.jar;other.jar;rss.jar -Dsun.java2d.noddraw=true -Xloggc:mygc.log -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintHeapAtGC -Xmx1G -XX:MaxPermSize=128M ibgateway.GWClient C:\Jts

For reasons of completeness, I will post a log with 932 in order to demonstrate that this version is not affected by the memory leak.

 

It's interesting that it's happening well before hitting 1G. Could you post the log itself?

Perhaps the TWS code is getting a little overzealous in it's memory micromanagement. For kicks, try the following:

add:
-XX:+DisableExplicitGC
-XX:+UseConcMarkSweepGC
-XX:+CMSClassUnloadingEnabled
-XXermSize=256m

change:
-XX:MaxPermSize=256m

 

ddecker,
below is the log for IB Gateway 938.1f.

I'll have another test with the parameters you suggested.

 

Files size of the log file was too big. Another try with zip:

 

And here is the heap usage for Gateway 932.4, for completeness.

With this version, Java garbage collection works as it should and no freeze occurs.