IB TWS taking a lot of memory

Discussion in 'Trading Software' started by gmst, Dec 5, 2012.

  1. As far as I remember, and I used the gateway last time about 1 year ago, it refused a connection several times from an external app that used it, and when a disconnect happened it crashed several times (Memory Access Violation). It is possible that at the time I messed around with my VC redist or java installations, but I found that unlikely. I haven't tried using it on my current development setup on Win7 64bit, but those issues stayed in my memory so I just lean away from the gateway nowdays. I do a lot of development and experimenting so it's possible my own code that went to the API had issues at the time, but it's strange that it didn't trigger errors via TWS connection. I also remember that support by IB or the yahoo group was almost non-existant, and the general attitude is "figure out our own bugs by yourself" which is really frustrating.

    Besides, I like to have the TWS in case some direct intervention is needed which my software or any of the front-ends do not support. It's useful to have a final checkup on open positions which you can trust, visually.
     
    #41     Dec 7, 2012
  2. gmst

    gmst

    Good point - otherwise you get in the situation similar to knight capital. lol
     
    #42     Dec 7, 2012
  3. pfranz

    pfranz

    Braincell,
    I've been with IB for ten years (I operate almost every day),and I used the gateway as soon as I could. I've always used the oldest possible versions of their software,and while there were problems with TWS (mainly with charts), I had only 1 problem with gateway: it sometimes disconnected from IB when I sent an order (no crashes,anyway),then re-connected.
    After a bit of struggle (and tons of logs),Raymund was convinced to escalate that to the development team, and the problem was fixed in subsequent gateway versions.
    There are data problems (volume some times decreasing,for example),but that doesn't depend on the software running on customer's machine.
    I have to say that I got rid very early of the ActiveX: I took the java sources and built a TCP connection to the gateway the same way java API does. Maybe this adds stability to my system,as I eliminated a piece of software (the ActiveX) which could add errors.
    But that's just because I use Visual Basic 6,if you use java,C++ or C# you shouldn't have this problem.
    Bob111:
    Me too I disable every autoupdate system and do it manually,but I see that with OS patches I always end up applying most of them.My suggestion was to braincell,not to you.

    And in the rare cases I needed TWS,I closed my software,closed the gateway,opened TWS and connected with it.Never had troubles.

    A quick note: memory sweeping on windows minimize doesn't happen in Windows 98 and Windows 7,just in Windows 2000/XP.Don't know about Vista.
     
    #43     Dec 7, 2012
  4. regulus

    regulus

    I run TWS as an API for Multicharts.
    I noticed a massive increase of memory usage since TWS version 933.5.
    This wouldn't be an issue at all (if you have plenty of RAM), but unfortunately, when plotting some charts of IB tick data in Multicharts, memory consumption will permanently increase during operation, similar to a memory leak.
    With TWS 932.4 and older versions, I have just around 400 MB of memory usage.

    Can anyone reproduce this issue?
     
    #44     May 7, 2013
  5. pfranz

    pfranz

    does Multicharts impose a minimum version for TWS? I'm using 917.9 and my API complies with API 9.65, and I don't have any problem.
    I suppose you are using the Gateway and not the whole TWS.So,why did you upgrade to 933?Is there any benefit?
     
    #45     May 7, 2013
  6. regulus

    regulus

    Multicharts does not impose a minimum version of TWS, but IB does and as far as I know, you are forced to upgrade when old TWS versions are considered "obsolete" by IB. As of now, TWS 932 still works, but presumably it will become obsolete in some months.

    IB Gateway is affected by the issue as well - since version 933, same as TWS. (I use TWS instead of IB Gateway, since it gives an extra level of control)

    Interestingly, the issue does not occur with Forex and Spot Metals, but only with Futures and ETFs.

    I wouldn't mind the issue at all if it was just high memory consumption (say several GB), as my rig has plenty of RAM. So I increased Java heap size to 5 GB for testing purposes, but then memory usage will rise up to several GB and finally TWS will freeze after some hours of operation.
     
    #46     May 7, 2013
  7. pfranz

    pfranz

    current minimum tws version imposed by ib has been for a long time 917.9.I'd advise to use this version (I'm still successfully using it) and use gateway instead of tws,because it not only reduces memory usage,but also latency(I've measured it against tws latency).
    IB changes minimum version disregarding how much time passes:it depends on problems arising and server updates (they changed minimum version many times in few months and left this for more than a year)
    I don't remember having ever needed tws extra level of control more than once or twice in years,and when it happened I loaded it manually.
     
    #47     May 8, 2013
  8. regulus

    regulus

    The oldest TWS I have available is 924. Does TWS 917.9 have any specific advantages over the more recent versions up to TWS 932 (e.g. TWS 924)?

    I think even if TWS 932 - which is the newest TWS that runs stable on my end - will be supported for several month (or even years) to come, it would make sense to make IB aware of this memory leak issue in due time, that's why I asked if anyone has experienced the same.
     
    #48     May 8, 2013
  9. pfranz

    pfranz

    I don't know, but I can tell that every time I tried a newer version of Gateway,its performance was the same or worse(increased memory consumption and latency),without interesting capabilities added.
    In particular,I found no Gateway version after 917.9 which gave me any advantage.
    As to the memory leak,it could be due to the combination of new TWS version + Multicharts: did you asked Multicharts developers?Best thing would be that they solved the problem together with IB.
    In the meantime,you could try another software to see if the problem is still the same.
    If I have time,I'll try with mine.
     
    #49     May 8, 2013
  10. pfranz

    pfranz

    I give you anyway my current numbers with 917.9,then I'll try with 933.5.
    Operating system is Windows 7.
    Getting historical data for 13 securities.

    Gateway memory consumption: flows around 85 MB
    My application memory consumption: 30MB
     
    #50     May 8, 2013