IB TWS: slow!, overloaded?

Discussion in 'Trading Software' started by Canoe007, Jan 19, 2011.

  1. Canoe007

    Canoe007

    With the stock command line:

    Moving one of the tri-chart chart windows from a monitor on the Nvidia 430 to one on the 8600GT. Chart updating appears the same, but there are huge delays in functionality. Trying to click and drag an order on the chart but the line won't be selected. Move mouse to a button to click to cancel order but the mouse arrow is over the button but now showing the up/down arrows of order line selection - another two seconds for the single mouse arrow to return and the button to be clickable.

    Very scary.

    Very counter to what we've believed about graphic cards for trading.


    p.s.
    2x 2G DDR2 800 5-5-5-18
    XP Pro SP3
     
    #21     Jan 20, 2011
  2. ok, only 5k for my perfectly working dual monitor system
     
    #22     Jan 20, 2011
  3. This maximum heap size option is probably the most important option for Java performance. If set too low, the Java run time will thrash around doing garbage collection looking for free memory in the heap and this will kill performance.

    You can if you like monitor heap usage graphically with the jconsole tool. You need to download and install the JDK. It is easy to do, educational and gives some insight to what is going on.

    Just add the option:

    Code:
    -Dcom.sun.management.jmxremote
    
    to the command line for running TWS and then run the jconsole tool. jconsole also reports on time spent doing garbage collection and lots of other stuff.
     
    #23     Jan 20, 2011
  4. The 430 is a very new Nvidia card and does DX11 (I think). There may be some issues with mixing it with the 8600GT that are being exposed by the way Java does it's rendering. I'm not really across all this Windows graphics stuff (I use Linux only) but it may be worth asking a question on the Nvidia support forum and emphasize that it a Java application that is causing the issue.

    http://www.nvnews.net/vbulletin/index.php

    Alternatively you could just buy another 430 as they are not very expensive and the problem may very well just go away.
     
    #24     Jan 20, 2011
  5. One crazy thought:

    If one of the cards is on PCI, it will be *extremely* slow. (I've tried it!).

    PCI simply cannot handle modern graphics load, especially if working alongside a PCIExpress card.

    I've seen 10x performance loss with PCI graphics on Windows 7 including disk IO, graphics and everything else.
     
    #25     Jan 20, 2011
  6. LeeD

    LeeD

    Have you tried disabling Aero?
     
    #26     Jan 21, 2011
  7. Canoe007

    Canoe007

    I did find a post concerning "some items not rendering when NVidia drivers were used" was solved by turning off OpenGL from the command line. Last evening, I got an improvement in performance, but that wasn't duplicated this morning under market data loads.

    p.s.
    Both NVidia cards are PCI-E.
     
    #27     Jan 21, 2011
  8. Canoe007

    Canoe007

    Sounds like the way to go to see what's going on. I already have the JDK from using Eclipse for developing my GUI in Java for IB's API, but I didn't get anywhere near learning as much about Java as I'm going to need to (order management is up next), and nothing about running it. From within Eclipse, the GUI runs amazingly fast, and that's with everything else running on my system.

    First is to figure out how to get TWS running with all of the Java recommended & IB selected parameters. Fortunately I know exactly who to throw that question at. ;-)

    I must be a glutton for punishment. In addition to sorting out a TWS configuration that works for me, running Java, Java development for multi-threaded Order Management IB's API, I'm about to pull the cord on building my next box (W3520 on i7 1366, 12G 1600 L7-8-7-20, PCI-E SSD, 3x Nvidia or 1x Eyefinity 6), upgraded before too long with a duplicate redundant system only with a 970, and it will be Fedora KVM hosting guests of: Fedora, two Win7-64 (one solely for trading) and a WinXP-32.
     
    #28     Jan 21, 2011
  9. Canoe007

    Canoe007

    Got some interesting observations to note.
    Very interestingly, it brings to question the common wisdom that a minimal graphics card on a lower-end box is sufficient.
    Firstly, these observations are when I'm running what appears to be larger than normal load of charting than most TWS users appear to use, but they're very intriguing.

    With my previously stated TWS windows open, by bringing -Xmx512M up to 1024, response was noticeably improved. With 4G on the MB it wouldn't load with a higher value, even though there was more memory available.

    Observations (made with market open, for data load):
    • changes in charting where reflected primarily in the usage of one of the four cores, which was tasked significantly more than the others
    • dragging & dropping one chart window to overlap another resulted in a large increase in the load on that core; removing that overlap resulted in the additional load gone
    • closing a chart window lowered the load on that core
    • minimizing a chart window lowered the load on that core by roughly the same amount!!!
    • minimizing each chart window showed a drop in that core, and with all minimized resulted a minimal load on that core, down to 3% to 6%
    • restoring a chart window would immediately present that chart back to its monitor, and it is completely up-to-date without any hesitation or lag!!!
    • additionally, running a chart window on a monitor on an older graphic card required noticeably more CPU
    From this, I conclude that the CPU load to process the data and know what the chart should look like is minimal. Actually painting that on a monitor requires a meaningful CPU load that is proportional to the complexity of the chart. Under WinXP, running on a newer graphic card uses noticeably less CPU but not overly significant given the huge load on that core.

    It also gave me suspicions regarding TWS multi-threading, but subsequent tests muddy those waters.
     
    #29     Feb 1, 2011
  10. Canoe007

    Canoe007

    With the box I had, running the TWS windows to present the charting data I'm accustomed to, TWS response was initially o.k. with the 1024 Java Heap parameter. The longer the platform was up and running from Market Open, the slower the response of the platform became. By mid afternoon, when clicking on a price in Book Trader it would take SIX SECONDS for the Transmit button to pop up. Lag in an order from a Chart Trader button populating tables would take seconds, with additional fewer seconds required before it appeared on charts. Trying to adjust parameters of the platform would often open a configuration window, but items were not selectable.

    This is not tradeable.

    Closing TWS and restarting it would resolve these issues, but response was just restored to its original slowness. (reminder - I'm running a lot of charts on four monitors)

    So, I decided to try Win7. With the same hardware, and a spare hard drive of the identical model/size, I loaded the new OS. In getting 6.23 Java, it noted that while I was asking for the 64 bit version, it noticed I was using a 32 bit browser, so it suggested getting both 32 and 64 bit Java so Java in the browser would work. I installed the 32 bit java, then the 64 bit java. Then I downloaded and installed TWS 913.6, same as I was running before. Also installed the same anti-virus (avast) and Firefox.

    TWS loaded right away and I was able to sign in first time. Rather neat to see my charts all open up on the new OS! ;-)

    I was able to make copies of the shortcut that launches TWS, with the Java Heap set to 1024 and the other to 2048. TWS launches and runs with both, with no difference that I can note. I have no idea which Java engine TWS is running with, be it 32 or 64 bit.

    Core usage has changed. While one core is significantly loaded more than the others, the difference is far less and all cores have less load, almost half of what they had under WinXP. The character of the cores' load is different: the load is more sparse, in that the load has way more time at the lower end of it's range between spikes - the average is down. Overlapping one chart with another results in no noticeable change in the CPU load. Minimizing a chart reduces the one core load, but that is less significant as that core's max was down so much from under WinXP. The box produces less heat...

    Circling the mouse still pops the one core, but to 60%.

    How does TWS run? Response of TWS is faster at everything! It runs much better. Lag between charts of the same security at different time frames is much less. Under a half second. Chart Trader button orders populate tables in under a second with another half second to appear on the charts. Multiple button clicks queue orders and there's an additional lag while it catches up, but not the huge delays like under WinXP.

    Unfortunately, by 11:00, response to Chart Trader buttons is starting to lag. By 1:00, is is so significant that it feels it would be unsafe to trade.

    On to next solution.


    edited: I believe that the difference is largely due to an entirely different type of graphic driver under Win7, with much better use of them being made by either Java or Java's calls to Win7. Don't know enough technically...
     
    #30     Feb 1, 2011