Performance problem with .NET Interop

Discussion in 'Trading Software' started by mottapd, Dec 21, 2009.

  1. mottapd


    Hello guys

    I´m trying to make a system, in C# .NET, that distribute market data to Excel sheets using RTD function.
    But I´m having performance problems to fire events to RTD.

    When i fire about 900 events per second, the processor usage go to 100%, and this is very strange

    This the COM Interop Interface code:

    public interface BovespaEvents
    void UpdateNotify(string strategyname, string symbolname, string fieldname, float fieldvalue);

    This is the method that I use the COM Interop Interface:

    public void UpdateEvent(DataObject sender)
    UpdateNotify(sender.strategy, sender.symbol, sender.field, sender.value);
    catch (Exception err)
    LogError("UpdateEvent", err.Message);
    if (err.InnerException != null)
    LogError("UpdateEvent", err.InnerException.Message);

    And this is the DataObject class:

    public class DataObject
    public string strategy;
    public string symbol;
    public string field;
    public float value;


    And I make an VB6 to consume this events, it could be a RTD

    Private Sub myInteropClient_UpdateNotify(ByVal strategyname As String, ByVal symbolname As String, ByVal fieldname As String, ByVal fieldvalue As Single)
    ' code here
    End Sub

    I´m stuck in this performance problem. Does anybody already see this problem before ?

    Thanks Guys
  2. Aitch Eff Tee

    Aitch Eff Tee Guest

    You are making an interop call to Excel (which is notoriously slow) roughly every millisecond. There's your answer in a nutshell.

    Excel interop is so bad I wouldn't be surprised if all method calls are routed through that fucking animated paperclip.
  3. wave


    As fugly as it is to program in, c++ will always be snappier.
  4. True of course, the .NET interop layer adds overhead in addition to the COM interface's overhead. Question is, if the op will gain much if it runs maybe 2 to 3 times faster at best. He will still have to cross the COM barrier and that's inherently costly.

    Maybe using the Excel C API would be a better choice in this case.

    But i doubt Excel is the way to go here anyways. Its a great product for what it was designed to do, its just that pumping a few thousand quotes and doing rt calculations + gui updates is not part of that.
  5. Aitch Eff Tee

    Aitch Eff Tee Guest

    Indeed. Like all things it's a case of horses for courses and I too doubt this is the right horse for this race.
  6. That last part is epic. I laughed out loud for quite a while..
  7. Really need to pay attention to how you are moving data. 900 single point updates per second versus an array passing 900 data points. The Data Array will be hard to beat. The paperclip guy prefers arrays ;)

    Attached is a screen shot of one of our Trading Bots using OEC's DDE / .net interop API. DDE updates are typically in microseconds... But how you handle and process the data will make or break you.

    Below are some comments from CogentRTS.

    We have only done some initial testing with RTD, but we have quite a lot more experience working with DDE, and with comments from our customers we have come up with the following thoughts.

    The bottom line comparison between RTD and DDE is that DDE is faster for single-point changes, because it has lower transmission overhead. RTD can group many point changes into a single message, so it overtakes DDE if there are many point changes happening at once. But DDE can send arrays, which are equivalent to large numbers of points grouped at once, thereby making it perform better than RTD again.

    If you design your system to make heavy use of arrays, DDE is likely to perform better. If you design your system to use large numbers of individual points, RTD will perform better.

    Tunneling (networking) DDE arrays, using the DataHub is a very efficient means of moving large amounts of data across a network. It will perform at least as well as RTD because it has lower overall transmission overhead.

    The design of the application will determine which is faster - it's not a slam dunk for either RTD or DDE. You can create grossly inefficient systems with either transmission protocol, and reasonably efficient applications with either as well.

    We don't support RTD because at the end of the day our customers really want to use arrays, so they end up going with DDE anyway.

    We had a customer who was moving away from RTD, but this needs to be explained carefully. The customer in questions is transmitting a large number of points into Excel from at least two sources, one using DDE and one using RTD. Excel seems to favour RTD updates, so if the RTD stream is exhausting the system resources then DDE is starved. That's simply an issue with Excel and the computer they are running on.

    Our impression from talking to past customers is that nearly all of them are using DDE exclusively and none report any problems with speed or reliability.
  8. +1.
  9. mottapd


    Traders likes to use excel to write their own strategies in VBA

    Becouse VBA is easy...

    I can't change this.

    There is another way without use Excel ?