what is the best language that suits massive calculation on every tick?

Discussion in 'Automated Trading' started by mcgene4xpro, Jan 23, 2011.

  1. nLepwa

    nLepwa

    There are several ways...

    Distribute the work, parametrize your strategy and don't give the parameters, have them make the platform flexible enough so that you can then implement the strategy (the easiest part),...

    Ninna
     
    #11     Jan 23, 2011
  2. i am trying this. However, it consumes huge efforts to conceal your logic. :D

    I think i would continue do this..

    This is the more practical way to do it..for my case.
     
    #12     Jan 23, 2011
  3. ramora

    ramora

    Anton, thank you for the benchmarks. Are you assuming the data in only going to RAM at those rates, or can you sustain those rates saving to a database? What is the best database to use with OpenQuant?

    Thanks again,
     
    #13     Jan 23, 2011
  4. One way to protect your intellectual property is to break the task into distinct pieces which are done by disconnected people. Let a professional build the interface to the platform around a bogus system which is constructed to have the same inputs.
    For example, if you want to do a pair-trading system on a tick-by-tick basis. Pay a programmer to setup a system to trade Bollinger Bands on the numerical ratio. Once you have the working platform and a working example, you can either transform the bogus system to your real trading logic yourself or again farm it out. Your real system could then be programmed in NinjaTrader based on the Last Price of the charted instrument. Once you have these two pieces, it should be easy to copy and paste them together and substitute ratio for Last Price.
     
    #14     Jan 23, 2011
  5. (1) If you have already a working strategy written in MQL4 you are already familiar with a programming language.

    Next step is to find one that is closest to the one you are using in order to reduce the time required for learning.

    (2) C++ is clearly the hardest prog. language to learn. No reason to go this way (speed is also ok with other languages). Visual C# would be a good choice.

    (3) How could you waste your time? How could there appear a limitation in the programming language?


    Keep in mind: If your strategy is too slow in a language like MQL4 that means nothing about speed in a true programming language.
    Some computations that take ages in an interpreted language like MQL4 can be done in a true language in the blink of an eye.
     
    #15     Jan 24, 2011
  6. Good1

    Good1

    In the days of QuickBasic, when speed was needed, a programmer could write routines in *assembly* and access them with Basic calls. MT4 does something similar with an ability to call DLL's. Have you researched this angle?
     
    #16     Jan 24, 2011
  7. Strategy OnTrade() / OnQuote() event handlers (where you place your strategy logic) get triggered at this rate in the backtesting mode. I.e. OpenQuant reads ticks from the disk, sorts and synchronizes tick streams for different instruments and emits ticks to the strategy engine, which in turn passes tick streams to corresponding Strategy objects (OpenQuant creates one Strategy instance per instrument) and triggers OnTrade().

    It's much more simple to just capture and write ticks to OpenQuant built-in database. We can achieve something like 1M events per second reading/writing plain stream of ticks to OpenQuant db. Not really a surprise since we use a custom binary db optimized to work with time ordered object streams (market data).

    Cheers,
    Anton
     
    #17     Jan 24, 2011
  8. LeeD

    LeeD

    Regarding platform, I'm sure lots of people would recommend their favourite platform (or the one they are developing/sellling) but hear you need to do your own legwork. Make a shortlist of platforms (remember, platforms that are most popular are those that come free in some form, like NinjaTrader and MetaTrader, not necessarily thsoe most suitable). Search the web, read each platform's forums. Pays special attention to platform starting to lag while tracking a busy symbol like S&P 500 emini during increased market activity. Some platforms are much better than others.

    If platform supports 1-tick, 5-tick etc bars, this can make programming a strategy easier.

    Given you don't consider yourself a "professional programmer", the advantage of using a "platform" is the support. Datafeed and broker interfaces change frequently. As a platform developer yourself you'll have to follow these changes and check if they affect the operation of your platform, modify the platform... then test, test, test. As an end user it's easier if someone does all of this for you. On the downside, a platform would not support all broker features. One platform might not support price modification in existing order (so in a trailing stop you have to cancel the old stop and place a new one). Other platforms wouldn't support OCO (one cancels other) orders etc...

    MQL is "compiled" but it's compiled into bytecode. So, by using a laguage that compiles into native code (C++, Delphi, Power Basic) or a laguge that runs in a virtual machine (C#, VB.NET, Java) you will gain at least 10x speed advanatge.

    I believe you can plug DLLs into MetaTrader. This may be an easier first option before swutching the platform.

    If you do complex calculations, these often fall into one or more popular classes such as matrix algebra or optimisation. For each class there are popular libraries that will likley do the task a few times faster than the code you write yourself (think of Intel MKL or Boost). If you need one of those, this limits the choice of programming laguage.

    Similarly, some trading platforms would work with any compiled or NET laguage (in the form of a DLL) while with others you have to use the one the platfrom developers have chosen.

    Further, the highest speed gain is usually obtained by improving your own algorithm. Mone calculations that don't have to be in a cycle outside the cycle. See if some calcualtions don't have to be done every tick etc.

    When choosing a programming laguage, consider speed of developement versus speed of execution. 10% - 20% gain of C++ versus C# or vice versa is immaterial because computers are becoming faster alll the time. On the other hand, if you are fluent at the laguage, you can easier do things in a different way, which can easily speed execution up 2 times or more.

    No, though it might help. Note most trading platfroms (including popular .NET ones) don't support calculating indicators/trading systems in multiple threads. This means they would use multiple cores for data collection, charting and housekeeping but all indicator/strategy logic will be executed on a single core.

    If you want to do parallel calculations yourslef, it's an additionla level of complexity (like thread synchronisation and locking) that you might not want to handle. If you have 4 cores, just run 3 instances of your favourite platfrom (and keep 1 core for the operating system housekeeping).

    C++ gives the developer more control over exact execution logic. For example, C# effectively enforces the use of built-in collections (list, hashmap etc). They are great for "general" use but the implementation may be massively suboptimal for your specific task. In C++ you can implement your own collections and finetune them for the specific task you are trying to accomplish.

    I would strongly advise againts even considering assembly language. With the complexity of what is happening in moderrn processors the usual effect of an experienced C++ programmer trying to write in assembly languge is a massive loss of execution speed. Optimizing C++ compilers are just that good.
     
    #18     Jan 24, 2011
  9. Yes, we have using 4 DLL functions now and i am thinking to increase my dependence on DLLs for calculations. I think this is one of the quick solutions i could do now.
     
    #19     Jan 24, 2011
  10. Actually, i am not familiar with MQL4. I have used professional coder to design my ATS. We are working now for more than 6 months and it becomes long code that took ages to backtest with the slow MT4 modest backtester. We have noticed there are some mistakes in execution we think it might be due to too many calculations at the same time is running.

    My plan is to use MT4 as a cheap research tool to enhance the automated strategy i am developing then once i have a sound trading logic i could switch to the most suitable for coding/execution platform.

    Now, even for research, MT4 is giving me hardtime. Mainly because, slow backtester and limited language.
     
    #20     Jan 24, 2011