AutoTrading 500+ instruments

Discussion in 'Automated Trading' started by apfx, Apr 15, 2012.

  1. apfx

    apfx

    Does anyone know if any other retail platform besides Amibroker is capable of Autotrading 500+ symbols simultaneously .
     
  2. At that point, why not just choose a basket or narrow your trades dramatically? Your broker will love you..
     
  3. Daily/weekly/monthly bars, or intraday bars/ticks?
     
  4. apfx

    apfx

    The number of trades is significantly lower depending on system parameter Max Open Positions but the signal calculation and ranking is performed on all 500 symbols.
     
  5. IB can do this no problem.

    seperately IB's data feed is sampled, so if this impacts your strategy you can combine another feed (nanex/iqfeed or feed from another broker) with IB's execution.

    tradelink allows you to run in both scenarios out of the box. (feed from broker, or split feed where you use feed from another broker or feed provider).

    google tradelink project or tradelink.org for more info
     
  6. apfx

    apfx

    Daily/ Intraday bars.

    Just curious if any other platform such as MultiCharts, OpenQuant, RightEdge, or NinjaTrader can handle this better

    In Amibroker I'm getting code execution times around 500 ms
    to complete the whole cycle

    calculate signals
    perform ranking
    handle orders

    for 500 symbols 300 bars/symbol
    on a Core2, 2.1GHz, 8GB machine.
     
  7. For ninja - make it a no.

    It is single tthreaded in the core - so that one will require a fast core and goes item by item.

    You are a lot better using a professional signal processing backend, like the one in new SQL Server (no joke - no idea WHY they bundled it there, but they have a nice CEP stack in there for complex event processing). You need something that properly works on multiple threads to handle the amount of signals. Every tick must be queued, then haneded off to queues by instrument that get handled by X threads in concurreny, otherwise you are totally limited. THen get a machinei with a decent thread count and you should be sub 10ms. A dual opteron these days would put you down to about 15 symbols per core.

    NInja also would b REALLY painful to set up - either a LONG basked of symbols in one strat, or 500 strategies, all to be manually configured. Regularly ;)
     
  8. I can guarantee you that none of the other packages you mentioned is able to handle the same throughput and latency that Ami offers. There is a reason that in 2012 Amibroker still runs under an antiquated UI and looks like it was coded up in the 1990s (which it was, to my knowledge). Its raw C++ and involves array computations whenever possible. It can't get much faster for a off-the-shelf product.

    If you want to go faster than that then you need to code things up yourself and start distributing tasks, such as running a segregated order execution gateway that is not running on the same threads (possibly not even on the same machine) than your strategy engine. I am not saying that there is nothing in between but if you really wanna push beyond trading 500 names simultaneously then I would look for a customized solution (given you are serious about this endeavor, otherwise save your time).

     
  9. from a cpu perspective this should not be difficult at all on almost any platform. The bigger issue will be whether your data feed can deliver 500 symbols of tickdata in low latency. With iqfeed or nanex you should be able to accomplish this though over decent internet pipe, or on most feeds via colocation.

    here is a benchmark constructued using tradelink which demonstrates this. It computes averages at least 300 bars for 500 symbols, the average computation run for all symbols at each calc run is 8-12 milliseconds.

    Even if you are doing something much more complicated it should be easy to keep it under 500ms (assuming you have low feed latency).

    This is on a quad core 3ghz xeon with 3GB of ram, no threading is used. It will peak a single cpu and used no more than 0.9GB of ram when running.

    Here is the benchmark measurement, constructed in tradelink :


    symbols with bars: 500
    bars per symbol: 1001
    averages computed: 350999
    average calc time (ms): 11.6068925666199


    Code:
      [Test]
            public void Test1()
            {
    
    
                avgs.Clear();
    
                // get 500 symbols
                string[] syms = TradeLink.Research.RandomSymbol.GetSymbols(4, 500);
                // prepare to hold tickdata 
                GenericTracker<Tick[]> tickdata = new GenericTracker<Tick[]>(syms.Length);
                // process
                const int maxtickspersym = 10000;
                foreach (string sym in syms)
                {
                    // get some tickdata
                    var ticks = TradeLink.Research.RandomTicks.GenerateSymbol(sym, maxtickspersym);
                    // timestamp it
                    DateTime cur = DateTime.Now;
                    int date = Util.ToTLDate(cur);
                    for (int i = 0; i < ticks.Length; i++)
                    {
                        ticks[i].date = date;
                        ticks[i].time = Util.ToTLTime(cur);
                        if ((i % 10) == 0)
                            cur = cur.AddMinutes(1);
                    }
                    
                    tickdata.addindex(sym,ticks);
                }
                // prepare to track time per run
                List<double> runms = new List<double>(700);
                // prepare to hold bardata
                blt = new BarListTracker(BarInterval.Minute);
                blt.GotNewBar += new SymBarIntervalDelegate(blt_GotNewBar);
                // prepare to track time
                System.Diagnostics.Stopwatch sw = new System.Diagnostics.Stopwatch();
                // run our simulation
                string lastsym = string.Empty;
                for (int i = 0; i < maxtickspersym; i++)
                {
                    // get current computation count
                    int curavgcount = avgs.Count;
                    // start timer
                    sw.Restart();
                    foreach (string sym in syms)
                    {
    
                        // build bars
                        blt.newTick(tickdata[sym][i]);
                        lastsym = sym;
                    }
                    // stop
                    sw.Stop();
                    // if we did something, record time 
                    if (avgs.Count>curavgcount)
                        runms.Add(sw.Elapsed.TotalMilliseconds);
    
                }
    
                // compute average run
                double avgrun = Calc.Avg(runms.ToArray());
                g.d("symbols with bars: " + blt.SymbolCount);
                g.d("bars per symbol: " + blt[lastsym].Count);
                g.d("averages computed: " + avgs.Count);
                g.d("average calc time (ms): " + avgrun);
                Assert.Greater(500,avgrun);
            }
            // save averages
            List<decimal> avgs = new List<decimal>();
            BarListTracker blt = new BarListTracker();
            
            void blt_GotNewBar(string symbol, int interval)
            {
                BarList bl = blt[symbol, interval];
                // if we have 300 bars compute averages
                if (bl.Has(300,(BarInterval)interval) && bl.RecentBar.isNew)
                {
                    avgs.Add(Calc.Avg(bl.Close()));
                }
            }
    

    you can construct the same benchmark using tradelink which is open source and works with 15+ brokers and data feeds.

    google tradelink project or tradelink.org for more info.
     
  10. I don't want this to sound harsh because I really do think that tradelink looks interesting, but several times I've gone to the website and always end up bailing because the tutorial / intro material is kind of a clusterfck...it's circular in spots, looks like big chunks of data / pics are missing, and it just jumps all over the place. Am I missing the user guide somewhere that follows a logical / sequential order?
     
    #10     Apr 24, 2012