Please recommend a platform for a black box

Discussion in 'Automated Trading' started by HiFreekTrader, Oct 1, 2008.

  1. I'm going to launch an autotrader and I need this:

    - real time tick data feed through API for 2000 NASDAQ and NYSE symbols simultaneously

    - access to historical ticks (at least last 7 days)

    - ability to submit orders (auto routed) through API.

    Unfortunately, trading volume will be nil - that particulat black box produces only about 5 signals in a month and I am going to invest only $2000 initially (if it succeeds, I'll invest >$50k).
  2. wow 2000 symbols with a real tick feed on all 2000 at once, I am interested to here if anybody knows of a platform that does this... that is tough to do.
  3. DTN.IQ gives 2300 symbols at once, if I understood their web site correctly, for around $200/month. Opentick gives you unlimited symbols for free, but they are closed to new users.

    And I need order entry API too - I don't want to stare at the screen all day long for only 5 trades a month :)
  4. 2000 symbols at a time is not so much the issue as how many updates per symbol per second occurs. I'd say at most you are looking at 100,000 updates per second (could be less, more like 20,000).

    While this is achievable, you probably aren't going to find a desktop, off-the-shelf platform that will handle this. Lots of people have platforms that provide this level of performance; however, they are generally proprietary. If you want to handle this much data, you will have to create your own "platform".

    I know there is another thread on ET that outlines building a trading platform, but it is dozens of pages of posts long. To address your exact scenerio:

    1. Get a hefty machine (Linux, not Windows, in my humble opinion). Connect this machine to DTN and suck data down a huge data pipe (T1). You are only as fast as your hardware and line-speed. If your selected 2000 symbols are generating 100,000 per second, DTN may want you to split them up into separate connections...

    2. Write data sucking app for Linux machine...this must be multi-threaded or the setup in (1) won't do you any good.

    3. Write your signal application that is subscribing to the data from (2). The question is, do any of the signals on the 2000 symbols you mention relate to each other, or are your signals related to an individual symbol alone. Depending on the answer to this, you will either run this code on your machine from (1), or on a second box (this will be better anyway).

    4. Write simple order submission application that connects to your broker or the exchange directly, listens for a message from (3) and happily makes you money.

    5. There might be a couple of other small items to develop, but this is most of it...

    Conclusion: You are probably looking at a setup consisting of a couple of machines, unless of course your 2000 symbols only generate a couple of hundred market updates per second, and then you can get away with less hardware. But, if the volume spikes, you'll be stuck. This still might be too much for most off-the-shelf products because all of the symbol prices come through the core platform, which handles everything else. I figure since you are talking about 2000 symbols, you probably aren't planning on looking at them yourself all at, you really don't need the GUI bells and whistles that come with the platforms, just the data (and your algos).

    Now, if you weren't planning on writing this yourself, feel free to private message me. If you haven't guessed, I am not a trader, I just write the software.
  5. I do know how many quotes/second I should receive - about 500 quotes/second at the peak. That's because I intentionally omit high-volume issues like QQQQ or MSFT.

    Depending on compression, that should be about 20-100 kilobytes/second.

    OK, assuming I have resolved the data feed issue, which broker will let me submit orders through API, without having to pay hefty monthly "platform" fee (TradeStation) or >$10K account size requirement (IB) ?
  6. Davdse


    If only DTN had a non-windows-compatible API...
  7. rdk


    Give that a look if you want a real high-perfomance datafeed.

    Now, how you're going to process your logic on 2000 symbol, tick by tick, that will be interesting. Maybe you can get away with 1 second bars? or 5 second? then maybe you can whip up a little cluster to handle your needs.

    openquant is a good platform, not sure if it will do what you require though.

    good luck.
  8. 5 seconds bar would be enough for me, but they generally offer 1-minute bar and the next step is raw ticks.

    Looks like NobleTrader may be suitable for me.
  9. rdk


    i am not familiar with noble trader.

    The reason I say second bars it's that it's a lot less resource intensive to compress ticks to second bars, THEN do your logic, then to try to run the logic on every tick. Building second bars doesn't take too many lines of code. Every second you record an open, then update the high/low as needed on every tick, then record a close, keep track of the volume. Evaluating the logic for each bar, probably more to it than that.

    For a few symbols, this prob wouldn't matter. For 2000, yes, unless you're running one massive cluster.
  10. This question may have been asked before but, why would you need tick data for a strategy that only generates 5 orders a month?

    I assume you're holding the positions for a while if these 5 trades per month are going to make any money for you. If this is the case, then less frequent data updates should work for you.

    #10     Oct 2, 2008