anybody know how to write a feed handler?

Discussion in 'App Development' started by fareastcoast, Jul 18, 2013.

  1. Guys, NxCore is not a multicast. I'm looking to build a raw feed handler.
     
    #11     Jul 19, 2013
  2. nitro

    nitro

    I have built many feedhandlers, some of which can handle data in the microsecond range for HFT.

    I don't understand your question. Do you want someone to program the feedhandler, or you want to be able to replay data and feed it to your feedhandler as if it were multicast?
     
    #12     Jul 19, 2013
  3. alpha123

    alpha123

    #13     Jul 19, 2013
  4. Actually it is. NxCore supports multicast distribution.

    It still is not what you likely want. That said - think what you really want.

    * If you colocate at the exchange and do NOT get into the game of direct feed you are fater than 99.9% of the people. Get a decent high performance profiver, done.

    * If you try to win the speed race, this is NOT about "writing a feedhandler" as in programming. Seen some HFT project let me tell you - software losses. The game now is FPGA for decoding. Hardware. Latency is not measured in 0.x miliseconds - more 0.050.

    So, this is not necessarily "next level", it is a totally different ball game. Be prepared to spend a lot more.

    Or just try to stay smart, not fast (most HFT work on speed alone) and if you manage to avoid the HFT rat race, then colocating and a decent provider will be good enough.

    Btw., NxCore is good to test this stuff - they hit you with a lot of symbols. I get the whole CME from them, and that are about a quarter million symbols.
     
    #14     Jul 19, 2013
  5. Basically, I want to receive the CTS/CQS/UTP multicast from the exchanges. To do that, I need to develop a feed handler to handle that type of data stream. But it costs $6k/month to get access to that multicast. So instead of spending $6/month while I am developing, I would like to simulate that stream, for instance, using a past day's tape, to develop my feed handler. I'm wondering if NYSE/Nasdaq provides a service like this and whether anybody on here is sophisticated enough to know about it.
     
    #15     Jul 19, 2013
  6. So yes, it is possible to generate fake data, and then receive it. We already have a setup that does that. However, I want something more realistic to test on, not some contrived setup. Only by getting an actual sample from the exchange can I really ensure that we have correctly taken into account all message types and possible exceptions.
     
    #16     Jul 19, 2013
  7. nitro

    nitro

    Have a buddy use wireshark to capture the packets [for a day]. Then replay it in your own network. A little clumsy but would work.

    http://wiki.wireshark.org/CaptureSetup/Ethernet
     
    #17     Jul 20, 2013
  8. alpha123

    alpha123

    nitro is right, network friends could help, just be prepared to change your feed handler based on exchange message changes (announced in advance and tested before changes pushed to prod) as well as to handle a back up line if primary is down, there are not too many app teams out there who can do it right (with hundred of thousands of messages per sec like OPRA and low latency), be prepared to handle slow consumers of data (algo code) flowing from your feed handler, have fun...
     
    #18     Jul 20, 2013
  9. Ah, this is a great suggestion, this is exactly the sort of thing I am looking for. So I can be in the production environment for a little bit, capture packets for a day, and then use the replay for development.

    I think I will start down this route first, just have to find somebody who's willing to help capture packets for me.
     
    #19     Jul 20, 2013
  10. onelot

    onelot

    which vendor is quoting you 6k for cts/cqs/utp raw feed? should be closer to 2k and they should be able to get you a day's worth of data and a spec to test with.
     
    #20     Jul 20, 2013