Complex Event Processing/Event Stream Processing

Discussion in 'Automated Trading' started by zzt, Apr 18, 2007.

  1. zzt


    If anyone has experience with Complex Event Processing(CEP)/Event Stream Processing(ESP), I would appreciate your input.

    My question(s) is/are very general.

    With regards to processing speed and scalability,what are the advantages of CEP/ESP engines vs dedicated C++ algorithms which process streaming data?

    There is an awful lot of hype around the subject space of CEP/ESP, but very little data in terms of comparing it to dedicated event based algo's which process streaming price data.

    For example, say i have a dedicated arb model, which subscribes to real time streaming data, builds its own time series arrays, processes the tick events and sends buy/sell/amend/cancel orders. In effect my data query is static, (i.e. same datafeed subscriptions, same algo) so i dont have need for a streaming data query language.

    Where is the advantage of a CEP engine? Where would it fit in the data processing chain so that it would give an advantage? Would there be saleability advantages if I were running 15 arb models in parallel?

    To my mind, any generic solution that attempts to solve all streaming data issues in one app. must have trade offs in terms of either data processing or speed vs dedicated data processing algorithms?

    Thanks in Advance
  2. rosy2


    the advantage is the query language in esp software. you can create a new algorithm in a minute. I use esper which is a java api.
  3. zzt


    Thanks for the response rosy, but what i was getting at was i dont need that query language when i already know how i want to process the data in a real time environment. Or am i underestimating the power of the queries in these CEP engines?
    I still cant see how a a sql based query language will give me the real time number crunching power of a C++ in memory process for example. I guess it would be much quicker to work with the data however.

    I'm just having trouble seeing how this much hyped technology fits in to the whole process.

    Thanks again
  4. Apama
  5. wave


  6. You are right.
    Generic apps can be MUCH slower than custom.

    But the FUTURE is "grid processing"...
    Stringing together 10 or 100 generic boxes...
    Into one virtual "supercomputer"...
    Making processing bottlenecks obsolete.

    The $1,000 product is Digipede.

    The Open Source product is Alchemi.

    I'm going to implement one of these soon.
  7. Algorithmic Trading = "The New Technical Analysis"

    New century... same scam.
  8. zzt


    HoundDogOne. Thanks for the links I will check it out.

    I agree with you.... Lots of slick salesmen and no real info.
    I went to a presentation on algo offerings at a major investment bank a few weeks back and the execution trader says something along the lines of "...well first we pull up the chart to see which way the stocks going, and then decide which algo to use..." we left soon after. Your right man. Same old sh..t!
  9. rosy2


    other than the query language then I cannot see an advantage. btw the data streams through the queries so it is realtime. give an example of your algorithm and I would think it can be done in a cep environment.
    #10     Apr 19, 2007