I think I am a bit getting what you're saying now. To clarify, all executables communicate using messaging, no files. QC->SES is alright, it's just a stream of quotes. SES->OMS should be SES<->OMS and I need State Pattern for that. You said: -- meaning I have to pass the State object from OMS to SES which I cannot do because I am sending only text messages between executables. I can send a text message instead with the State variables from OMS to SES so SES will process the message and act accordingly. Is this what you mean?
Yup. Your SES's main loop should look something like the following: execute() { 1. receive market data (price, volume, etc) from QuoteConnector 2. update values of any required technical indicators (20 EMA, etc) based on data updates from QC 3. receive data from OMS //position size, entry/exit prices, and anything else your Strategies will need to know about your existing orders 4. store all the above data into a State object //this is the State pattern 5. invoke BaseStrategy.process(State object) //this is an abstract function. implementation will depend on individual strategies. this is the Strategy pattern 6. individual Strategy subclasses will process the latest State of the market, and determine whether any orders need to be opened/closed/modified 7. send order changes to OMS }
Take in all of the quotes and trades...filter as needed, provide a separate "feed" for each service you need by filtering out what each service does not need. It seems you have done this I have seen other "systems" where quotes come from one data provider...but "they" try to trade off feed data from another provider - VERY BAD! HINT: GET SOME SERIOUS F'ING RAM - take your TCP buffers and jack them WAY UP. If you are pulling down OPRA data...10G NICS or nothin'! More precisely...if you can tweak the buffers for your NIC drivers...do so...then tweak your TCP buffers. You may have coded some really fast stuff...but, stuff like NASDAQ TV...well, ummm, cranks down data. OPRA...like I said, if you don't have 10 G NICS...you are wasting your time...I have watched OPRA burst well over 1 Gb/s when you look at it under 1s intervals....do NOT trust simple netstats or ifconfigs to try to estimate your bursts - netstat and ifconfig will not show you that granular. Also, your code may be quick...but, when things like the "Flash Crash" hit...you will be SOOOO happy you have that extra room in your buffers. If you want to see if you are buffering in the OS...run "netstat -nap" - if you see stuff in your Recv-Q - you aren't keeping up with your data provider. If you see stuff in your Send-Q - your a buffering stuff between your data receivers (internal stuff). If you are trying to use TCP to transfer data internally...move it over to UDP - scales better and will not slow down stuff when the sh!t hits the fan - Flash Crash....better to spill stuff on the floor (UDP) then have your whole system fall on its face (TCP backed up like somebody who lives off prunes and fiber!!!) -gastropod
You mean getting quotes from one data provider but placing orders through a (different) broker? I think some people do it, if you have a good data provider then if you move to a different broker you can still be using your tested data provider. Memory yes, I had TCP buffer at 8K and it was cutting the messages.
Sorry - I edited my post above - please see some of the changes. My statement about people who use different providers is something like "going native for" for quotes - you get your NASDAQ data for NASDAQ stocks from NASDAQ...but, then trying to use something like Reuters data for placing the trades. That is when stuff can be very screwed up. Think more Megs rather than Kilos for your buffer sizes
The problems with multiple data sources...the symbols can be different. You want consistent symbols - either S&P style or not. When you have multiple data sources...your symbols can get hosed. The other thing about multiple data providers - missed events. If your quote server HAS a quote or trade...but your order execution environment does not...and your system tries to make a trade based off a quote from your system...then the OE system balks at the thought of making a trade at a price out of the market...even though your quote system says it was "in the market" - weird situation...but, not impossible...you pick up an event...the data provider for the OE does not...should your strategy planner make the trade or not???? One thing you could do if you are having problems like that...connect up to "somebody" who has a LOT of FIX connections...and see if your strategy planner can make the trades based off YOUR data....better the data you know...than the data you don't know.
Would you like some "extra" hints? I have been "out of the game" since being laid off...but, the financial services world still has the "coolest" tech field out there. I designed a system set up for feeds.....I could show you a quick hint that would probably VERY much help you out. Ciao, gastropod
One thing which is important to me, is that from a logical point of view, price needs to trade through a LMT order before a strategy considered it (potentially) filled. Although this makes for realistic backtesting, in live mode an order can be filled when touched, and for stocks it can be not-filled even once price has traded through. Which creates the need for a strategy to keep track of both the logical state of an order (what it should be according to live data) and its actual state (according to the order management feed), and behave properly in case of a discrepancy.