lol data never has to be stale or late. As you yourself mentioned, work in parallel or work serially. Do not work fresh then stale fresh then stale. Stale will not perform like fresh.
The reactor pattern does not dictate the number of threads you use to consume events. I have a thread dedicated to network I/O and another thread dedicated to timers. I dispatch/post these events through an asio::io_service into my main thread in which I do the business logic, there is nothing stopping you having multiple threads consuming the events, but I choose not to since I didn't want locking in my business logic. I would start with the most simple and logical program structure, then I would use a profiler to identify bottlenecks, if needs dictate. In my experience, most programs are I/O or global resource bound anyway, arbitrary adding a bunch of threads will just cause context switching.
So much software engineering.... so little trading.... In all likelihood, your algo (SMA, std, whatever) will require so little actually processing that you are just wasting your time with all of this; Unless you are doing 2000x2000 matrix inversion in real-time, why bother with all this needless complexity.
All the years of back and forth chit-chat on ET about HFT, and people still haven't figured out that at the timeframes being discussed in this thread "algo" and "OMS" are inseparable....
+1 Being the Original Poster, I'd like to remind that the subject of the thread is "Building an ATS". Even though at the moment my strategies do not go near HFT, any contributions on techniques that aim to improve performance are fair game. Also, please keep your posts on the positive side. Glass half-full if you please. It's easy to criticise but hard to give good advice on how to improve. I, for sure, have been learning from this thread and applying some of the recommendations given on previous posts. Stay cool and keep the ideas flowing. Progress report: I rewrote the simulator engine as a separate module and class. I've also done some experimenting with Python's threading and multiprocessing modules. Much to some posters advice, I'm leaning to avoiding threads and willing to try an iterative approach since I'll only be trading on a few spot forex instruments.
Might be worth keeping in mind that without tackling the threading/synchronization issues it will be challenging to grow beyond "a few" symbols. Of course there's no requirement that you have to grow beyond that, just something to keep in mind. But at that point you'll most likely be dropping Python anyway, so perhaps it really doesn't matter. Another really useful experience is to build the other kind of ATS - ie, a proper matching engine ("exchange"). Going through that process unveils an entire world of subtleties in the way market plumbing is put together - subtleties that can lead to interesting insights into trading techniques. When you have an entire system going - multiple ATSes issuing orders to your own "exchange" using market data generated by that "exchange - it all starts getting very interesting and illuminating. Good luck!
Might be worth keeping in mind that without a profitable strategy it will be challenging to grow beyond zero. That being said, every firm I know of does this a similar way.
+1 OP may find that all of this programming is a bit academic, without having an actual trading strategy. All very complex & clever, but that does not put cash in the bank.
Yes, it IS academic. I invite you to read the first post of the thread. I do not - nor intend - to make this my primary income. I want to try and prove it is doable by someone who has taken the challenge with minimum resources. I find it odd pleople try to impose their own objectives into others. If I fail, then I fail, if I do it, then I'll let the facts speak for themselves. Again, this thread is about building an ATS. If you don't like it, there's plenty of threads on the strategy section.