My initial model I was using C++.NET extension. The extension remoted over to my external process app. This wokred very well since I was doing much historical analysis outside ANVIL process. In my latest strategy I need performance so I natively made everything in a C++ extension dll. I do monitor about 2k equities. Orders are executed in about 100ms. Not bad considering is not a direct exchange API. If you still want to use external app, your best perfromance is going to be shared memory. Once you get accustomed to the messaging architecture everything starts to make sense and you extract evey bit of perfromance possible. I do agree that it would of been better to add the API. But one thing that I like is that I can use ANVIL GUI itself as my oms system. I can see all my orders, pnl, etc ... all I have to worry about is my strategyy and order entry/exits. For exmaple, I can expire orders and positions in any exchange since I wrote the monitors into my extension. I hook into the pump when the DLL initializes. Here is partial code: g_AnvilHook = SetWindowsHookEx( WH_GETMESSAGE, CAnvilStatic::ANVILThreadMainMessageProcHook, NULL, g_MainThreadId ); I send WM_USER* messages from my worker threads to the main thread for ANVIL functionality, like place orders, etc ... without interferring with ANVIL processing. IT is pretty fast. All you have to make sure your code that runs on the main thread is FAST. This way your ANVIL perfroms better.. I run a gray box so I want to see it running jsut in case . But you can use the service if you want it as black box. ALso to run it off market hours use login: A and no password. You will not get real time data but you can load you extension and basically interact with ANVIL. It is limited but you can workout those bugs like loading DLL exception, etc... If you are using MFC, set module state on eahc API call. AFX_MANAGE_STATE(AfxGetStaticModuleState());
I chose named pipes for a simple reason... I've never used LPC before! I had used named pipes for a similar project before, so I was able to just snag the code -- they're pretty simple, and I can easily create any sort of command objects I want. Since my apps are running on the same machine, local named pipes operate in kernal mode and are very fast, but if necessary I could easily move apps to other machines without breaking the communication code (as long as I'm on the same LAN). I use the API for gathering quotes, which I process and filter in another application. My routines for named pipes will allow me to have multiple applications listening to what is sent out from the API, which was desirable, and theoretically, if I were passing commands back, such as orders, etc., I could still have multiple apps running off the same instance of Anvil. These were some of my thoughts when I went with named pipes. I was just trying to decide between sockets and pipes, but I like the message structure and speed (when local) of the pipes. Perhaps you could do all this with LPC as well, but I just don't know anything about it. ct-armour's app is an active trading app, where mine is analytical, and his input is probably more valuable to you than mine.
This is pretty interesting ... So if I understand this correctly, you are creating a few worker threads as soon as the extension initializes and do all of the major order placement through the main thread. So a couple of followup questions: 1. How are you sending quotes over to the worker threads(Forgive my inherient weak understading of the Anvil architecture ...)? 2. Why do you need to hook into the Main Anvil window? You could probably easily create another window through your main thread and intercept all WM_USER++ messages sent to it as you can call the B_API from any thread right ... How bad was the performance that you moved to inherient anvil extension processing and placement model? I am planning to go with the namedpipe/shared memory route as I will be using Openquant for processing. Finally, both of you seem to know quite a bit about OS fundamentals and windows in general. Do you work for a hedge fund or are you self employed traders? In any case, that's the route I want to head out on and would appreciate some advice (feel free to PM me!) Thanks
I create a CWinThread object as soon as my DLL is loaded. For each of the symbols that I'm interested in, I have a class that derives from Observer (I think that's the API class -- sorry, I don't have it in front of me right now), and I tell them to listen to the prints/L1/L2 that I want. These classes are all stored in a CMap. When I receive a message from the API, I use a PostThreadMessage to send the message to my CWinThread so that I can process it, send it to my other app via named pipes, etc. Now what I don't do is place orders/track trades through the API, so I can't help you there. ctarmor-et has told me that I won't be able to call the B_API from my CWinThread for placing orders, etc., but that I'll have to transfer back to the main Anvil thread to do that. This is where the hook comes in.
I think named pipes will be great for multiple machine configuration in the same subnet ... I used named pipes many years ago and they worked very well. I have never used LPC either. Pandabear, I have only 2 extra worker threads, one that takes care of writing logs and ticker files (for historical analyzer I have). The second thread receives post messages from the main anvil thread to perform longer tick analysis after certain criteria (my model) via :ostThreadmessage() to my worker thread. Yes, you could do create another UI thread but it will not be the main anvil thread. I was told by the development team that you can only call B_* apis from the main thread since they do not support multi-threaded access into the APIs. Hence I hooked into the main thread and send my WM_USER* msgs to be processed. This is similar to what tfjied is doing with CWinthread. Although I do not send every *_BOOK_* and NW2_QUOTE* to my worker thread, only selected messages and prints. At the end, the less you can have the main ANVIL thread do the better perfromance you get. Keep your observable handlers in a short leash. I do use stl for my containers and not MFC for that reason. I find performance to be good. I can have orders executed in about 100ms. I do not work for a hedge fund (although I'll like to) . I do work for large financial firm. My expertise has been 18yrs in Windows server/client development (and tech lead) in VC++ and .NET. Do you have a exciting Hedge Fund job for me ? :eek:
I forgot, you can automatically load your extensions by adding the folder location where your extension(s) are to the ini file. [Extension] Path=C:\SomeFodler ANVIL will automatically load all the extensions that it finds in that folder.
Yes, you'll probably get better performance with STL. I use MFC for easy-to-access collections, such as a list of observers that only need to be looked up/cycled occasionally, i.e. for daily updates, destruction, etc. For the actual analysis (in my separate app) I use STL, as well. (I have not actually directly compared the performance of MFC versus STL, but I've got to believe STL will be faster.) By the way, there's an important caveat to sending messages from thread to thread: If you're going to use a PostThreadMessage or the like and you've got a lot of messages, like you're filtering over thousands of stocks, be aware that the Windows message pump has a queued message limit. I think it defaults to 15k messages, but I believe it can be expanded. I ran into this limitation in another app, and some of my messages were getting lost. For this reason if I'm doing a PostThreadMessage occasionally I always check the result to make sure it was actually queued, and retry if unsuccessful. If I'm doing it a lot, like sending a lot of prints from one thread to the other, I actually build my own queue so as to allow my threads to run asynchronously yet not miss any messages if one thread gets backed up for some reason. Another benefit to this is that I now have a list of all the messages that are still queued. If I need to shutdown the thread, for example, I can be sure to delete any allocated data that is part of the various messages. This might be common knowledge, but it sure surprised me when my messages were getting lost!
Yes you are right there is a max limitation. I have not reached it yet. My worker thread processes the messages in less that 100ms (I have warning set up if over 100ms). If the ANVIL thread overfills the queue there is also a warning. I just dropp the quotes. So far I have not reached teh limitations and perfromance is good. If the queue reaches that point then I really do not want those quotes anyway. They'll be far behind.