Thank you so much for your helpful answers, nbates! Is there some book that I can read on the 'cached, state aware' techniques? I finally found this link, they hide it pretty well: http://www.progress.com/realtime/products/event_engine/index.ssp see what you think of their 'distributed in-memory caching', is that really a new software architecture? Is that something that I can build with the way I structure my database and query? Is that their kind of architecture can save you all the time in networking and therefore have a big time margin so that you don't necessarily need to hand-tune your algo? I heard that their platform is written in Java, is there any reason that Java could be better for this kind of tasks, or just that it is simpler for people to use? So can you make your algo fast enough to automatically respond to all these situations, with computations? So that takes care of the risk management part. So what you are saying is that if the risk management part is simplified then the algo for positions tracking is easy. Got that, I finally got that advice that everyone is telling me here. That is interesting. I never quite understand why the preference for C++, because people like its class hierarchies or some other reason? So maybe I can write my GUI part in .Net and have my trading algo to call on that module, or call it through Octave? Would you use something like Octave in your trading algo? It seems that everybody uses Solaris or Linux, because Windows insecure or unstable? Anybody care to comment on operating systems for algo trading? Thanks a lot.
Here I finally figured out how to use quotes. So fifo is some kind of swtich you use for networking and you use two processors, one run octave all the time, and another the trading algo? and mmap is some kind of temporary database you use? See other posts on questions on networking and memory time. See the other link on progress software and their event streaming processing. So here you are doing the equivalent of what they call 'event streaming processing' (esp) there, what is the relative advantages and distadvantages of this way of doing esp versus their way of doing esp? Comments from everybody welcomed. So this octave runs on a separate processor, it doesn't need to be complied even though it is a high level language? on the other hand I do need to compile the trading algo with QuickFIX, of course. So what you are saying is that you want to anticipate the outcome of your execution and ask octave to do computations before the fact instead of responding to the outcome of execution after the fact, and do computations on the fly. Is this motivated by a constraint of the speed of the algo? The question here is whether your strategy uses historical data, such as vwap for decision-making, or uses current market condition. The criterior for that I guess is either the trading model or computing speed of algo. I would take that execution is an indication of market condition. But then nbates' post would indicate that it is an art itself, to find the 'Sweet spot'.
Hey ecoscien, Only realtime operating systems can do real 'hard-realtime' processing. I am pretty sure there are not many hard-realtime trading systems out there. No chain is stronger than its weakest link, right? Well, in our case the weakest link is the network. TCP/IP isnt 'hard-realtime' so as long there is a TCP/IP network between you and your broker, all you do will be 'soft real time'. Just as a curiosity, the average current generation game crunches more floating point data per second than a trading system would crunch in a week Now, about keeping your system response/reaction times low: -> Make good, educated use of threads -> Optimize for L2 cache usage: treat RAM the same way you treat your harddisk - not as infinitely fast memory! -> Do your own memory management, avoid having your application being paged to the disk as much as you can -> Keep your codebase simple and small, do not overdesign and do not sacrifice performance to favor reuse. I hope that helps
If you are concerned about performance, your only choice is C++. Really. Java is a great language, but all it is 'great' features comes at the cost of execution perfomance. Same thing for all .Net languages. I personally love C# but it isnt a hammer for all nails, that is for sure. Python is interpreted and should only be used in situations where performance isnt a critical issue. It is a great, well designed language though Now, about memory leaks on C++, remember: THEY ARE MAN-MADE. Yeah, I said what no one wants to hear, lol, it is ok, things are the way they are. There are no excuses for memory leaks on C++. really! You allocate, you deallocate, it is all about discipline But most important thing is: IT IS ALL ABOUT INTENT What do you want to do? Crazy AI running at high frequency monitoring 200 symbols? That is a machine-language compiled language's job, and by that I mean C/C++ Deterministic systems running on a couple dozen of symbols? That is a Python job for sure! Anything in between? Java and .Net will do I would recomend that you sit down for a couple of hours in a quiet room with a piece of paper and write down exactly what are you trying to accomplish, and then make your decision.
Adaptive Weight Estimation for Interference Nulling on a DSP http://www.astron.nl/tl/thea/publications/awe2002_alliot_CTIT_submitted_twente_2002.pdf The case for real-time visibility http://www.embedded.com/showArticle.jhtml?articleID=174300427 [great site!] Real-time Systems with Windows NT http://www.theragens.com/misc/MR - Windows NT Real-Time.htm
Thanks, Mrtwo, Good Point. what does 'good, educated use of threads' mean? How do I manage my own memory? Thanks.
Thanks a lot, mrtwo, for the ranking of languages. I wonder whether any one here can give a comparision of operating systems--- Unix, Linux, or Windows---their algo trading pros and cons. Thanks
It means, pick your language, study how threading is implemented on it, study threading best pratices, parallel computing theory, scheduling, etc. Study it until you know how to manage threads as well as you can drive your car About memory, it is quite simple actually, use C++, write your own memory poll and overload the new and delete operators to use it and your good to go! Parallel, high performance systems are made 80% of education and 20% of code so be prepared to become a master C++ programmer if you want to follow this path Peace, MrTwo
I sense this will be the beginning of a flame war lol DISCLAIMER: I LOVE ALL OPERATING SYSTEMS! Windows 2003 Server: Use it if you actually want to get anything done. The Visual Studio IDE and its optimizing compiler are the ONLY serious development platform out there right now. Windows 2000 / XP: NO NO NO NO. (NO) Linux: Awesome operating system but their development tools are primitive to say the least. Optimizing compilers are still more to 'promised land' than reality at this moment. Even the best Linux based IDE pales in comparison to Visual Studio 2005. Unix: I gave up Solaris and HP-UX about 5 years go. Simply put, Unix stopped in time Peace, MrTwo
Unless you tell us the details of your system ideas, you have to find this out for yourself. I would try some commercial trading programs like Tradestation and discover for yourself their limitations with respect to your system design, before you realize itâs worth the extra time, blood, sweat and tears to write your own programs from scratch. Because, alongside the time you need to work your statistical algorithms, you will also need to learn about asynchronous function calls, threading, hashtables, class libraries, and so on. Yes, you can do âanythingâ if you could write your own automated trading platform, but it will take a lot more of your time. If you are not an experienced programmer it will require a lot of time (a lot of traders in here were programmers-turned-traders and they'll act as if it's easy.) Programming is a lot more accessible and âuser friendlyâ today, but there is no way around the extra-time issue.