I think there is a lack of documentation on the web, because threads are not 'sexy', and quite boring. Everyone studied threads, locks and mutexes in their OS class, and don't really have a need to rehash them on the web. Threads have been around for soooo long, and there still is not a standard way to implement them in C++. The next version of C++ will have native thread support, and I think you will start seeing much more about threads on the web. And when you throw in how Microsoft decided to do reader/writer locks, who would not be confused.
It does take years to get really good at squeezing (misspelled) every microsecond out of C++ and other less efficient languages. In the contrary, it is not hard these days to be a "avg" developer. Look at the Indian's. Sub-education and they are getting our low end developer/support jobs ... Get a Dummies book and Visual Studio....You'll be developing your own apps in a few weeks. I am not saying that you will become an expert but you will learn as much to get you started writing simple algorithms, analyzing historical data, reading files, etc ... But to get the high performance you want in todays markets for high frequency trading, you will need a real "experienced" developer to get real-time performance.
I have been trying to find the article I read years ago about smart optimizations that turned out to make performance worse. I haven't done C++ for ages, at least at a decent level, but I'm sure of one thing: don't try to be smart if you don't know how the compiler works. It can turn against you. Instead, write good code. As it has already been suggested, write your code first. If it's too slow then you profile. write programs for people first, computers second Steve McConnell
Here are some books that appear to be current regarding thread handling: http://www.amazon.com/Multithreadin...Technology/dp/0201442345/ref=pd_bxgy_b_text_b http://www.amazon.com/Modern-Multit...tithreaded/dp/0471725048/ref=pd_bxgy_b_text_b http://www.amazon.com/NET-Multithreading-Alan-Dennis/dp/1930110545/ref=pd_bbs_sr_1?ie=UTF8
I do somewhat recommend the "Modern Multithreading : Implementing, Testing, and Debugging Multithreaded" book. The writing is a bit dry, and the code is pretty bad, but it does do a decent job talking about multi threaded debugging. On second thought, I only recommend the Testing and debugging parts of the book.
By Visual Basic, do you mean the classic Visual Basic 6 or VB.net? That's a significant difference. I don't think you really need c++ performance wise, many professional applications with huge load were written in c# or Java. These languages also take care of things like garbage collection for you, something that you are likely to benefit from as you expressed you do not have that much experience yet (I use c# myself for writing applications). The best thing to do is profile and see why your program lags .. only then you can fix the bottleneck. I would suggest also suggest you don't use vb and move on to c# (or vb.net alternatively).
Update: I decided to try out C# on the weekend. Spent half a day learning enough about C# and the VS IDE to program what I needed to (already had a base in C/C++), then the rest of the weekend porting the code from VB6.0. It was a certainly a nice language to program in. The IDE was great, better than BCB I thought. Making Windows apps is just as easy as it is with VB. During the testing phases on the weekend it felt "heavy" starting the program (connecting to quote server and downloading preliminary data). I didn't know exactly how fast it would run because there weren't any streaming quotes coming in but it didn't seem as fast as the program I had already written in VB6. So I fired it up on Monday and it worked as I wanted but it was really slow!! It was only able to process up to 500 quotes per second (from the release .exe) whereas my VB program easily does 2000-3000 q/s without even being compiled. 2000 q/s generally is enough to keep up most of the day except for the open and close. So I thought it might have been some bad coding so I rewrote a new program in C# and only put a counter in the quote_update event to measure the q/s. Still, only 500 q/s. And quotes that don't get processed in time get backed up (not dropped/lost). So at that rate it would probably take me an hour to process 10 minutes worth of quotes. I was looking at 150 stocks from a sterling API feed and I'm using VS2008 with 3.5 framework. Does that make any sense?? Is it because I'm referencing a COM? Not sure what that means exactly. I don't think it can be C# itself as that seems absurd. But is it because of .NET or something, like the way it processes activex? The sterling .dll is quite old. I think the last time they updated it was 2004 and that was only to increase the orders sent per second limits. Any help is much appreciated.
Without seeing your code or implementation, you are probably on the right track in blaming COM. I had experience with implementing COM using IB's TWS API and it was slow as heck. Bottomline: There is a lot of overhead in the COM calls and callback processing. It's best for you to consider a DLL interface instead. The other thing to consider is using threads to handle the streaming quote processing. It's definitely overkill to dedicate a thread for each symbol, but probably 10 symbols would be OK. Finally, have you attempted to contact Sterling to ask them why their DLL interface has not been updated for over 3 years ? Wow, the state of software in this country since all of the outsourcing is just abysmal IMHO.
Thanks for the reply. I'll give them a call on Monday. I rewrote my backtesting platform in C# and tested it on tick data to hopefully simulate what occurs during the day in real-time. It works 5-6 times faster than my VB6 backtester so it seems that it's not a problem with my code, so its most likely the COM. I would like to use the DLL but I don't know how to get the classes and such from it. Is there a tool in VS to look through a DLL? The support documentation is severly lacking. I thought Sterling was one of the most used platforms....