I code multithread in C++, I would strongly disagree your statement that my code would have problems catching up Erlang code.
You're better off using Multicharts 64 bit that's already programmed for C++ for multi thread and multi-core processing.
Um. Yes. More details are needed here. Why does the strategy require a large number of cores/threads? What are the time constraints? What hardware is this strategy being run on, and what are the bottlenecks? For example, if low latency is a requirement then single threading may very well be the optimal solution. Threading or pushing out to the cloud definitely has overhead.
you are light years behind. Here is a list of keywords you want to look up when trying to get into the .Net concurrent and multi-threaded space: Task Parallel Library (TPL) Async CTP TPL Dataflow Btw, Erlang only makes sense when targeting projects that deal with massive amounts of threads, and especially need to handle state machines well, it was originally designed in and for the telecom industry. I strongly recommend to not throw computationally intensive stuff at it.
you can't give up pushing Assembler, huh? I disagree with your religious believe that you can throw Assembler at anything there is. I would challenge you to code a mergeSort that is much faster in Assember than C++ and which is in turn much faster than in C#. Today's compilers get more and more efficient that a simple List.Sort() in C# .NET does not fall by much behind the fastest algorithm you could code up in C++ (to be fair, competing algorithms in C++ and Assembler would also need to be coded to target single threaded models, or there should be a fair comparison re parallel code). Look up related posts at Stackoverflow, there are guys who tried all that and posted performance measurements. My point is that while you try to solve stuff in Assembler to get your 10% or 15% faster "edge" I code up the same stuff in C# 5-10 times faster and run circles around you while you still code. Please take it for what it is, I try to make a point, its not a pissing contest and I am not even addressing you directly but those who believe a low-level language is in all cases the best solution. This was maybe the case 10-15 years ago but surely not anymore today. Heck, I can code up a sorting algorithm in C# and if you allowed me access to Cuda through a library then I would beat any of your pure Assembler routines hands down. My point is that you can do so much in C# and other higher level languages today that Assembler's advantages are all but gone. P.S.: I am aware that some very low level, near-hardware stuff was coded up originally in Assembler. Its great and I welcome it, but if you are agitated by now then you probably did not get my point which was that I can access all those libraries out of C# (well, most of them) and do not have to limit myself to crude, command line type of development instruments but can access state of the art development tools, plus cross language libraries and bindings on top of all the goodies that .Net 4.0 and the upcoming new version provides.
I do not fully agree with you. Look at Windows, how many tasks does it run at any given point in time? It could be hundreds. It all comes down at design time to be aware of the trade-offs one has to make, task switching overhead vs. benefits from running operations in segregated tasks. I agree with you though that a single application that runs on too many tasks probably can be optimized and there is most likely something wrong.
Ok, I will bite. Those who know me knows that I am not a .NET person by any means. But for a large number of concurrent lightweight threads (with the computing threads and I guess the arbitrage opportunity seeking threads), I would think some language constructs like "coroutines" maybe close to the desired goal. I am currently using a customized version of coroutines (libpcl with stack jumps for anyone that cares) along with my thread pool library in production platform, and it has yielded significant performance improvements (over simple "green-ish" threads). I believe MSFT calls coroutines "Fibers" under the .NET, but that's about the extent of my knowledge there. For those of you who are not familiar with coroutines (other than the original Don Knuth's definition), lookup the Wikipedia entry. I think of it as a concurrency semantics free (as concurrency is implied in the "Yield" operation) cooperative execution, which, incidentally (!), works quite well for data processing, analytics generating, etc.
Re .Net there are plenty public sites out there that published performance results that show that task switching and generally running light-weight work on tasks rather than generating threads is always advantageous in terms of resource allocation and computational efficiency. I am not sure about the term "fibers" and thus cannot comment but my understanding of "coroutines" is a concept that was implemented through the arrival of task programming (at least in .Net). Note that most coroutine libraries date back to 2002-2003 when .Net was at 2.0. It is very easy for anyone to run a performance comparison by starting up n-number threads and have them operate as worker threads and then trying to solve the same problem using tasks. The new Async CTP framework is terrific because it is very logical and the beauty of it is that it requires minimal code changes though I always recommend to think parallelism/concurrency at design time, its not optimal to try to squeeze sequential code into a concurrent framework through code changes.