Python versus C++ Speed

Discussion in 'Trading Software' started by nononsense, Jan 29, 2006.

  1. Nonon,

    My point about threading is completely valid. And at this point of time, there seems a real probability of multi core processors becoming the norm. I don't particularly advocate the use of threads. Unix managed very well for a long time without them. Eric Raymond suggests that they became popular in the Windows world because process creation was abysmally slow rather than on their inherent merits.

    However, if you want to avail yourself of the extra processing power in multi core machines, you must use threads and the threading must be 'native' ie kernel threads. (Assuming you can't split the app into mutiple processes, which for better or worse is running against current programming paradigms).
    Python does not cut it for this purpose - at the moment. My expectation is that should this shift to multi core become fully confirmed the Python developers will probably address this issue. I'm also saying it will be a technically HARD job to reach the stage where Java is wrt threading. I'm not saying it won't happen.
     
    #41     Jan 31, 2006
  2. fatrat

    fatrat

    I don't think it would be hard to add that kind of threading in the pure technical sense. Maybe they just opted to make it user-space scheduled because they wanted it to run on every possible platform out there, including even some simpleton embedded procs that might not even support threads scheduled by a kernel. When you've got one process and you handle your own threading, you've almost ensured yourself total portability.

    I mean, think of what's involved to add threading -- they'd have to take care of a few synchronization issues and allow the thread contexts to be maintained by the kernel instead. Or, at least, their interpreter has to be aware of multiple threads and synchronize itself properly before interpreting a chunk of code.

    The reason it might be "hard" is because of all the different platforms out there and maybe there are some inherent design flaws with the way their interpreter handles the code.

    And if what No-Nonsense said is true -- if there IS a .NET version of Python, then that version can just make use of the .NET classes for instantiating threads. Right there and then you'd have a truly multi-threaded Python platform.
     
    #42     Jan 31, 2006
  3. Hi fatrat,

    Regarding python's threads you are correct, but see my last post.

    As for python itself.. I very much love the language (thanks again nonon :) ) but, here are my personal gripes. The biggest is that It should allow for voluntary type declaration. And the second is the iteration within "for/while" loops needs a lot of improvement. I spent alot of time trying to optimize as python's limits would allow. Now don't get me wrong! This is only in the sense of crunching very large data sets (6,7, and 8 figures). When I started looking into embedding C into it, I realized that for my personal desires, C in itself was the way to go.
     
    #43     Jan 31, 2006
  4. nitro

    nitro

    #44     Jan 31, 2006
  5. #45     Jan 31, 2006
  6. What nonon said is true, but I don't know whether you can assume you 'inherit' the threading capabilities of .NET. My gut feeling is that it may well not be the case. There is also Jython which uses the Java JVM I assume in a way somewhat similiar to the way Iron Python relates to .NET ie access to all Java libraries such as Swing and so forth.

    You really need to know the gritty details about threading before making any assumptions.

    And I think the native threading is much harder than you believe. It took years before Java managed to make a decent go of it. Of course crappy thread implementations on some of the operating systems it had to run on (eg older Linux kernels) didn't help.
     
    #46     Jan 31, 2006
  7. fatrat

    fatrat

    Well, if that IronPython project really does allow you to interact with the .NET framework, then I'd have a hard time believing you couldn't make use of the threading objects. But I will admit I have not delved deeply into whether they fully integrate with .NET, or if they are some kind of half-assed implementation. The threading object would get managed by the CLR, and would have a starting point related to some point in the IL byte code generated by the .NET python version. It seems do-able. The basic synchronization primitives would already be provided for someone because of the .NET framework.

    Also, I was under the assumption that modern kernels with modern threading support would be used to bring threading to Python. Once your kernel supports threads properly, it should not be more than a 1-2 year long effort to bring threads to an interpreted language. Then another year of testing to uncover issues perhaps. Understandably, if the standard function library for a given language is not thread safe, these issues have to be fixed. But beyond standard synchronization issues and managing thread contexts, there should not be a problem.
     
    #47     Jan 31, 2006
  8. Hi All,

    Obviously, the anti-Python party will always come up with any argument that seems to fit.
    Threading? For many years, threading has been extensively made use of by Python programmers. Python, like most languages fall back on Operating System sevices to accomplish certain things. Threading is on of these.

    If you really approach the hardware, you always can come up with language limitatons or "shortcomings". For applications closely touching hardware, only something like C will do. Nay, assembler. I kind of lost track of the exact figures, but in Unix some 2000(?) lines of code had to be done in assembler. XP (?) Don't say too much how today's languages are going to be shaped to meet tomorows architectures. SMP is around for many years now. Dual core is coming up. Software, except for servers, not much is available to make existing programs using these, except if such programs have been specifically designed to do so, most do not.

    Coming to suggest that Java or C# are the way to the future insofar hardware is concerned is plainly laughable.

    Python has changed a lot over the years. Probably few comparable open forums exist like Python's where continous discussion goes on about future directions. I have not read anywhere that it's going downhill yet, unlike what is said about other languages.

    You like Java, stick to it. You like C#, stick to it. (You got anything like scipy in these? for free BTW :D )
     
    #48     Jan 31, 2006
  9. fatrat

    fatrat


    Your understanding of user-space threads vs kernel-space threads is lacking. Consult books by Tanenbaum or Silberschatz regarding OS design to understand where your knowledge has fallen short.

    The Python interpreter is incapable of scheduling fibers on multiple processors. Fibers exist within 1 process, which [I'm guessing in this case] has only thread of execution. It would need to instantiate a thread that is managed and scheduled by the kernel. Only then can it take advantage of future architectures. Java and C#'s threads are scheduled by the kernel, and the kernel scheduler in modern OS's that support SMP are aware of multiple processors and cores to schedule those threads on.
     
    #49     Jan 31, 2006
  10. A couple of links

    http://www-1.ibm.com/businesscenter...featurearticle/gcl_xmlid/8649/nav_id/emerging

    http://www.blachford.info/computer/Cell/Cell0_v2.html

    You can find a gazillion more links by googling.

    Like most new products, there's a lot of hype and don't expect to find one of these in your desktop PC anytime soon, but the key point is that there is a move to multicore because of the difficulties associated with making single core processors with significantly higher clock speeds and with decent power consumption to performance ratios and reasonable cooling requirements.
     
    #50     Jan 31, 2006