Good C++ code will likely run much faster than anything else (except for maybe assembly or bytecode). Does not mean that everything should be written in C++, IMHO. I found that prototypes become live trading models and stick around, so "prototyping language" should be chosen very carefully. Personally, I'd go for Python, especially considering how widely it's being used in finance these days. This is very dated (started in 2008 and the thread was closed in 2012). Since then, humans have developed higher cerebral functions
Of course not. I think I said "likely." There are never absolutes. That's the 'fault' of the coder. If you are one that finds you 'likely' won't follow through with your plans, then you should plan accordingly. That's fine, as long as you are sticking the just libraries and doing exactly what everyone else is 'widely' doing in finance these days (and getting the same results) ... but nothing custom--not a lot of loops. Otherwise, it's super slow. I would respond here ... but it would 'likely' be a political post.
It can help ... if your code 'fits' what Numpy is designed to optimize. And how does Numpy speed up python? With code written in a faster language. Why not just cut to the chase? Rather than Numpy, Cython, etc. Python will allow for fast prototyping, however. Again, it all depends upon what your doing and how many loops it takes to do it. See also, for example of the research that can be found on the interwebs: " ... Conclusion I eventually ported over this entire algorithm to C++. Given its highly iterative nature, overall performance sped up over 100 times. NumPy is an amazing numerical library, however it has its limitations. It is optimized to work on larger data sets, but sometimes all you have is small data. Sometimes you have iterative algorithms that don’t nicely translate into the NumPy way of doing things. ..."
numpy/pandas are nice but not a panacea. I really wish they would have put more thought into their time series logic and such, borrowing some ideas from the existing time series languages like Fame. Also, some of the things are pretty clunky - for example, multi-dimensional arrays are still very poorly developed (though xarray is supposed to fix all that). Most of my research is done without loops using vectorized logic, that's where numpy/pandas really shine. I really wish they would add an ability to apply a path-dependent function (same way they have added "rolling" and "ewm") so I can do things like applying hysteresis without loops too. This said, if anything I am doing really requires "speed" at the execution stage (which is mostly reserved for the layer working the orders), I hack it in C++. Even at my amateur level, it comes out reasonably fast.
I use Numba myself for some of the more intensive stuff but obviously Numpy is a common component via Pandas. Obviously for very intensive computing Python isn't the best choice, the development speed more than makes up for it though, IMHO.
That looks pretty exciting. I might actually write something in it tomorrow to try it out! At the very least, I can move all of my option pricing and implied vol calculations to something like that - looks more friendly then cython.
numba is pretty nifty, I just installed it. A simple test: Code: import numpy as np from numba import jit @jit def hysteresis_numba(s, h): s_ = np.zeros(len(s)) s_[0] = s[0] for i in range(1, len(s)): s_[i] = min(max(s_[i-1], s[i]+h), s[i]-h) return h def hysteresis_plain(s, h): s_ = np.zeros(len(s)) s_[0] = s[0] for i in range(1, len(s)): s_[i] = min(max(s_[i-1], s[i]+h), s[i]-h) return h n = 100000 s = np.random.uniform(-10,10,n).round(0) h = 5 %timeit hysteresis_numba(s, h) %timeit hysteresis_plain(s, h) produces striking results: Code: 1000 loops, best of 3: 210 µs per loop 10 loops, best of 3: 97.5 ms per loop
I see people recommending C#. I absolutely agree with them, just want to add more specifics: 1. You will get more luck looking for a job if you stick to Microsoft technologies, like it or not. Corporate/finance mostly use MS products, they have tons of legacy code to support and dozens of never-ending internal projects. From my personal and my friends experience, people with MS background often get paid more for the same task. Especially when it comes to business/finance solutions. 2. C# is really fast, sometimes maybe even faster than C++. It may sound superfluous, but there is the reason: C# code gets compiled to a specific platform on the very first launch, not when you build the project. It means that it gets optimized for the exact hardware configuration, not for some abstract safe one. I developed some pieces of unsafe code which I share between all my projects which enable me to reach speeds that wasn't possible to get when I used C++ 10 years ago. Yes, I have enough C++ background to compare, I developed really complex and time-critical applications like SCADA and satellite communication systems in the past. My previous business (keyword research) was built entirely on C#, I was able to do full-text search of 4+ billions of keywords in 20ms. It's about 70Gb of data, by the way. Just reading a file of this size from a cheap HDD will take over 20 minutes, not even taking into acoount the full-text search itself. 3. When you see slow C# apps, it's not about the language itself, it's about algorithms or third-party libraries without optimizations. Or some resource-hungry visual things, like WPF. It's also a common mistake to link "fast" C++ library without taking into account that there are some costs involved when you do cross-boundary calls. There is always a room for speed optimization, when you really need it. I'm not a fan, I optimize only bottlnecks detected by the profiler. It alone enabled me to serve 3,000+ customers from a single server silently eating 60W/hr near the fridge at my kitchen, while my competitors paid up to $100,000/mo for "smart" AWS solutions. 4. The software development environment is great. It's the main reason why I'm stuck there. I like Visual Studio and I literally love ReSharper. The speed of software development/testing/optimization/bug fixes is enormous. I was able to develop and polish a full-featured backtesting tool with charts/data/simulation/drawing tools and indicators in about 3 months. Yes, it took me 20+ years of previous experience to do it, but anyway It's always a pain when I'm forced to use some other languages or IDEs. Not because I haven't got used to them, but because they lack so many obvious features that it's hard to convert. When using C# in VS+ReSharper, I don't "code", I just imagine what do I need and the code just magically appears on the screen 5. There are plenty of ready-to-use libraries and solutions in NuGet. Don't invent the wheel, just google anything you need first. At 99%+ of cases you will find that somebody somewhere had the same problem, which resulted in a great solution. Even if it's not in NuGet, Stack Overflow have enough code samples to copy/paste and see if it works, or not. 6. You can use the same language and IDE to develop any type of product: Windows, Mac, iOS, Android, online or combine them. They will share the same libraries, so you don't need to write, debug and optimize the same code again and again.
Well, I have earlier had a hunch that one of the problems with loading the data is that my tables are color coded using VBA and not using the native built-in formatting in Excel. This issue have been raised by me in the past and I even suggested removing formatting completely, but since I didn't get a reply, I figured that wasn't the case. But, this programmer once again complained that color formatting does take up resources. Really? So, I suggested that they remove the VBA code and that I format my own sheets using native built-in rules in Excel. Voila. Speed is improved 10 fold and I have full control over formatting myself. Incredible that they have been spending days trying to improve speed and couldn't figure out this on their own. Anyway, my software is working to satisfaction now.