I have started to rewrite the guts of my back-testing website in Go and I am blown away by its speed! I will be creating Go packages for each piece of functionality and will be open sourcing them. The first package I am going to write is the marketdataloader package. It will handle CSV data for now but other data adapters could be added. The data loader will be able to load multiple time frames of data (having them linked together) as well as market events (i.e. OPEC cuts production, etc.) and return the data in such a way that you can easily create complex backtest conditions. Stay tuned! fan27
At the point where you're using either Go or C++, their speed is pretty heavily dependent on how you use them. (I've used both languages for projects.)
Go vs C++ http://benchmarksgame.alioth.debian.org/u64q/compare.php?lang=go&lang2=gpp Based on the link above, Go is very competitive with C++. Go vs Python https://benchmarksgame.alioth.debian.org/u64q/compare.php?lang=go&lang2=python3 Assuming the benchmarks above are accurate, Go totally destroys Python in terms of speed. ------------------------------------------------------- I have used C/C++ minimally, C#, Javascript (Node.js) to name a few. Go is so much simpler to learn/understand than C/C++. Give it a try!
Hmmmm Do you write loop-based simulators or event-based ? I tend to believe that backtest speed is more the result of good architecture over language used. Go is very bad in binary tree so you will have a very hard time crunching using half machine learning techniques available.
Good to know wintergasp. To reproduce what I have in my website I just need to read data into arrays, calculate indicator values and then loop over the data. I have not advanced yet to any machine learning techniques. One option with Go is wrap C code in Go. What language(s) do you use for your backtesting needs? Thanks! fan27
I really like golang. If it had a native decimal type, it would be even more convenient for writing financial applications.
Is it like a public website you have ? (If so whats the url) I have a similar setup where i use a web app for the front and multiple runners written in c# that just sit in the cloud and wait for backtesting tasks, then backtest it and return the result. I backtest about 16 years of intraday data in c# under 300ms, with about 4000 trades total so it would porbably be longer for hft simulations I used to do that in 2 minutes but just through architectural improvements i got down to 300ms
Go is not the fastest on the block, but it is a new language that refines many paradigms at once. It can be said to offer best of many worlds: UNIX philosophy, C, C++, garbage collection, concurrency-support, safety, packages, interfaces with no/loose bindings, strict typechecking, multiple composition instead of multiple inheritance, "one way to do it", standardized & automatic pretty-printing of code, using Uppercase to export/make public, minimalistic dependencies, standard built-in libraries, one binary, open source, coherent community, etc. It'll take a while to explain all the benefits and language choices, so better research and experience yourself. If you want to code like in Java in Go, you'll be miserable. However, if you want to code similar to C/C++, the language is much simpler, cleaner and prevents many typical ambiguities at runtime. It does away with complications like templates/generics (for now at least), inheritance (which class methods are active now?), multiple inheritance (which parts of the method class-tree is active now?), polymorphism by virtual method calling (which type is this variable acting as here again?), plus some more feature bloat mature programming languages can't seem to avoid implementing by default. More importantly, the language compiles most things almost instantly and feels like Python or Ruby, though executes much faster and usually uses way less memory. On the downside, Go do not offer exception-handling in the way it is convenient in Java to delay fixing errorhandling by tracing the entire call-stack, but which also means code execution usually become exceptional in Java, alas faster/cheaper/easier to deal with for the developer. You'll also need to code more boiler-plate code in Go for things like iterating over containers/arrays, stuff, which in more dynamic languages mostly is abstracted away. Some of this you can choose to do in Go, but then you'll need to write your own libraries. You get more control over memory layout of structs (ie. byte-alignment and order like in C/C++) plus actual code execution, so you can avoid bloated design patterns and lazy usage of suboptimal algorithms. The Go language is now frozen at version 1 with the developers working on improving performance, GC, tooling and community, not feature-creep and junk. In terms of performance Go is in the middle of the bunch: C < C++ < Go < Java < Python / Ruby It's possible to link C-programs with Go in various ways. In terms of memory, Go is not smallest either, but may be compared to C/C++ depending on careful usage, though not really made for the most minimalistic embedded systems. Of course you won't automagically code great programs using Go right away, but being a new best of "all" worlds language, you might benefit anyways before you learn "The Go Way". I don't think even the developers are quite there yet where they know how to do everything themselves. Indeed, they emphasize thinking for yourself rather than blindly following the "nuggets" of "design patterns" and awful language abstractions/workarounds like FactoryFactory and Singletons. So for projects that aim to simplify, I think Go and its developers' careful and deliberate way of rational thinking may be worth investigating and implementing, also for your own projects. As for raw speed, that is possible too in a way that is often out of reach of interpreted and even some VM language implementations. I'm thinking here of raw computation, something you'd usually want to do in C/ASM (fastest) or C++. There are even ways to use Go's ASM directly with your Go programs, as this is even used by some of the built-in standard libraries. As for all types of optimizations, you should be lazy about them, delay their conclusion, so you free up time and brainpower to improve innovation, designs and organic development. But it do help that raw power is so easily available under the hood. Prefetching, precalculating, caching, etc. may always be used where raw cpu-power falls short, but always come with their own limitations, complications and costs.
Those are some pretty good stats! Here is my site: http://fasterbull.com/ It's current architecture is not very good but building it was a good learning exercise and while the strategy portfolio feature is somewhat crude in its implementation, it does offer features that I have not seen in other web based products.