I guess some of you know the news that Steven Cohen invested in Quantopian. I tried that platform before but i found documentation poor and lack of support. Then I noticed few reports of out-of-sample tests. Has anyone tried it and do you believe it is worthwhile to spend the time to get familiar with it as opposed to using a standard backtesting platform?
I've used it extensively, it is far more powerful than most backtesting platforms I've seen. You can certainly do out of sample tests, you pick the timeframe for every simulated run you do so simply save some out of sample timeframes and after you've perfected/overfitted your algorithm you can test it during another timeframe. It does have some pitfall/limitations: 1. You have to learn at least rudimentary Python. If you've ever programmed anything before it will probably be a couple evenings of self-study before you're up and running. If you've never programmed or don't have an engineering/math/cs background or aptitude this probably isn't for you. The good part is that the more you know the more features you can make use of, while a relatively new person can easily do basic algorithms. 2. It uses minute bars, so no HF trading. 3. No futures, options, or forex, just U.S. equities. 4. It's not designed for chartists, so if you're looking to implement "head and shoulders", "tea cup on a saucer" or the like you're going to need to come up with the underlying logic yourself rather than picking an option from a menu. I'd say the comment on a lack of support and documentation is very different from my experience. Every interaction I've had with them has yielded an intelligent response by a human, and I've contacted them probably a dozen times. Ironically I'm not paying them a cent and get far better responses than I get from some brokers who I pay quite a bit in commissions to. Their API documentation is better than almost any I've seen, although again you need to know Python or it will be all Greek. They also have a forum where users are surprisingly happy to help newcomers with code and share their ideas and backtests. Obviously few people show successful backtests, but knowing a bunch that failed is also helpful so you don't have to try them yourself. In general the level of sophistication of their users is also far higher than your typical trader bulletin board, so you don't end up with inane arguments about established facts like put/call parity, the definition of Beta, leveraged ETF "tracking error", or the impact of reverse stock splits on an ETN.
I see no advantage using quantopian as compared to Tradestation or NinjaTrader. I see a big disadvantage in fact because as they admit the reason they designed the platform was to put others discover profitable algos and profit after giving them a small cut. Only irrational traders or traders with no money to trade would use something like that. Besides there is nothing you can do there you cannot do in other standalone platforms.
I think you're mischaracterizing their business plan, they're all about the idea that they have no access to your algo's unless you grant it to them. Clearly they technically could break this promise, as could any other system. If you're paranoid then no backtesting system is for you, you'll need to code your own. Quantopian's Python implementation is quite a bit more powerful than something like NinjaTrader's scripting language, and there are certainly things you can do there that you can't do with other standalone platforms. If you're not going to need that and/or already have significant sunk costs in another scripting language then there's no compelling reason to switch. If you're coming at it with no previous investment in another system then it's certainly worth considering. I've found it very useful for backtesting, even though I actually code my own trading software to implement my live trading.
Some traders insist never to use any backtesting program that "calls home", even so a web based one unless you think you will never find any edge. In that case use anything free. It's not a matter of paranoia but IP protection.
I use quantopian and really like it overall. I don't fall into the paranoid camp, fearing someone's going to steal my Holy Grail. Particularly since I have no Grails to lose. The one thing is I absolutely detest goddamned Python but it's become so pervasive that I've had to bite the bullet and start using it.
If you're "some traders" then an online system like this will never be for you, as I indicated. There's really two camps, one feels like they've found "the answer", and they must protect this golden egg at all costs. If you're that kind of person, just ignore threads like this, write your own backtesting code, and protect that golden egg at all costs. And I wouldn't advise posting on sites like this, you don't need the brainstorming effect of all the ideas because you already found the answer and you just risk giving something away. The rest of us churn out new algorithms on a regular basis. A powerful platform that allows us to discover more working algorithms more quickly and at a lower cost (our labor) is far more valuable than the very small risk that someone is going to "steal" our "IP" and somehow hurt us. In the same vein, places like ET stimulate new ideas and are worth engaging in even if we do let slip a few trade secrets. In my view, if a rogue Quantopian employee stole all my algorithms tonight I'd simply move on to another platform, write some more, and be back where I was in a couple of weeks. I'd be more annoyed at the moving costs (having to learn a new system/programming language) than the actual theft. My way may not be better than the "golden egg" way, but I certainly prefer it.