What Analytics Environment?

The first step in comparing, selecting and/or designing trading systems, “expert sources” and indicators would seem to be some analytics as to their performance over time, over different instruments and in various market conditions. My question is how to best do those analytics?

Trading environments such as tradestation, ningatrader, amibroker and metastock do a good job showing you an indicator on the screen and giving you buy/sell recommendations from a strategy. They will also give you some level of back testing capability, abet limited. Tradestation in particular is very limited in that it can’t even back test a group of stocks, compare a set of strategies or support rotation. Yet TS seems like the best supported environment (I had hopes for OOEL, but it also is very limited).

I can dump data out of these and do some analytics in Excel, but that can be very slow and clumsy. For example it would not be easy at all to compare 2000 stocks over 10 strategies. If I want to answer questions like “what is the probability of a gap being closed within 2 days” and “what are the probabilities of the gap getting wider by various percentages prior to gap closing”, etc. The trading platforms are simply not set up for this kind of analytics and back testing is insufficient. I would also have to do some major excel programming in VB to support such analytics.

The analytics environment would seem to be a good place to explore optimizations and alternatives and may include genetic algorithms, etc. On the other hand, I would really prefer to have the indicators and systems all defined in one environment – reproducing formula in 2 environments would be time consuming, error prone and expensive.
So the options I am looking at include:
• doing all the formula in a trading environment (probably TS) and exporting data to excel.
• Having formula in both the trading environment and excel.
• Using Excel as the primary environment (i.e. AnalizerXL).
• Buying/Learning MATLAB or "R".
• Extending ningatrader to provide some of these capabilities (since it is based on a general language)
• Using some kind of business intelligence (i.e. pentaho) or data mining (i.e. rapidminder) package.
• A “technical” solution would be to do formula in a DLL which could be used by multiple environments, but that could be a lot of work.

I don’t mind writing some code but don’t want this to be a life’s work either. So my question is: what is the best way to do analytics in support of selecting and/or designing trading systems? The group on this forum would seem to be the kind of people that would have some answers to a common need.

Thanks in advance for your thoughts!

 

I use MS Excel to do back testing and it is not slow, actually I can perform an analysis of a simple trading system in less than 3 hours with over 500 data points providing at most 23 test trades; complete with expectancy and trading statistics.

You can read my tutorial here: <a href="http://www.stock-trading.me/2010/11/how-to-perform-free-backtesting-of-stocks-using-ms-excel-the-easy-way/">Using MS Excel for backtesting</a>.

I find this more convenient plus MS Excel is common in most Windows computers. So if you are new to trading, this is a cost efficient
way to start and analyze your trading systems without the need to purchase expensive back testing software.

 

If you work with daily charts, the best backtester is Tradingblox. It's not as good with intraday, as you have to finesse the data to fit the program.

 

Writing code and random testing is a thing of the past. Machine generated strategies is what most professionals exploit. Instead of coding you can concentrate on testing the various startegies for an edge.

http://www.elitetrader.com/vb/showthread.php?s=&threadid=207633&perpage=6&pagenumber=1

 

What do you mean by probability? Correlation? Portfolio optimization? Maximum likelihood?

What programming language(s) do you already know?

R is not a bad choice. As mentioned already, Excel is not that slow. For the non-programmer, I would recommend either Excel or MiniTab. If you have some knowledge of programming, R is probably the way to go. I also use Weka (Java) and have used MatLab in the past.

 

http://www.jsoftware.com/. The new J7 is coming up and they mostly likely be moving to an open source model. The language will look daunting at first, but it's really quite simple if you're mathematically oriented and can grasp a few concepts.

 

Thanks for the comments!

In reply: Sure, backtesting a strategy is important. But to create such a strategy I would like to first analyze probabilities and patterns, something backtesting doesn't do well. Once those probabilities and patterns are understood I would be in a position to create a strategy. There are SO MANY opinions on what "works", I want to test and build with confidence. I am very interested in how combinations of factors could influence a strategy. What I am mostly looking at now are the volume centered approaches.

For example, I saw a great post on a sector rotation strategy (http://www.copstrat.com/) , I would love to see for myself and try some alternatives – doing so should be easy, but is not. Do to market changes, strategies must change and “compete” as well over time – doing the analysis for that would be a bit of a project.

As for me: I know multiple programming languages, most active with Java of late but don't feel any reasonable programming language is a barrier for me. However, I don’t think I should have to be writing much code – I would think the statistical environments must exist, something where I can import and work with data (kind of like business intelligence) more than writing code, but I have not found such a thing yet – at least not for cheap. I have looked at R a bit, but such an environment doesnl.t really seem to exist – it is more of an interactive programming IDE. I would prefer something a bit more general and mainstream than R, but it is a contender. While I am not math stupid, I don't have the statistics background that R seems to expect. There are some statistical capabilities with Python that look interesting as well. Of course MATLAB seems to be the “market leader” for $. I do think it is important to be able to build on the work of others, here MATLAB and EXCEL are the front runners.

When I have tried to do this with Excel I feel frustrated with having to treat everything as a “sheet” and the very crude capabilities for doing anything with a data vector. I tried to get it to do a surface graph of time/price/volume and it never seemed to work (perhaps due to a bug). That said, I am not an Excel guru and perhaps not using it well. Any of these require a substantial investment, even for someone with a professional software background.