NinjaTrader is definitely superior to OpenQuant. I tried, openQuant for 1 month and subsequently bought a 6months license. Was intending to use it with Patsystems. Problems i found: 1. Couldn't do reconnection if connection to Pats is dropped. 2. Couldn't get broker information. What a absolute waste of money. I am using ninja now.
The problem I've having with NinjaTrader is that if you use a stock indicator on a backtest it does not calculate the value of the indicator until the close of the bar. Nothing I'm doing is indicator dependent, so I didn't think this would be a major issue. However, lets say that you want to take whatever program you've whipped up and for the sake of entertainment, throw a SMA in the mix. Now, your SMA will only calc on the close, and so this has the possibility of throwning the whole game off. Consequently, I'm being forced to uniquely script each indicator to avoid this problem, and its chewing up all sorts of time. Does anybody know of a way to get around this issue? I'm ending up needing to walk forward test everything and this is also consuming considerable time. Isn't there a program in existance which can simply take a c# script or code sample and run it over ever data unit (bid, ask, bar, tick, spec, whatever) for backtesting? In rebuttle, I don't know if it even matters as backtesting is really a worthless exercise, but it would be so easy to do, it would seem that someone is offering this.
This is pretty standard from what I have seen and in my opinion it is logical to either use the (for example) moving average value for the prior bar's close, or to wait until the current bar is closed. Using the current value for a non-closed bar can lead to some very deceptive effects in my opinion. For example you can have a situation where with hindsight it appears a signal was given while in real time no signal was there.
I'm not exactly sure what you mean. Do you want to backtest strategies based on indicators, but want to use intermediate values of a candle? If your strategy is based on the 5 min candle, how could you get consistent analysis if you randomly took values from within that candle? I would assume that by definition, you would want to analyze at the close of every candle in order to be consistent. That being said, you can always create a 1-tick chart and run your strategy against that. Your indicators would be relatively useless, but if your strategy isn't based on indicators, then maybe that will help. I found NT's backtesting too constrictive and it wasn't multi-threaded, so I wrote my own backtesting software so that it could do exactly what I wanted it to do.
And you mention my other issue with the multi thread. As you say, you could take a single tick layout and backtest upon that which is basically the goal in order to not create an imbalance of importance for some trades over others (say the close vs those others in the bar)
That's what I was going to say. Backtesting can be useful to test out ideas and prototype, but that's about it. The read edge in trading is in context & trade management, both are difficult to program. Tradestation can use tick or 1minute data to peek inside the bar which is useful for fills that would occur mid-bar. But doesn't update indicator values mid-bar. AFAIK no one does this. Really the quicker you drop the indicators the better off you'll be.
I guess it depends on your system. I have extensively backtested my system, and the purpose is to give me confidence that I actually have an edge. Obviously, it won't tell me if the system will break the day I start using it, but there's no way I would be able to stick with a system through a drawdown unless I had confidence that it worked reliably before. However, I agree that risk management is one of the most paramount things in trading.
@cunparis, What do you mean by "Backtesting can be useful to test out ideas and prototype, but that's about it." Are people using backtesting for something other than testing ideas and prototyping? I'm not trying to be a smart ass, just wondering what else people would be doing w/a backtest other than the 2 things you mentioned.
Good question. The difference is subtle. Many backtest a system, optimize it to perfection, and then trade it. That's what I did. and it worked for a few weeks to a month and then fell apart. Many reasons for this: markets change, curve fitting, etc. Now I rarely do backtests. When I do it's something like "when X occurs, what is the price Y minutes later?" and I can test to see if X has an edge. If price is higher Y minutes later a majority of the time it has an edge. I'd then take this edge and incorporate it into my (manual) trading. I plan to post some backtests like this in my blog real soon. I've been testing some tools I've been using like the eminiwatch better sine wave & better momentum and I found zero edge, so I removed them from my trading and my trading improved. I need to go back and redo it and take screenshots to make the blog post, just haven't had time. So backtesting to test an idea is fine. Backtesting a "system" to optimize it and then run it with real money.. that rarely (never?) works.
I can tell you from experience all systems fall apart because the markets change. Continuously. And your excellent backtest results will have you think it's just a drawdown until you find the biggest drawdown you've seen and then you pull the plug. Which is often about the time it starts working again. Try forward testing your system on simulator for 3 months before trading it with real money. I wasted over a year backtesting & automating. Now I know that context is the most important part of trading and context is impossible to automate because it changes every day..