I might have to eat crow here. The closer I look at the problem, the more I believe the bug is with NinjaTrader. (And it's a pretty significant bug, too.) Once I was able to get through to someone, IQFeed customer support staff has been excellent in tracing through the problem. I hope to know a little more by end of day today.
Yep. There is an unbelievable bug within NinjaTrader in its handling of ticks (at least from IQFeed), with *any* thinly traded stock. Unbelievable. Basically, NinjaTrader doesn't "finish" a bar until it sees a trade *after* the current bar... even if there are already trades in the current bar. Even if that means waiting 20 minutes, NinjaTrader will wait 20 minutes. Example, say these are the trades coming across T&S: 10:00:30 AM, $5.05 10:00:35 AM, $5.09 10:25:10 AM, $5.15 I think anyone writing a strategy for NinjaTrader would be expecting to receive a 1 minute bar at 10:01 AM, with low = $5.05, high = $5.09. Well... don't. You won't be receiving your 1 minute bar for 10:01 AM until 10:25:10 AM. You won't realize there have been any bars for this symbol until the *next* bar occurs.
Wow. Good thing that you caught this! I'm always amazed at all of the "late" prints that are "off" the market that come through the SPY's just after the close and continue on for 30-40 minutes . . . even with the DTN "IQ-Feed" that I use, and it often makes me wonder how guys who use automated systems handle this.
There is NO bug in NinjaTrader. If you want accurate bars based on time stamps from the data source... - You can only close a bar once you have the 1st tick with time stamp greater than the close time of the current bar - If you want to use your local PC time to determine when a bar closes then you will be guaranteed to have inaccurate bars since >> Your connection can have lag and thus your bars will be cut of prematurely >> Your bar more often than not will NOT include all of the ticks that make up that bar >> Your PC clock can be off which also determines what ticks may or may not make it into your bar >> As you then reload data into your chart from the historical data source --> Your chart will just look different NinjaTrader is not unique in this approach. TradeStation for example, will do the exact same thing.
Perhaps someone that uses TS can come in and verify if that's the way TS behaves. Your explanation as to *why* this design choice makes little sense. Just about any alternative seems better than the choice being made here. Why not close a bar after a user-programmed threshold? 30 seconds, for example? Give me the responsibility of sync'ing my PC clock. I'd even understand late arriving ticks being accumulated into the *next* bar... it's just luck anyways whether a trade occurred at 10:29:59 or 10:30:02 AM. I do *not* understand, however, bars arriving 15+ minutes late.
Tradestation does the same thing. It is as logical as it can be. There are a couple of workarounds, you can have your script watch your pc clock as a watchdog timer or you can build your own bars in arrays and apply your own rules to how bars are formed.. I've done both actually, I loved having bars in my own arrays, I never had any ambiguity or dissatisfaction with how they were formed... I do recall trying to program a MACD on bars I built in arrays.. it was the task from hell believe me.. so you trade off convenience for ambiguity, but known ambiguity.. Happy trading, I gave up autotrading and am learning to screen trade, I got more strategy ideas in the last two weeks on the screen than the last six months with historical stuff... I might actually come to know and love it... what's best for a strategy that autotrades and for a screen trader is very diverging sets of knowledge imo..
Let me just ping this thread again. After several other encounters, I just wanted to say I'm thoroughly impressed with IQFeed customer support. Once I found the right person to talk to, they've been incredibly helpful on various issues. Very solid business, and I trust them thoroughly at this point.