Just to round off the thread: I looked into the following that provide extensive historical 1-minute data for ETFs www.grainmarketresearch.com - from what I can determine, this data is timestamped beginning-of-bar, so problematic to covert for import to NinjaTrader www.pitrading.com - data is timestamped end-of-bar, + Pi Trading provide a free drag-and-drop ASCII-to-NinjaTrader data format converter that makes the whole process a breeze - the only drawback I can see is that the equity data provided covers only the standard trading day (09h30 - 16h00 EST), so it is not a source of historical data for the overnight equity session (but I am going to get the Pi Trading data anyway!). Anyone know of a good source of 1-minute historical data for equity prices in the overnight session (i.e. 04h00 - 09h30 EST, and then 16h00 - 20h00 EST)? Thanks.
Do you trade the pre/post market session? Other than news days, most markets (98%?) are illiquid and basically not tradable for any use other than academic. Unless you actually are one of the few to trade the off-session times, I wouldn't worry about including outlier data in your testing model. If you have a real-time data feed, do a time&sales on the symbols you trade and see what kind of volume your portfolio symbols generate outside the regular session..
I agree with everything you say above about the inadvisability of trading the pre/post sessions (my opinion is - unfortunately - formed also from personal experience as I tried to trade it for a few weeks once, with unhappy results!). All my (auto)trading is done during the core session. However, the reason I am interested in pre/post session data is that some of the signals and indicators I use incorporate data from the non-core session period. And to be able to backtest and research effectively, I need this data historically ... seems it's hard to get hold of.
There are some other options. You can get one minute data for the 50 most liquid ETFs for $250 from kibot.com. Their data contains pre/after market sessions and the regular market session. Also, if you select SSO only, you can get data since 2008 for only $5 from analytichouse.com.
For, say, AAPL, BAC, BGU, FCX, GS, JCP, SPY, SSO and MSFT: a) what do you have available? b) what is the source? c) what is the cost?
Thanks for the links. Look back through this thread. Someone recommended the stocks edition from pitrading. $159 for 1000+ stocks including 450+ ETFs going back to 2002. I looked at analytichouse, but they want $500 for 125 ETFs only going back to 2008. I guess they limit the dates to save download bandwidth. I am probably going to pick up the pitrading data next week after the holiday since they just released a new updates today.