Here is a way to get FREE 1-minute data for the same time interval: http://www.elitetrader.com/vb/showthread.php?s=&threadid=161548&highlight=TDAmeritrade+free+data You need to be a TDAmeritrade client, and sign up for their Free API. I also have a WL-based utility that I wrote that scrubs the data to eliminate after-hours data.
That is a good idea. You should place an auction on ebay and see what the fair market value is for the data. I think I have tried almost every data source available and $110 appears very high for only 2 years of data in an unknown state with unknown reliability. Most professionals won't back-test with less than 4 years of data for intraday systems due to market cycles. You may also want to consolidate the symbols down to 1 or 2k based on average volume. The number of symbols included is skewed since most stocks are not tradable since they have no volume/interest. Also you can eliminate all the low price/penny stocks that barely have a pulse. hayman is also correct in stating you should remove all pre/after market data. I have yet to meet a pro trader who uses these sessions in their back-testing.
In fairness to Mr Xiao (whom I don't know and can't speak to the merits of), $100ish isn't much money, and saves you screwing around with an API.
Hi all, Maybe I should provide a little perspective to clarify what is going on. It might be helpful to people who like to quantify their tradings as much as possible. I was determined to trade only on statistics 3 years ago because I didn't think I could beat the market based on my own gut feelings and limited experices (I started trading since 1999). So I downloaded end of data from Yahoo. My specialty is computer science so I have no trouble to build my own backtesting platform to test all kinds of strategies (searching for EDGE). Then I found one really promising. So I traded with it only to find out that the open, high, low, and close data is extremely unreliable. I needed those prices also tradable with at least certain volume. I knew I need higher resolution data to compute such. So I tried to find 1 minute data. The closest one I found back then asked serveral grands. I decided to download them myselft. On 09/2007, I had about 6 months of data on hand and I finally was able to have a system realistic to trade with. I sticked with the system until 03/2008, I deployed a new system. The system performed slightly worse than backtested results, but far better than my trading records before. Then I decided to take care of my own 401K. I directly rolled over my 401K (about $50,720.48) in 07/2008. It is $106,001.62 yesterday. I just did a activity flex from Interactivebrokers and the result is attached for your information. Since I mainly consider daytrading, I won't even use any data more 9 months old. Trading is extremely simple and boring if: 1. You developed a system that is fully based on numbers and has a decisive EDGE 2. You use good money management (KELLY) You do need data to backtest things. I would say, my data should be sufficient if you are develoing a system trading stocks. However, this is NOT to say those data are complete and perfect. In fact, I missed serveral days during the two-year period. My suggestion to those who like to be on the quantitative track: focus on strategy development, backtests, and make your system torlerant to 'bad' and 'incomplete' data. The more complicated data your system can handle, the easier when it goes live. Now since I have those data on my harddrive, why not try my luck to see if I can get $100 risk free money ($10 for media and shipping)?
Someone wanted a sample look of the data stored. Well, they are stored in .MDF files (Free Microsoft SQL Express can read them). I strongly suggest you work with a database instead of flat files. But it is up to you. I can export format that you want, just let me know. I attach screenshot of how does the data look like in database. It has open, high, low, close, and volume.
I'll give you 50 dollars if you can put a few months worth of 1 minute data for the top 1000 volume stocks in ascii files and email it to me (compressed). I'm just looking to map my old trades. I used to use opentick for free but sadly it's no longer up. PM if price is right...