I am looking to download historical stock data for all stocks on NASDAQ and NYSE to my computer. I plan to dump all this data into a database like SQL Lite. I also want to have an automated way to keep this database up to date so I need a service similar to OpenTick which I can use to download the data (preferably for free). I want the last five years of data, but I only need basic information such as high, low, close price, open price, volume, etc...information needed to run the top 20 or so technical indicators. Does anyone know of a preferably free service to download this historical stock data? I am starting out so I don't want to pay for anything until I at least have a system up and running which has demonstrated some success. The ugly way is to hack the Yahoo Finance connector that Eclipse trader uses and use that to download all my data. This would certainly be better than downloading CSV files manually and importing, but the interface that Eclipsetrader uses for Yahoo is not official by Yahoo and could fail to work any day. So does anyone know where I can download free historical data through a programming API. I plan to program in Java.
Any ideas on where else I can go (besides hell) to ask this question? Is there a place that is oriented more towards TA programmers?
It's not a big deal to write a bit of Java to get historical prices from yahoo, or use the Eclipse thing if you prefer. Yahoo seems pretty stable and they tend not to change things without good reason. If it does change, it shouldn't be a big deal to change your code to deal with it. I use a bit of Java code to download historical prices for all US stocks into a MySQL database and it's quite reliable. You can speed things up by issueing concurrent requests, probably in separate threads. I use 15 and it works well. I also use OpenTick for historical intraday data.
You didn't say whether or not you were looking for intraday data or just end of day. I know you mentioned free data but if you can afford about $30/month and are looking for end of day, check out TC2007 from Worden Bros. www.worden.com You can download the entire US stock market every day. In addition they have a SDK kit that comes bundled with the software that you can use to access all of the data from your own third party program. I program in Delphi and have no problem accessing the data, and it is pretty fast (after the initial slow startup).
I went with suggestion and I hacked the Eclipsetrader Yahoo Finance driver to fetch stock quotes by ticker symbol and time period. Right now, end of day data should be sufficient for what I am trying to do. If I end up needing more than that, I will probably get a pay OpenTick account. I think they only charge $9 a month for all NASDAQ quotes, but it's the hassel of setting up an account. Anyway, now I need to figure out a solution to store the data, I will probably use SQLLite JDBC since I use it at work for C++ development (http://www.zentus.com/sqlitejdbc/).
Why don't you just use opentick? I download all intraday tick data at the end of the day for equities. It takes a few hours so I do it overnight then I join the data the next morning (cause you unfortunately can't download multiple exchanges together) On a side note, I don't believe you'll find any successful strategies just throwing on the top 20 technical idicators.