I have a portfolio of 4000 stocks and am looking for a website or software (preferably free) that I can use to download the OHLCV of 4000 stocks at the end of the day. I need them to be able to be downloaded in .xls or .csv format. I've tried a number of free sites, but they usually have a limit of 100 symbols. If I can find a site that allows at least 500-1000, then that might be acceptable if I can build a URL for the download, then I could do it programatically with Excel. I have Redi, but there's a 1024 effective symbol limit and it would be tough to do the download programatically. I also have Tradestation, but they're limit is 2000, also tough programatically to grab. Any suggestions?
you can also just do a small VBA script to do 100 or so Yahoo finance requests for their CSV formatted data.
There are bulk downloaders that will get historical data from yahoo finance. But it seems wasteful to download the full set of historical data ever day. eoddata.com lets you download end of day ohlcv data for a large number of stocks for free. You could update a fairly simple database daily and always be up to date after doing an initial historical fill from a free source like yahoo.
if eod data is available directly from exchange website, in my view its a better idea to get data from exchange website rather than any other source like yahoo etc.
better to have a data from few sources,so one can compare and use it for a cross reference. that's what i do. for EOD i use IQ and yahoo. keep in mind that most likely you won't be able to get any data from exchange website. try it with nasdaq.com(for example). you will be banned after few requests. yahoo on other hand allow you to get the data for 200 tickers at once. downloading eod data for let say 1000 tickers takes probably 5 sec or so.
Thanks for all the recommendations. eoddata.com will do what I need for free, but its data is unavailable til 9pm EST and I'd rather have things done sooner than that. I can get it at 5pm, but then it's a pricey monthly subscription. I've found http://www.yloader.com/ which will do it for $25 (after a 15 day free trial) and I can call it with command line parameters from another program. And it's quite fast. For $25, it's worth it to have my program call it. This is also good: http://www.gummy-stuff.org/Yahoo-data.htm It showed me how I can use something like this: finance.yahoo.com/d/quotes.csv?s=XOM+PFE+WMT+TGT+LLY+BBY+JNJ+MSFT&f=sohgl1v to download what I need, but I could only do 200 symbols at a time. I need to investigate the gummy stuff SS's more. While I like yloader, I'd rather have a solution that I can tweak on demand in case Yahoo changes anything. I can't afford to be waiting for a software developer to make the change on their schedule.