Scraping is slow and wasteful. But well written code for scraping can handle some changes in code. I hope you realize that if you only update the latest day, the past data won't be split or dividend adjusted. Google's data is full of missing days and it's split but not dividend adjusted.
I used Yahoo finance to pull certain data from them (stock price, dividend rate, dividend per share, and dividend pay date) I did this by using some code in VB in Excel that would allow me to update the spreadsheet automatically anytime I hit update all. Is there another site I can pull this same data into my excel spreadsheet? Preferably something that is free. Do you guys have any recommendations?
Why? https://www.elitetrader.com/et/threads/bring-back-yahoo-api.313050/page-2#post-4517956 (hint: data is crap)
I have no recommendations for on-demand web requests for live or delayed quotes to replace what Yahoo or others do. I use NxCore for my live intraday feeds and have a second feed of Norgate data for daily timeframe data. I supplement my systems with cross-checks from a number of sources. Why? Many events cause big changes in the prices of stocks. If those events are just due to normal capital changes/dividends to a stock, I need to know that. For example, I need to know if there is a split or dividend that means I need to adjust my stops. The quality of Yahoo and Tiingo is not good enough... Too many errors in their systems that generate false positive events that take too much time to validate (mostly invalidate).
No problem, it can even be automated if you really want, but that involves working with the command line and a download utility called 'curl', it's a bit fiddly. With freebie data, it wouldn't be a bad thing to have two different sources worked out, then if one changes or disappears it doesn't leave you in the lurch right away.
Actually curl works with alot of programming languages. No command line required . I use it with c++. The library is libcurl. There are other links that yahoo has that gives pure json data. I found one that requires no scraping or cookies. I have been wanting to post the link that I found and use but I worry if to many know yahoo make take that away also.
Yeah, have a backup source is good. I will probably build a backup using my IB or TD Ameritrade feeds. FYI, there's an old command line utility called WebFetch.exe, which you can probably hunt around for, which could assist in the automation within a command line script.
There is a fix for pandas data reader that scrapes the data from yahoo finance for you and stores it in a data frame. I've used it to acquire some data and it seems to work fine. Please find the link below. https://pypi.python.org/pypi/fix-yahoo-finance