Looks like Yahoo Download API is now Dead - 11/1/2017

Discussion in 'Data Sets and Feeds' started by hayman, Nov 1, 2017.

  1. aqtrader


    Thanks a lot for discussions in this thread. Although none of the solutions solved the problem I have, it gives me valuable clues to finally allow me continue automatically download from yahoo financial site a list of tickers in a format that can be easily converted into csv format. For those who know how to program, it is easy. Otherwise, please do not bother to read the following. Here is what I do:
    1. use your browser either google chrome or firefox;
    2. create a yahoo account, and log in, and create a portfolio. You can add upto 1000 tickers into each portfolio.
    3. create a view to customize the display for the portfolio. for example, you can select: ticker, open, high, low, last, vol, time.
    3. you can manually download the web page into a single html file by selection "Save as: Web Page, Complete". (Wait this is not automatic yet. I will tell how to make it automatic if you continue to read). In the browsers's address bar, you could see the URL such as - https://finance.yahoo.com/portfolio/<your-fortfolio-id>/view/view_<your-view-id>, copy it to be used for later automatic download.
    4. After you get the html file, use the program "html2text" (free available for Linux/Unix platform) or use lynx -dump <the html file>. You get the text format of the quote table. From there, it is easy to convert the table into a csv format or anything you like if you know how to write a script (python, or anything you like).

    To make the above download automatic, that is, write a simple program (robot) to do downloading by simulating the clicks and keyboard types. Use the free tool automatic-save-as (from github.com) and xvkbd.

    The problem I cannot use curl or wget or lynx is they do not support full web page downloading into a single web html file (or I still do not find a way to use them correctly) even I added cookies.txt to start programs.
    #61     Dec 5, 2017
  2. just21


    Last edited: Dec 5, 2017
    #62     Dec 5, 2017
  3. Riovero



    Why does not a hyperlink to the watchlist created in excel work?

    Thank you very much
    #63     Dec 10, 2017
  4. omardebar


    I am new to this forum. My problem is I wanted to use Yloader. I have download and install and all is fine. But I am unable to convert the ascii or csv files to metastock format. I am asking for help from this forum. Thank you and god bless.
    #64     Dec 15, 2017
  5. just21


    www.databull.com may still get data into metastock. If ascii will do then Amiquote http://www.amibroker.com/products.html has more free sources than just yahoo. Otherwise you will have to get free data from an api.
    #65     Dec 15, 2017
  6. It's still possible to get the end of day data from Yahoo finance.
    1. Timestamp has been changed to unix format. It's important to submit the right time for start and end period e.g. GMT 4pm. Do a historical download and analyze the unix timestamp in the url.
    2. The new API must include crumb code.
    3. Get request must be with cookie.
    Some helpful chap has posted an Excel VBA script on how to retrieve the crumb and cookie at StackOverflow. You can adapt it to your script.
    #66     Dec 20, 2017
    Riovero likes this.
  7. alchemy11


    There is another way to get this data without a lot of effort: .
    #67     Jan 3, 2018
    userque and Riovero like this.
  8. Riovero


    Not work (Excel 2016)
    #68     Jan 10, 2018
  9. aragorn


    Hi, there is a paid solution (payment range according to how many symbols you need and the frequency you ask for an update) no real time and not expensive...
    I'm using it to my dynamic data since yahoo... well you know

    I don't know if i can post here their name but if you reach me in private massage i can send their details
    Last edited by a moderator: Jan 24, 2018
    #69     Jan 24, 2018
  10. VirtuaPT



    For almost a year a friend (that used metastock 11) has been asking me to do a software to download free EOD data from yahoo (since mldownloader broke).

    On the start of this year i developed a web app for him and expanded so that he could get free data from quandl and tiingo (better quality). The web app will in the near future also support continuous futures via quandl. We though you guys here at ET might also enjoy it and thus i made qdowloader(dot)com. It's not commercial and i'm still thinking about it, but in the meantime it's 100% free and very usable, even if it is somewhat amateurish (i'm a finance guy not a programmer). Feel free to use it and maybe give some feedback.
    #70     Mar 17, 2018