Looks like Yahoo Download API is now Dead - 11/1/2017

Discussion in 'Data Sets and Feeds' started by hayman, Nov 1, 2017.

  1. aqtrader

    aqtrader

    Thanks a lot for discussions in this thread. Although none of the solutions solved the problem I have, it gives me valuable clues to finally allow me continue automatically download from yahoo financial site a list of tickers in a format that can be easily converted into csv format. For those who know how to program, it is easy. Otherwise, please do not bother to read the following. Here is what I do:
    1. use your browser either google chrome or firefox;
    2. create a yahoo account, and log in, and create a portfolio. You can add upto 1000 tickers into each portfolio.
    3. create a view to customize the display for the portfolio. for example, you can select: ticker, open, high, low, last, vol, time.
    3. you can manually download the web page into a single html file by selection "Save as: Web Page, Complete". (Wait this is not automatic yet. I will tell how to make it automatic if you continue to read). In the browsers's address bar, you could see the URL such as - https://finance.yahoo.com/portfolio/<your-fortfolio-id>/view/view_<your-view-id>, copy it to be used for later automatic download.
    4. After you get the html file, use the program "html2text" (free available for Linux/Unix platform) or use lynx -dump <the html file>. You get the text format of the quote table. From there, it is easy to convert the table into a csv format or anything you like if you know how to write a script (python, or anything you like).

    To make the above download automatic, that is, write a simple program (robot) to do downloading by simulating the clicks and keyboard types. Use the free tool automatic-save-as (from github.com) and xvkbd.

    The problem I cannot use curl or wget or lynx is they do not support full web page downloading into a single web html file (or I still do not find a way to use them correctly) even I added cookies.txt to start programs.
     
    #61     Dec 5, 2017
  2. just21

    just21

    Last edited: Dec 5, 2017
    #62     Dec 5, 2017
  3. Riovero

    Riovero

    Hello:

    Why does not a hyperlink to the watchlist created in excel work?

    Thank you very much
     
    #63     Dec 10, 2017
  4. omardebar

    omardebar

    I am new to this forum. My problem is I wanted to use Yloader. I have download and install and all is fine. But I am unable to convert the ascii or csv files to metastock format. I am asking for help from this forum. Thank you and god bless.
     
    #64     Dec 15, 2017
  5. just21

    just21

    www.databull.com may still get data into metastock. If ascii will do then Amiquote http://www.amibroker.com/products.html has more free sources than just yahoo. Otherwise you will have to get free data from an api.
     
    #65     Dec 15, 2017
  6. Peter8519

    Peter8519

    It's still possible to get the end of day data from Yahoo finance.
    1. Timestamp has been changed to unix format. It's important to submit the right time for start and end period e.g. GMT 4pm. Do a historical download and analyze the unix timestamp in the url.
    2. The new API must include crumb code.
    3. Get request must be with cookie.
    Some helpful chap has posted an Excel VBA script on how to retrieve the crumb and cookie at StackOverflow. You can adapt it to your script.
     
    #66     Dec 20, 2017
    Riovero likes this.
  7. alchemy11

    alchemy11

    There is another way to get this data without a lot of effort: .
     
    #67     Jan 3, 2018
    userque and Riovero like this.
  8. Riovero

    Riovero

    Not work (Excel 2016)
     
    #68     Jan 10, 2018