Yahoo Historical Data - Did they change the URL recently?

Discussion in 'Educational Resources' started by Rationalize, May 16, 2017.

  1. bellman likes this.
  2. just21

    just21

    Change to https
     
  3. With https -

    Did you try it before posting your reply ?

    Because suggesting a fix that hasn't been tested is kind of douchy :p
     
    Last edited: May 17, 2017
    SigmaX and bellman like this.
  4. Wow, this affects me also somewhat. It looks like the new URI structure requires a cookie "https://query1.finance.yahoo.com/v7/finance/download/COF?period1=1492408787&period2=1495000787&interval=1d&events=history&crumb=(cookie goes here)/Q"

    They also have a new date format which is not making things any easier. This "update" was designed to stop people from downloading and scraping data. You can still manually download the data by visiting each symbol's page and clicking "Download Data".

    If you tried to scrape the data directly from the html then you still have to deal with this new cryptic date format that does not make any sense at first glance. This was designed to keep people from scraping without a doubt.

    WTF is this date format? edit: Update I think it is the Unix Date Format?
    {"date":1492435800,"open":36.349998474121094,"high":36.4900016784668,"low":36.0099983215332,"close":36.25,"volume":1825100,"unadjclose":36.25},
     
    Last edited: May 17, 2017
    bellman likes this.
  5. bellman likes this.
  6. etfloyd

    etfloyd

  7. Hmm. Looks a bit different. You see the same structure?
     
  8. The new URI structure I gave you is correct. The old one that you posted wont work anymore. The data in the html is a JSON object. It can be scrapped. You will just need to recode your downloader/scraper to convert a regular date to a Unix Timestamp. You won't be able to just plug in a URL into your browser anymore and have it automatically download. You will need automated tools to use the new URI structure and scrape/format the JSON objects.
     
  9. Previous was 15 years of daily history in csv format - looks like i have some work to do.

    Thx for the info. Appreciated.
     
  10. You can still get a .csv, I think the date format will be different. The only way to get a .csv is to download it manually from the yahoo Historic Data Page located within the same page for whatever symbol you are downloading for. Parsing the JSON is no big deal, I am sure there is code online to convert a Unix Timestamp to YYYY-MM-DD.

    I'm going to rewrite my downloader for a different site. I'm done with yahoo.
     
    #10     May 17, 2017