Yahoo Finance V7 API now requiring cookies? (Python) – Python

The Solutions:

Solution 1: Using HTTP cookies

1. Make an HTTP GET request to https://fc.yahoo.com to obtain the set-cookie header.
2. Include the obtained cookie in the subsequent HTTP GET request to https://query2.finance.yahoo.com/v1/test/getcrumb to retrieve the crumb value.
3. Replace [crumb-value] in the URL https://query2.finance.yahoo.com/v7/finance/quote?symbols=TSLA&crumb=[crumb-value] and make an HTTP GET request with the cookie.
4. Cache the cookie and crumb values to optimize future requests.

Solution 2: Fetching Cookies and Crumb

To resolve the Invalid Crumb error, a cookie and crumb are required for Yahoo Finance V7 API requests. This Python code demonstrates how to obtain the necessary elements:

The get_yahoo_cookie() function fetches the cookie by sending an HTTP request to "https://fc.yahoo.com" with a specific user agent header. It then extracts and returns the first cookie from the response.

The get_yahoo_crumb() function, using the obtained cookie, sends a request to "https://query1.finance.yahoo.com/v1/test/getcrumb" and extracts the text response as the crumb.

The retrieved cookie and crumb can be stored and used in subsequent API requests to prevent the Invalid Crumb error.

Solution 3: VBA with crumb implementation

The following VBA code retrieves the crumb and cookie required for the Yahoo Finance API using MSXML2.XMLHTTP60 object.

Public crumb As String
Public cookie As String

Public Sub YahooGetCrumb()
    Dim http As MSXML2.XMLHTTP60
    Dim strHeader As String
    Dim strFields() As String

    Set http = New MSXML2.XMLHTTP60

    http.Open "GET", "https://fc.yahoo.com"
    http.setRequestHeader "User-Agent", "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36"
    http.send

    strHeader = http.getAllResponseHeaders
    strFields = Split(strHeader, vbCrLf)
    cookie = Trim(Split(Split(strFields(5), ";")(0), ":")(1)) & ";" & Trim(Split(Split(strFields(6), ";")(0), ":")(1))

    http.Open "GET", "https://query2.finance.yahoo.com/v1/test/getcrumb"
    http.setRequestHeader "User-Agent", "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36"
    http.setRequestHeader "Cookie", cookie
    http.send

    crumb = http.responseText

End Sub

Public Function GetYahooData(sSymbol As String) As String
    Dim http As MSXML2.XMLHTTP60

    If crumb = "" Then
        YahooGetCrumb
    End If

    Set http = New MSXML2.XMLHTTP60

    http.Open "GET", "https://query2.finance.yahoo.com/v7/finance/quote?symbols=" & sSymbol & "&crumb=" & crumb, False
    http.setRequestHeader "User-Agent", "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36"
    http.send

    GetYahooData = http.responseText

End Function

Public Sub test()
    Dim JSON As Object

    responseText = GetYahooData("AAPL")

    Set JSON = ParseJson(responseText)
end Sub

This implementation stores the crumb and cookie in global variables for subsequent use, and also includes a test function to retrieve and parse Yahoo Finance data for the symbol "AAPL" using the ParseJson function from the VBA-JSON project.

Solution 4: Using Javascript to Extract Credentials

This solution utilizes JavaScript to obtain the necessary cookie and crumb for making Yahoo Finance API V7 requests. The process involves two steps:

1. Visit https://fc.yahoo.com to retrieve the A3 cookie from the response headers.
2. Send a request to https://query2.finance.yahoo.com/v1/test/getcrumb with the A3 cookie in the request headers to fetch the crumb.

Once these credentials are acquired, they can be used in subsequent API requests to retrieve financial data.

const API = 'https://query2.finance.yahoo.com';

async function getCredentials() {
  // Retrieve A3 cookie from fc.yahoo.com
  const { headers } = await fetch('https://fc.yahoo.com');
  const cookie = headers.get('set-cookie');

  // Retrieve crumb from query2.finance.yahoo.com
  const url = new URL('/v1/test/getcrumb', API);
  const request = new Request(url);
  request.headers.set('cookie', cookie);
  const response = await fetch(request);
  const crumb = await response.text();

  return { cookie, crumb };
}

async function quote(symbols, cookie, crumb) {
  // Construct API request URL
  const url = new URL('/v7/finance/quote', API);
  url.searchParams.set('symbols', symbols.join(','));
  url.searchParams.set('crumb', crumb);

  // Send API request
  const request = new Request(url);
  request.headers.set('cookie', cookie);
  const response = await fetch(request);
  const { quoteResponse } = await response.json();

  // Parse and return quote data
  return quoteResponse?.result || [];
}

// Main functionality
const { cookie, crumb } = await getCredentials();
const quotes = await quote(['GOOG', 'TSLA'], cookie, crumb);
if (quotes.length) {
  for (const quote of quotes) {
    console.log(`${quote.symbol} price is ${quote.currency} ${quote.regularMarketPrice}`);
  }
}

Solution 5: using `requests` and `json` packages in Python to get credentials and quote data.

The code below handles a change in the Yahoo Finance V7 API that now requires cookies for requests. It uses the `requests` and `json` libraries to retrieve credentials, including cookies and crumbs, and then uses those credentials to make a request for quote data.

“`python
import requests
import json

apiBase = ‘https://query2.finance.yahoo.com’
headers = {
"User-Agent": "Mozilla/5.0 (Windows NT 6.1; Win64; x64)"
}

def getCredentials(cookieUrl=’https://fc.yahoo.com’, crumbUrl=apiBase+’/v1/test/getcrumb’):
cookie = requests.get(cookieUrl).cookies
crumb = requests.get(url=crumbUrl, cookies=cookie, headers=headers).text
return {‘cookie’: cookie, ‘crumb’: crumb}

def quote(symbols, credentials):
url = apiBase + ‘/v7/finance/quote’
params = {‘symbols’: ‘,’.join(symbols), ‘crumb’: credentials[‘crumb’]}
response = requests.get(url, params=params, cookies=credentials[‘cookie’], headers=headers)
quotes = response.json()[‘quoteResponse’][‘result’]
return quotes

credentials = getCredentials()
quotes = quote([‘GOOG’, ‘TSLA’], credentials)
if quotes:
for quote in quotes:
print(f"{quote[‘symbol’]} price is {quote[‘currency’]} {quote[‘regularMarketPrice’]}")

Video Explanation: